Information processing apparatus, information processing method, and program

Abstract

To provide an information processing device capable of more accurately performing inference of a task using a learned model generated by multitask learning.SOLUTION: The information processing apparatus includes a data acquisition unit configured to acquire data input to a first layer included in a plurality of layers constituting a learned model generated by multitask learning, an inference result acquisition unit configured to acquire an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data to the first layer, a subtask label acquisition unit configured to acquire a subtask label corresponding to the data, a gradient calculation unit configured to calculate a gradient in the data of a function using the inference result of the subtask and the subtask label as input, and an inference unit configured to perform inference of a task using the data, the gradient, and the learned model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A method for training a machine learning model using multi-task learning has been proposed. Multi-task learning is a method for improving accuracy by having one model simultaneously learn multiple tasks related to a target task. For example, Patent Document 1 describes the detection of disease symptoms and signs of aggravation using a trained model constructed using a machine learning algorithm such as multi-task learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-174428 Summary of the Invention [Problem to be solved by the invention]

[0004] When performing inference using a trained model generated by multi-task learning, it is possible to improve the accuracy of task inference by allowing a user to change the inference results of a subtask. The technology described in Patent Document 1 has a problem in that it is not possible to reflect the effects of user changes in the inferences of other subtasks.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and one exemplary purpose thereof is to provide a technology for more accurately inferring tasks using a trained model generated by multi-task learning. [Means for solving the problem]

[0006] An information processing device according to an exemplary aspect of the present disclosure includes: a data acquisition means for acquiring data to be input to a first layer included in a plurality of layers constituting a trained model generated by multi-task learning; an inference result acquisition means for acquiring a subtask inference result output from a second layer that is a layer subsequent to the first layer by inputting the data to the first layer; a subtask label acquisition means for acquiring a subtask label corresponding to the data; a gradient calculation means for calculating the gradient of the data of a function that receives as input the subtask inference result and the subtask label; and an inference means for performing task inference using the data, the gradient, and the trained model.

[0007] An information processing method according to an exemplary aspect of the present disclosure includes: a data acquisition process in which at least one processor acquires data to be input to a first layer included in a plurality of layers constituting a trained model generated by multi-task learning; an inference result acquisition process in which the at least one processor inputs the data to the first layer and acquires an inference result of a subtask that is output from a second layer that is a layer subsequent to the first layer; a subtask label acquisition process in which the at least one processor acquires a subtask label corresponding to the data; a gradient calculation process in which the at least one processor calculates the gradient of the data of a function that receives as input the inference result of the subtask and the subtask label; and an inference process in which the at least one processor performs task inference using the data, the gradient, and the trained model.

[0008] An information processing program according to an exemplary aspect of the present disclosure is an information processing program for causing a computer to function as an information processing device, and causes the computer to function as: a data acquisition means for acquiring data input to a first layer included in multiple layers constituting a trained model generated by multi-task learning; an inference result acquisition means for acquiring a subtask inference result output from a second layer that is a layer subsequent to the first layer by inputting the data to the first layer; a subtask label acquisition means for acquiring a subtask label corresponding to the data; a gradient calculation means for calculating the gradient of the data of a function that receives as input the subtask inference result and the subtask label; and an inference means for performing task inference using the data, the gradient, and the trained model. [Effects of the Invention]

[0009] According to an exemplary aspect of the present disclosure, an exemplary effect is provided in that a technology can be provided that performs task inference with higher accuracy using a trained model generated by multi-task learning. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 3] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 4] FIG. 1 is a diagram illustrating an example of the configuration of a trained model according to the present disclosure. [Figure 5] FIG. 1 is a flow diagram illustrating an example of the flow of an information processing method according to the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating an example of a flow of task inference using an information processing device according to the present disclosure. [Figure 7] FIG. 10 is a diagram illustrating an example of change information output by an output control unit according to the present disclosure. [Figure 8]FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technologies (part or all of the products or methods) employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] First Exemplary Embodiment A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0013] (Configuration of information processing device) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes a data acquisition unit 11, an inference result acquisition unit 12, a subtask label acquisition unit 13, a gradient calculation unit 14, and an inference unit 15.

[0014] The data acquisition unit 11 acquires data to be input to a first layer included in multiple layers constituting a trained model generated by multi-task learning. The inference result acquisition unit 12 inputs data to the first layer and acquires subtask inference results output from a second layer, which is a layer subsequent to the first layer. The subtask label acquisition unit 13 acquires subtask labels corresponding to the data. The gradient calculation unit 14 calculates the gradient of a function that takes the subtask inference results and the subtask labels as inputs. One example of the function is, but is not limited to, a function that represents the error between the subtask inference results and the subtask labels. Another example of the function may be a function that represents the correlation between the subtask inference results and the subtask labels. The inference unit 15 performs task inference using the data, gradient, and trained model.

[0015] (Effects of information processing devices) As described above, information processing device 1 is configured to include data acquisition unit 11 that acquires data to be input to a first layer included in multiple layers that make up a trained model generated by multi-task learning, inference result acquisition unit 12 that acquires subtask inference results output from a second layer that is a layer subsequent to the first layer by inputting the data to the first layer, subtask label acquisition unit 13 that acquires subtask labels corresponding to the data, gradient calculation unit 14 that calculates the gradient of the data of a function that receives as input the subtask inference results and the subtask labels, and inference unit 15 that performs task inference using the data, the gradient, and the trained model. Therefore, information processing device 1 has the advantage of being able to more accurately perform task inference using a trained model generated by multi-task learning.

[0016] (Flow of information processing method) The flow of information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of information processing method S1. As shown in Fig. 2, information processing method S1 includes data acquisition processing S11, inference result acquisition processing S12, subtask label acquisition processing S13, gradient calculation processing S14, and inference processing S15.

[0017] In data acquisition processing S11, at least one processor acquires data to be input to a first layer included in multiple layers constituting a trained model generated by multi-task learning. In inference result acquisition processing S12, at least one processor inputs the data to the first layer and acquires the inference result of a subtask output from a second layer, which is a layer subsequent to the first layer. In subtask label acquisition processing S13, at least one processor acquires a subtask label corresponding to the data. In gradient calculation processing S14, at least one processor calculates the gradient of the data of a function that takes the inference result of the subtask and the subtask label as input. In inference processing S15, at least one processor performs task inference using the data, the gradient, and the trained model.

[0018] (Effects of information processing methods) As described above, the information processing method S1 includes a data acquisition process S11 in which at least one processor acquires data to be input to a first layer included in multiple layers constituting a trained model generated by multi-task learning, an inference result acquisition process S12 in which at least one processor inputs the data to the first layer and acquires a subtask inference result output from a second layer that is a layer subsequent to the first layer, a subtask label acquisition process S13 in which the at least one processor acquires a subtask label corresponding to the data, a gradient calculation process S14 in which the at least one processor calculates the gradient of a function that receives the subtask inference result and the subtask label as input, and an inference process S15 in which the at least one processor performs task inference using the data, the gradient, and the trained model. Therefore, the information processing method S1 has the advantage of enabling more accurate task inference using a trained model generated by multi-task learning.

[0019] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0020] (Configuration of information processing device) The configuration of the information processing device 1A will be described with reference to FIG. 3. The information processing device 1A is a device that performs inference on a task using a trained model generated by multitask learning. An example of the task is an image recognition task. More specifically, the information processing device 1A is used, for example, for decision-making support such as diagnosis using medical images.

[0021] 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A includes a control unit 10A, a storage unit 20A, a communication unit 30A, an input unit 40A, and an output unit 50A.

[0022] (Communications Department) The communication unit 30A communicates with devices external to the information processing device 1A via a communication line. While the specific configuration of the communication line does not limit the present exemplary embodiment, examples of the communication line include a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination thereof. The communication unit 30A transmits data supplied from the control unit 10A to other devices, and supplies data received from other devices to the control unit 10A.

[0023] (Input section) The input unit 40A is configured to receive input to the information processing device 1A, and includes, for example, input devices such as a keyboard, a mouse, a touch panel, a camera, a microphone, etc. The input unit 40A may also be configured to receive data from the input devices via an interface such as a USB (Universal Serial Bus).

[0024] (output section) The output unit 50A is a component for performing output from the information processing device 1A, and includes, for example, output devices such as a display, a printer, a touch panel, a speaker, etc. The output unit 50A may also be configured to include, for example, an interface such as a USB, and to output data to the output device via the interface.

[0025] (Storage part) The storage unit 20A stores various types of data referenced by the control unit 10A. In particular, the storage unit 20A stores instructions of a computer program executed by the control unit 10A. Examples of data stored in the storage unit 20A include a trained model 21, first data f, and a subtask inference result y. Here, storing the trained model 21 in the storage unit 20A means that parameters defining the trained model 21 are stored in the storage unit 20A. The first data f and the subtask inference result y are data referenced by the gradient calculation unit 14A, which will be described later. Details of the first data f and the subtask inference result y will be described later.

[0026] (Pre-trained model) The trained model 21 is a machine learning model generated by multitask learning. The trained model 21 has multiple layers. The multiple layers include, for example, an input layer, multiple intermediate layers, and an output layer. The input data input to the trained model 21 is, for example, text data, audio data, image data, or a combination thereof. More specifically, the input data is, for example, data representing medical images. Examples of medical images include, but are not limited to, X-ray images, endoscopic images, pathological images, MRI images, and CT images.

[0027] The output of the trained model 21 is the inference result of the main task, and as an example, a discrimination result regarding a medical image. The discrimination result includes, as an example, data indicating the name of a disease and symptoms, but is not limited to these. The trained model 21 may also output multiple findings regarding the medical image as the inference results of the subtask. Examples of the inference results of the subtask include, but are not limited to, shape findings (e.g., "circular"), position findings (e.g., coordinate information), etc.

[0028] (Control unit) The control unit 10A includes a data acquisition unit 11A, a model unit 12A, a subtask label acquisition unit 13A, a gradient calculation unit 14A, a re-inference unit 15A, an output control unit 16A, a reception unit 17A, and a re-update unit 18A. Each unit of the control unit 10A is realized by the control unit 10A reading and executing instructions of a computer program stored in the memory unit 20A. The model unit 12A is an example of a data acquisition means and an inference result acquisition means according to the present disclosure. The subtask label acquisition unit 13A is an example of a subtask label acquisition means according to the present disclosure. The gradient calculation unit 14A is an example of a gradient calculation means according to the present disclosure. The re-inference unit 15A is an example of an inference means according to the present disclosure. The output control unit 16A is an example of an output control means according to the present disclosure. The reception unit 17A is an example of a reception means according to the present disclosure. The re-update unit 18A is an example of a re-update means according to the present disclosure.

[0029] (Data acquisition section) The data acquisition unit 11A acquires input data to be input to the trained model 21. As an example, the data acquisition unit 11A may acquire the input data by receiving the input data from another device connected via the communication unit 30A. Alternatively, the data acquisition unit 11A may acquire the input data input to the input unit 40A. Alternatively, the data acquisition unit 11A may acquire the input data by reading the input data from a storage destination specified by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device external to the information processing device 1A).

[0030] (Model section) The model unit 12A performs multitask inference by inputting input data to the trained model 21. The model unit 12A also acquires first data f to be input to a first layer included in the multiple layers constituting the trained model 21, and acquires a subtask inference result y output from a second layer by inputting the first data f to the first layer. The first data f may be input data to be input to the input layer of the trained model 21, or may be intermediate features extracted from the input data. In other words, the first data f may include at least one of the input data of the trained model 21 and features extracted from the input data.

[0031] Fig. 4 is a diagram showing an example of the configuration of the trained model 21 to which the model unit 12A receives input data. In the example of Fig. 4, the trained model 21 includes an encoder 211, a first linear layer 212, and a second linear layer 213. Each of the encoder 211, the first linear layer 212, and the second linear layer 213 includes multiple layers.

[0032] The model unit 12A inputs input data to the encoder 211. The encoder 211 outputs first data f, which is an intermediate feature. In the example of FIG. 4, the first data f is, for example, an n-dimensional (n is a natural number) vector. The model unit 12A also inputs the first data f, which is an intermediate feature, to the first linear layer 212, which outputs a subtask inference result y, which is an inference result of the subtask. The subtask inference result y may be, for example, binary data indicating the inference result, or may be a real value indicating the inference result.

[0033] Furthermore, the model unit 12A inputs data obtained by combining the first data f, which is an intermediate feature, and the subtask inference result y to the second linear layer 213. The second linear layer 213 outputs the inference result y0 of the task. The inference result y0 may be, for example, binary data indicating the inference result, or may be a real value indicating the inference result.

[0034] (Subtask label acquisition part) The subtask label acquisition unit 13A acquires the subtask label~ Get the y subtask label ~ y is a subtask label inferred by the trained model 21, and is, for example, a class classification label or a bounding box, but is not limited to these. More specifically, the subtask label ~ For example, y may be a label indicating an inference result of a medical condition related to a medical image. Here, for example, the inference result may include findings that can be diagnosed from the image (findings on the shape of tissue, findings on anatomical position, etc.). ~ The number of y may be 1 or more. ~ The expression "y" stands for "y with a tilde."

[0035] As an example, the subtask label acquisition unit 13A acquires a subtask label from another device (such as a user terminal) connected via the communication unit 30A. ~ By receiving y, the subtask label ~ y may be acquired. In addition, the subtask label acquisition unit 13A may acquire the subtask label y input to the input unit 40A. ~ y from a storage location designated by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A). ~ Subtask label by reading y ~ You may also get y.

[0036] Subtask Label ~ For example, y is data input by a user of the information processing device 1A using an input device such as a mouse or a keyboard. An example of a user of the information processing device 1A is a medical professional. ~ When y is input, the subtask label acquisition unit 13A, for example, displays a pull-down list for selecting a subclass on the display device, and the subclass label selected by the user from the pull-down list is ~In addition, the subtask label acquiring unit 13A may, for example, display a seek bar on the display device for specifying a reliability score, and acquire the reliability score input by the user by operating the seek bar as the subtask label y. ~ y. Alternatively, the subtask label acquisition unit 13A may display a screen for adjusting the bounding box on the display device, and acquire the bounding box corrected by the user using the input device as the subtask label y. ~ It may also be obtained as y.

[0037] In addition, as an example, the subtask label acquisition unit 13A acquires the subtask label from a predetermined database. ~ Subtask label by reading y ~ More specifically, for example, if the subtask is a genotype inference task, the subtask label acquisition unit 13A may acquire the genetic test results from a blood test from an electronic medical record. Also, the subtask label acquisition unit 13A may acquire information such as age, sex, and race from a predetermined database as a subtask label. ~ It may also be obtained as y.

[0038] (Gradient calculation part) The gradient calculation unit 14A calculates the subtask inference result y and the subtask label ~ Gradient ∇ of the first data f of the error with y f L y where the gradient ∇ f L y is the error function L y is differentiated by the first data f.

[0039] (Re-reasoning part) The re-inference unit 15A receives the first data f and the gradient ∇ f L y and the trained model 21 to perform inference of the task. f L y The first data f is updated using the above, and the updated data obtained by the update is ~ Inference is performed using f and the trained model 21. Updated data~ For example, f is expressed by the following equation: In the following equation, λ is a scalar constant. ~ f=f+λ∇ f L y

[0040] In other words, the re-reasoning unit 15A calculates the gradient ∇ f L y Backpropagate to the first data f and find the gradient ∇ f L y The re-inference unit 15A further updates the first data f using the updated data ~ From f, other subtasks are inferred again by forward propagation. More specifically, in the multitask inference, the re-inference unit 15A, for example, ~ f is input to the third layer included in the plurality of layers that make up the trained model 21. In the example of FIG. 4, the re-inference unit 15A inputs the updated data ~ f and subtask inference results ~ The combined data with y is input to the second linear layer 213.

[0041] (Output control section) The output control unit 16A outputs information indicating the multitasking inference result. The output control unit 16A also outputs update information indicating the content of the update by the re-inference unit 15A. As an example, the output control unit 16A may output the information by writing it to a storage destination (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A) designated by the user of the information processing device 1A. The output control unit 16A may also transmit the information via the communication unit 30A, or may output the information to an output device such as a display.

[0042] The information indicating the multitasking inference result output by the output control unit 16A may be, for example, information indicating the inference result of the pathology related to the medical image. More specifically, the information indicating the inference result may be, for example, the malignancy level (e.g., "0.3", "0.8", etc.).

[0043] The update information output by the output control unit 16A is, for example, post-update data ~ f, and the updated data ~ The update information may be information representing the difference between the updated data f and the first data f before the update. ~ Information indicating how much f has changed from the data before the update, or the updated data ~ It may also be information indicating how f has changed from the data before the update. ~ If f is text, the output control unit 16A may use, for example, an attention map to visualize and output words related to the change. ~ When f is image data, the output control unit 16A may display the data before the update and the data after the update side by side on the display device, or may display an image showing the difference between the data before the update and the data after the update on the display device.

[0044] (Reception Department) The reception unit 17A receives the updated data ~ The receiving unit 17A receives, as an example, change instruction information that instructs a change to f. The receiving unit 17A receives, as an example, change instruction information input to the input unit 40A by a user operation or the like. The change instruction information is, as an example, information by which the user specifies a part that should not be changed in the update information output by the output control unit 16A. For example, the post-update data ~ If a part of f that is not related to the subtask has changed, the user performs an operation to prevent that part from changing, and the receiving unit 17A receives change instruction information based on the user operation.

[0045] (Re-update section) The re-updating unit 18A uses the change instruction information received by the receiving unit 17A to update the updated data. ~ The re-update unit 18A updates the updated data f by the re-inference unit 15A, for example. ~ At step f, the re-inference unit 15A performs processing such as restoring the part indicated by the change instruction information to the original state. ~ Re-updated data obtained by updating f~~ Multitask inference is performed using f.

[0046] (Flow of reasoning method) 5 is a flow diagram showing an example of the flow of an information processing method according to the present disclosure. In step S101, the data acquisition unit 11A acquires input data. In step S102, the model unit 12A calculates first data f, which is a feature, and performs multi-task inference based on the input data. Also in step S102, the output control unit 16A outputs a subtask inference result y.

[0047] In step S103, the subtask label acquisition unit 13A acquires some of the subtask labels ~ y from outside. The number of subtask labels acquired by subtask label acquisition unit 13A may be one or more. In step S104, gradient calculation unit 14A calculates the subtask label acquired by subtask label acquisition unit 13A. ~ From the error between y and the subtask inference result y by the model unit 12A, the gradient ∇ f L y Furthermore, the re-inference unit 15A calculates the gradient ∇ f L y The first data f is updated using the above formula.

[0048] In step S105, the output control unit 16A outputs update information indicating the content of the update by the re-inference unit 15A. The user of the information processing device 1A checks the output update information and checks whether or not any part that should not be changed has changed. If any part that should not be changed has changed, the updated data ~ When the user desires to modify f, the user inputs change instruction information using an input device or the like.

[0049] In step S106, the reception unit 17A receives the change instruction information. In step S107, the re-update unit 18A updates the updated data obtained by the update of the re-inference unit 15A based on the change instruction information received by the reception unit 17A. ~In step S108, the re-inference unit 15A corrects the re-updated data f obtained by the correction by the re-updating unit 18A. ~~ Multitask inference is performed again using f and the trained model 21. In the example of FIG. 4, the re-inference unit 15A uses the re-updated data ~~ Inference is performed by inputting f to the second linear layer 213.

[0050] Fig. 6 is a diagram showing an example of the flow of task inference using information processing device 1A. In Fig. 6, screen SC11 is a screen displayed on, for example, a terminal carried by user U1 of information processing device 1A, and includes subtask inference result y output by output control unit 16A. In screen SC11, "shape findings: circular," "position findings: {coordinates}," and "malignancy level: 0.3" each indicate subtask inference result y.

[0051] A user U1 such as a doctor checks the screen displayed on the screen SC11 and selects the subtask label ~ In this way, the user U1 provides feedback on the findings (subtask inference result y) output by the trained model 21, and the trained model 21 is asked to make a malignant diagnosis (subtask 0) again. The re-inference unit 15A uses the input subtask label ~ The gradient ∇ calculated using y f L y Run task inference again using

[0052] Screen SC12 shows the result of the re-inference. Screen SC12 includes the subtask inference results such as "shape findings: ...", "position findings: {coordinates}", and "malignancy level: 0.8". Gradient ∇ f L y By using the above, the result of the re-inference differs from the subtask inference result included in the screen SC11. In this way, the information processing device 1A can reflect the feedback of the user U1 in the inference of the trained model 21. The user U1 checks the result of the re-inference and makes a final decision on the treatment policy, etc.

[0053] (Examples of change information) 7 is a diagram showing an example of change information output by the output control unit 16A. In FIG. 7, an image Img11 is an image represented by the first data f before updating, and an image Img12 is an image represented by the first data f after updating. ~ Image Img13 is an image representing the first data f and the updated data ~ These are images that show the difference from f. By checking these images, users can understand how the data has changed due to their feedback.

[0054] (Effects of information processing devices) As described above, in the information processing device 1A, the re-inference unit 15A calculates the gradient ∇ f L y The first data f is updated using ~ The system employs a configuration in which task inference is performed using f and the trained model 21. Subtask labels input by the user ~ When task inference is performed using y, the effect of the user's change can be reflected in the task inference, but the effect is limited. In contrast, according to the information processing device 1A, the subtask inference result y is reflected in the subtask label ~ In addition to replacing it with y, the first data f is updated ~ By substituting f for f and using it in the inference of other subtasks, the feedback from the user on a certain subtask can be effectively reflected in the other subtasks. In this way, according to the information processing device 1A, when the user changes the inference result of an arbitrary subtask, the effect of the change can be reflected in the inference results of the other subtasks.

[0055] In the information processing device 1A, the re-inference unit 15A uses the updated data in the inference of the task. ~ The structure adopted is that f is input to a layer (third layer) different from the first layer. ~When task inference is performed using y, the effect of the user's change can be reflected in the task inference, but the effect is limited. In contrast, according to the information processing device 1A, the subtask inference result y is reflected in the subtask label ~ In addition to replacing it with y, the first data f is updated ~ By substituting f and inputting it into the second linear layer 213 (third layer), the user's feedback on a subtask can be effectively reflected in other subtasks.

[0056] Furthermore, the information processing device 1A employs a configuration in which the first data f includes at least one of input data of the trained model 21 and features extracted from the input data. Rather than simply changing the inference result of the subtask, at least one of the input data of the trained model 21 and features extracted from the input data is changed based on user feedback, and the changed data is used to perform task inference, thereby making it possible to effectively reflect the results of user feedback in the task inference.

[0057] The information processing device 1A also has an output control unit 16A that outputs information indicating the content of the update by the re-inference unit 15A. For example, by checking the output information, a user of the information processing device 1A can understand how much the feature recognized by the trained model 21 has changed and how the feature has changed.

[0058] In addition, in the information processing device 1A, the updated data ~ a receiving unit 17A that receives change instruction information that instructs a change to f; ~ The re-inference unit 15A further includes a re-updating unit 18A that re-updates f. ~~The user of the information processing device 1A checks the information output by the output control unit 16A, and if a part that is completely unrelated to the subtask has changed, for example, the user specifies that the part should not be changed. Based on this operation, the re-update unit 18A updates the updated data ~ Update f and re-update data ~~ By using f to perform task inference, task inference can be performed that effectively reflects user feedback.

[0059] Furthermore, the information processing device 1A employs a configuration in which the input data of the trained model 21 is data representing medical images. Therefore, the information processing device 1A has the effect of enabling task inference using medical images as input to be performed with higher accuracy.

[0060] Furthermore, the information processing device 1A employs a configuration in which the subtask label is a label indicating the inference result of the medical condition related to the medical image, thereby achieving the effect of more accurately inferring the medical condition from the medical image.

[0061] <Modification> The functions of the information processing device 1 and information processing device 1A described above may be shared and implemented by multiple devices. For example, the information processing device 1 or information processing device 1A described above may be realized as an inference system in which two or more devices are connected via a communication network. In this case, the inference system may be, for example, a system including a first device including a data acquisition unit 11A, a subtask label acquisition unit 13A, an output control unit 16A, and a reception unit 17A, and a second device including a model unit 12A, a gradient calculation unit 14A, a re-inference unit 15A, and a re-update unit 18A. In this case, the functions of the information processing device 1A are realized by the cooperation of the first device and the second device.

[0062] Furthermore, in the above-described information processing device 1A, the case where the trained model 21 is stored in the storage unit 20A of the information processing device 1A has been described, but the trained model 21 may be stored in a device other than the information processing device 1A. In this case, the information processing device 1A transmits input data to the device in which the trained model 21 is stored, and receives the inference results of the subtasks and the inference results of the tasks transmitted from the device in response to the transmitted input data.

[0063] In addition, in the above-described information processing device 1A, the first data ~ f and the subtask label ~ The data combined with y was input to the second linear layer 213 (an example of the third layer). ~ f and the subtask label ~ y. For example, the first data obtained by updating ~ Alternatively, f may be input to the second linear layer 213 as is.

[0064] [Software implementation example] Some or all of the functions of the information processing devices 1, 1A (hereinafter also referred to as "each of the above devices") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0065] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 8. Figure 8 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0066] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0067] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0068] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0069] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0070] Furthermore, the functions of each of the above devices may be realized by a single processor provided in a single computer, by multiple processors provided in a single computer working in cooperation, or by multiple processors provided in each of multiple computers working in cooperation. Furthermore, the programs for causing each of the above devices to realize the functions may be stored in a single memory provided in a single computer, or may be distributed and stored in multiple memories provided in a single computer, or may be distributed and stored in multiple memories provided in each of multiple computers.

[0071] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0072] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims. (Appendix A1) a data acquisition means for acquiring data to be input to a first layer included in a plurality of layers constituting a trained model generated by multi-task learning; an inference result acquisition means for acquiring an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data into the first layer; a subtask label acquisition means for acquiring a subtask label corresponding to the data; gradient calculation means for calculating the gradient of the data of a function that receives the inference result of the subtask and the subtask label as input; an inference means for performing inference on a task using the data, the gradient, and the trained model; An information processing device comprising:

[0073] (Appendix A2) The inference means updates the data using the gradient, and performs the inference using the updated data and the trained model. 10. The information processing device according to claim 1,

[0074] (Appendix A3) the inference means inputs the updated data into a third layer included in the plurality of layers in the inference; 10. The information processing device according to claim 9, wherein the information processing device is a

[0075] (Appendix A4) The data includes at least one of input data of the trained model and features extracted from the input data. An information processing device according to any one of appendices A1 to A3.

[0076] (Appendix A5) an output control means for outputting information indicating the content of the update performed by the inference means; The information processing device according to claim A2 or A3, further comprising:

[0077] (Appendix A6) a receiving means for receiving change instruction information instructing a change to the data updated by the inference means; a re-updating unit that re-updates the data updated by the inference unit using the change instruction information, the inference means performs the inference using the data updated by the re-updating means. 10. The information processing device according to claim 9, wherein the information processing device is a

[0078] (Appendix A7) The input data of the trained model is data representing a medical image. An information processing device according to any one of appendices A1 to A6.

[0079] (Appendix A8) The subtask label is a label indicating an inference result of a pathology related to the medical image. 10. The information processing device according to claim 7,

[0080] [Appendix B] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0081] (Appendix B1) a data acquisition process in which at least one processor acquires data to be input to a first layer included in multiple layers constituting the trained model generated by multi-task learning; an inference result acquisition process in which the at least one processor inputs the data into the first layer and acquires an inference result of the subtask that is output from a second layer that is a layer subsequent to the first layer; a subtask label acquisition process in which the at least one processor acquires a subtask label corresponding to the data; a gradient calculation process in which the at least one processor calculates a gradient in the data of a function that receives as input the inference result of the subtask and the subtask label; an inference process in which the at least one processor performs inference on a task using the data, the gradients, and the trained model; An information processing method including:

[0082] (Appendix B2) In the inference process, the at least one processor updates the data using the gradient, and performs the inference using the updated data and the trained model. 1. The information processing method described in Appendix B1.

[0083] (Appendix B3) In the inference process, the at least one processor inputs the updated data into a third layer included in the plurality of layers. 1. The information processing method described in Appendix B2.

[0084] (Appendix B4) The data includes at least one of input data of the trained model and features extracted from the input data. 10. The information processing method according to any one of appendices B1 to B3.

[0085] (Appendix B5) an output control process in which the at least one processor outputs information indicating the content of updates in the inference process; The information processing method according to Appendix B2 or B3, further comprising:

[0086] (Appendix B6) a receiving process in which the at least one processor receives change instruction information instructing a change to the data updated in the inference process; a re-updating process in which the at least one processor re-updates the data updated in the inference process using the change instruction information; In the inference process, the at least one processor performs the inference using data updated in the re-update process. The information processing method described in Appendix B5.

[0087] (Appendix B7) The input data of the trained model is data representing a medical image. 10. The information processing method according to any one of Appendix B1 to B6.

[0088] (Appendix B8) The subtask label is a label indicating an inference result of a pathology related to the medical image. An information processing method as described in Appendix B7.

[0089] [Appendix C] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0090] (Appendix C1) A program that causes a computer to function as an information processing device, The computer a data acquisition means for acquiring data to be input to a first layer included in a plurality of layers constituting a trained model generated by multi-task learning; an inference result acquisition means for acquiring an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data into the first layer; a subtask label acquisition means for acquiring a subtask label corresponding to the data; gradient calculation means for calculating the gradient of the data of a function that receives the inference result of the subtask and the subtask label as input; an inference means for performing inference on a task using the data, the gradient, and the trained model; An information processing program that functions as a

[0091] (Appendix C2) The inference means updates the data using the gradient, and performs the inference using the updated data and the trained model. An information processing program as described in Appendix C1.

[0092] (Appendix C3) the inference means inputs the updated data into a third layer included in the plurality of layers in the inference; An information processing program as described in Appendix C2.

[0093] (Appendix C4) The data includes at least one of input data of the trained model and features extracted from the input data. An information processing program according to any one of appendices C1 to C3.

[0094] (Appendix C5) an output control means for outputting information indicating the content of the update performed by the inference means; The information processing program according to claim C2 or C3, further comprising:

[0095] (Appendix C6) a receiving means for receiving change instruction information instructing a change to the data updated by the inference means; The computer a re-updating means for re-updating the data updated by the inference means using the change instruction information, the inference means performs the inference using the data updated by the re-updating means. An information processing program as described in Appendix C5.

[0096] (Appendix C7) The input data of the trained model is data representing a medical image. 10. An information processing program according to any one of Appendices C to C6.

[0097] (Appendix C8) The subtask label is a label indicating an inference result of a pathology related to the medical image. An information processing program as described in Appendix C7.

[0098] [Appendix D] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0099] (Appendix D1) at least one processor, a data acquisition process for acquiring data to be input to a first layer included in multiple layers constituting the trained model generated by multi-task learning; an inference result acquisition process for acquiring an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data into the first layer; a subtask label acquisition process for acquiring a subtask label corresponding to the data; a gradient calculation process for calculating a gradient of the data of a function that receives the inference result of the subtask and the subtask label as input; an inference process for performing inference on a task using the data, the gradient, and the trained model; An information processing device that executes the above.

[0100] The information processing device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes.

[0101] (Appendix D2) In the inference process, the at least one processor updates the data using the gradient, and performs the inference using the updated data and the trained model. 10. The information processing device according to claim 9, wherein the information processing device is an information processing device according to claim 1.

[0102] (Appendix D3) In the inference process, the at least one processor inputs the updated data into a third layer included in the plurality of layers. 10. The information processing device according to claim 9, wherein the information processing device is a

[0103] (Appendix D4) The data includes at least one of input data of the trained model and features extracted from the input data. An information processing device according to any one of appendices D1 to D3.

[0104] (Appendix D5) an output control process for outputting information indicating the content of updates in the inference process; The information processing device according to claim D2 or D3, further comprising:

[0105] (Appendix D6) a receiving process for receiving change instruction information instructing a change to the data updated in the inference process; the at least one processor: and further executing a re-update process for re-updating the data updated in the inference process using the change instruction information. In the inference process, the at least one processor performs the inference using data updated in the re-update process. 10. The information processing device according to claim 9, wherein said information processing device is a

[0106] (Appendix D7) The input data of the trained model is data representing a medical image. An information processing device according to any one of appendices D1 to D6.

[0107] (Appendix D8) The subtask label is a label indicating an inference result of a pathology related to the medical image. 10. The information processing device according to claim 7,

[0108] [Appendix E] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0109] (Appendix E1) A program that causes a computer to function as an information processing device, The computer, a data acquisition process for acquiring data to be input to a first layer included in multiple layers constituting the trained model generated by multi-task learning; an inference result acquisition process for acquiring an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data into the first layer; a subtask label acquisition process for acquiring a subtask label corresponding to the data; a gradient calculation process for calculating a gradient of the data of a function that receives the inference result of the subtask and the subtask label as input; an inference process for performing inference on a task using the data, the gradient, and the trained model; A non-transitory recording medium on which an information processing program for executing the above is recorded. [Explanation of symbols]

[0110] 1, 1A Information processing equipment 11, 11A Data acquisition section 12 Inference result acquisition part 12A Model Section 13, 13A Subtask label acquisition part 14, 14A Gradient calculation section 15 Reasoning part 15A Re-reasoning part 16A Output control section 17A Reception 18A Re-update section 21 Pre-trained models

Claims

1. a data acquisition means for acquiring data to be input to a first layer included in a plurality of layers constituting a trained model generated by multi-task training; an inference result acquisition means for acquiring an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data into the first layer; a subtask label acquisition means for acquiring a subtask label corresponding to the data; gradient calculation means for calculating the gradient of the data of a function that receives the inference result of the subtask and the subtask label as input; an inference means for performing inference on a task using the data, the gradient, and the trained model; An information processing device comprising:

2. The inference means updates the data using the gradient, and performs the inference using the updated data and the trained model. The information processing device according to claim 1 .

3. the inference means inputs the updated data into a third layer included in the plurality of layers in the inference; The information processing device according to claim 2 .

4. The data includes at least one of input data of the trained model and features extracted from the input data.

3. The information processing device according to claim 1 or 2.

5. an output control means for outputting information indicating the content of the update performed by the inference means; The information processing device according to claim 2 or 3, further comprising:

6. a receiving means for receiving change instruction information instructing a change to the data updated by the inference means; a re-updating unit that re-updates the data updated by the inference unit using the change instruction information, the inference means performs the inference using the data updated by the re-updating means. The information processing device according to claim 5 .

7. The input data of the trained model is data representing a medical image.

3. The information processing device according to claim 1 or 2.

8. The subtask label is a label indicating an inference result of a pathology related to the medical image. The information processing device according to claim 7 .

9. a data acquisition process in which at least one processor acquires data to be input to a first layer included in multiple layers constituting the trained model generated by multi-task learning; an inference result acquisition process in which the at least one processor inputs the data into the first layer and acquires an inference result of the subtask that is output from a second layer that is a layer subsequent to the first layer; a subtask label acquisition process in which the at least one processor acquires a subtask label corresponding to the data; a gradient calculation process in which the at least one processor calculates a gradient in the data of a function that receives as input the inference result of the subtask and the subtask label; an inference process in which the at least one processor performs inference on a task using the data, the gradients, and the trained model; An information processing method including:

10. An information processing program for causing a computer to function as an information processing device, the computer comprising: a data acquisition means for acquiring data to be input to a first layer included in a plurality of layers constituting a trained model generated by multi-task training; an inference result acquisition means for acquiring an inference result of a subtask output from a second layer that is a layer subsequent to the first layer by inputting the data into the first layer; a subtask label acquisition means for acquiring a subtask label corresponding to the data; gradient calculation means for calculating the gradient of the data of a function that receives the inference result of the subtask and the subtask label as input; an inference means for performing inference on a task using the data, the gradient, and the trained model; A program to function as a

Images