A device for generating data merging rules for machine learning models, a method and program for operating the data merging rule generator, and a machine learning model training device, a method and program for operating the training device.

Abstract

A device for generating a data merging rule for a machine learning model, said device for generating a data merging rule comprising a processor and a memory connected to or embedded in the processor, wherein the processor executes an identification process for identifying a combination of feature vectors that are included in a data set having correct answer labels and that can be merged, and a rule generation process for generating a merging rule for the feature vectors on the basis of the combinations of feature vectors that can be merged.

Description

[Technical Field]

[0001] This disclosure relates to a device for generating data merging rules for machine learning models, a method and program for operating the data merging rule generator, and a machine learning model training device, a method and program for operating the training device. [Background technology]

[0002] In the medical field, machine learning models have been developed to predict patient prognosis based on patient medical data. For example, Japanese Patent Publication No. 2020-529057 discloses a machine learning model that predicts medical events from patient medical data including symptoms, medications, test results, diagnosis, and vital signs. [Overview of the project] [Problems that the invention aims to solve]

[0003] Let's consider patient symptoms as an example of information included in patient medical data. Typically, the symptom field in medical data contains textual information such as "cough," "headache," or "fever," entered by the doctor. Such textual information is input into a machine learning model as a feature vector using one-hot encoding. A one-hot encoding feature vector is a vector in which only one component is 1 and all other components are 0, such as (1,0,0).

[0004] If we attempt to convert textual information into one-hot encoding feature vectors by focusing solely on differences in notation, a large number of feature vectors with identical or similar meanings will be generated. For example, if there are variations in notation for patient symptoms entered by doctors, such as "cough" and "sigh," or "high fever" and "fever," these will be represented as different feature vectors. Inputting such feature vectors, which are essentially identical or similar in meaning, directly into a machine learning model often fails to yield sufficient prediction accuracy.

[0005] Furthermore, regarding patient age, for example, creating feature vectors by grouping them into categories such as "20s" rather than creating feature vectors for each year age can be expected to improve prediction accuracy. However, in this case, the granularity of the grouping becomes larger, and if the grouping is done at an excessively large granularity, the prediction accuracy will decrease.

[0006] Traditionally, the dimensionality of feature vectors input to machine learning models was reduced by manually merging feature vectors that were essentially identical or similar in meaning. However, manually merging feature vectors is extremely time-consuming and does not guarantee improved prediction accuracy.

[0007] This disclosure provides a data merging rule generator for machine learning models and a machine learning model training device that can improve the prediction accuracy of machine learning models by merging mergible feature vectors contained in the input data to reduce the number of dimensions, compared to cases where the number of dimensions is not reduced by merging feature vectors. do. [Means for solving the problem]

[0008] A first aspect of the present disclosure is a data merging rule generator for a machine learning model, comprising a processor and memory connected to or embedded in the processor, wherein the processor performs an identification process to identify combinations of mergible feature vectors included in a data set having ground truth labels, and a rule generation process to generate merging rules for feature vectors based on the combinations of mergible feature vectors.

[0009] A second aspect of this disclosure is that, in the first aspect described above, in a specific processing, the processor may create a frequency distribution of the correct label for each feature vector included in the data set, and identify combinations of feature vectors whose similarity of the frequency distributions of the correct labels is greater than or equal to a predetermined first threshold as combinations of feature vectors that can be merged.

[0010] A third aspect of this disclosure is that, in the second aspect described above, in a specific process, the processor may further create a frequency distribution that takes into account the combination of items for a combination identified as a combination of mergeable feature vectors, and exclude a combination from the combination of mergeable feature vectors if the similarity of the frequency distributions that take into account the combination of multiple items is less than a predetermined second threshold.

[0011] A fourth aspect of this disclosure is that, in the first aspect described above, in a specific processing, the processor may create a frequency distribution of the correct level for each feature vector included in the data set, taking into account combinations of multiple items, and identify combinations of feature vectors whose similarity of the frequency distribution of the correct labels is equal to or greater than a predetermined seventh threshold as combinations of feature vectors that can be merged.

[0012] A fifth aspect of this disclosure is that, in any one of the first to fourth aspects described above, in the rule generation process, the processor may terminate the generation of merged rules when the number of combinations of mergeable feature vectors included in the merged rules exceeds a predetermined third threshold.

[0013] A sixth aspect of this disclosure is that, in the first aspect described above, in a specific process, the processor may generate and train a provisional model that takes feature vectors included in the data set as input, select a combination of feature vectors from the data set, and identify the selected combination of feature vectors as a combination of feature vectors that can be merged if the change in the prediction result of the provisional model when the selected combination of feature vectors is swapped is less than a predetermined fourth threshold.

[0014] A seventh aspect of this disclosure is that, in the first aspect described above, in a specific process, the processor may generate and train a provisional model that takes feature vectors included in the data set as input, select a combination of feature vectors from the data set, and if the similarity of the prediction results of the provisional model when the selected combination of feature vectors is swapped is greater than or equal to a predetermined fourth similarity, the selected combination of feature vectors may be identified as a combination of feature vectors that can be merged.

[0015] An eighth aspect of the present disclosure, in any one aspect of the first to seventh aspects described above, in a specific process, candidate feature vectors that can be merged may be determined based on at least one of the feature vector's edit distance, distributed representation, or related information.

[0016] A ninth aspect of this disclosure may further include, in any one aspect of the first to eighth aspects described above, a display process that displays a combination of mergeable feature vectors on a display unit, and a receiving process that receives from a user whether or not the combination of mergeable feature vectors can be merged.

[0017] Furthermore, a tenth aspect of this disclosure is a machine learning model training apparatus that trains a machine learning model using a set of training data merged according to merge rules generated by a data merge rule generation apparatus of the first to ninth aspects.

[0018] Furthermore, an eleventh aspect of this disclosure is a prediction device for causing a machine learning model to make predictions, which takes as input data merged according to merge rules generated by a data merge rule generation device of the first to ninth aspects, and causes a machine learning model to make predictions.

[0019] A twelfth aspect of the present disclosure is a method for operating a data merging rule generator for a machine learning model, comprising the steps of: identifying a combination of mergible feature vectors contained in a data set having ground truth labels; and generating a feature vector merging rule based on the combination of mergible feature vectors.

[0020] A thirteenth aspect of this disclosure is a program for generating data merging rules for a machine learning model, which causes a computer to perform the steps of: identifying combinations of mergible feature vectors contained in a data set having ground truth labels; and generating feature vector merging rules based on the combinations of mergible feature vectors.

[0021] A fourteenth aspect of this disclosure is a machine learning model learning device comprising a processor and memory connected to or built into the processor, wherein the machine learning model includes a merger layer that converts a first feature vector into a second feature vector and outputs it, the processor performs a learning process to train the machine learning model with the second feature vector as input, and in the learning process, the processor merges the second feature vectors output from the merger layer by changing the conversion rule from the first feature vector to the second feature vector in the merger layer.

[0022] A fifteenth aspect of this disclosure is that, in the fourteenth aspect described above, the processor may, in the training process, modify the transformation rules in the merged layer using an algorithm that assigns a score based on the value of the loss function used to train the machine learning model.

[0023] A sixteenth aspect of this disclosure is that, in the fifteenth aspect described above, the algorithm score may include the number of second feature vectors merged in the merge layer.

[0024] A 17th aspect of this disclosure, in the 15th or 16th aspect described above, may determine the initial value of the algorithm score based on at least one of the edit distance, distributed representation, or associated information of the first feature vector input to the merger layer.

[0025] An eighteenth aspect of the present disclosure is the machine learning model in the fourteenth aspect described above, further comprising an embedding layer that outputs an embedding vector corresponding to a second feature vector, and the processor may further similarize similar combinations of embedding vectors in the learning process.

[0026] A 19th aspect of this disclosure is that, in the 18th aspect described above, the processor may, in the training process, introduce a term that forces the loss function used to train the machine learning model to further resemble similar combinations of embedding vectors.

[0027] A 20th aspect of this disclosure is that, in the 18th aspect described above, the processor may, in the learning process, swap combinations of embedding vectors having a similarity of a predetermined second similarity or higher with a predetermined probability.

[0028] A 21st aspect of the present disclosure is that, in the 18th aspect described above, the processor may, in the learning process, add a correction value to at least one of the combinations of embedding vectors having a similarity of a predetermined third similarity or higher, to further make the combination of embedding vectors similar.

[0029] A 22nd aspect of the present disclosure is that, in any one aspect of the 18th to 21st aspects, the processor may, in the learning process, merge a combination of second feature vectors corresponding to a combination of embedding vectors having a similarity of at least a predetermined first similarity.

[0030] A 23rd aspect of this disclosure is that, in any one aspect of the 18th to 21st aspects described above, the processor may, in the learning process, merge the second feature vector combinations corresponding to the embedding vector combinations if the change in the prediction result of the machine learning model when the embedding vector combinations are swapped is less than a predetermined 7th threshold.

[0031] A 24th aspect of this disclosure is that, in the 18th aspect described above, the processor may, in the learning process, merge the second feature vector combinations corresponding to the embedding vector combinations if the similarity of the prediction results of the machine learning model when the embedding vector combinations are swapped is equal to or greater than a predetermined fifth similarity.

[0032] Furthermore, a 25th aspect of this disclosure is a method for operating a machine learning model learning device, wherein the machine learning model includes a merger layer that converts a first feature vector into a second feature vector and outputs it, and the method includes a step of training the machine learning model using the second feature vector, the training step includes a step of merging the second feature vectors output from the merger layer by changing the conversion rule from the first feature vector to the second feature vector in the merger layer.

[0033] Furthermore, a 26th aspect of this disclosure is a program for training a machine learning model, the machine learning model including a merger layer that converts a first feature vector into a second feature vector and outputs it, the program causes a computer to perform a step of training the machine learning model using the second feature vector, the training step causes the computer to perform a step of merging the second feature vectors output from the merger layer by changing the conversion rule from the first feature vector to the second feature vector in the merger layer. [Brief explanation of the drawing]

[0034] [Figure 1] This diagram shows the schematic configuration of the hospitalization period prediction system according to exemplary embodiment 1. [Figure 2] This is a block diagram showing the hardware configuration of a prediction server according to exemplary embodiment 1. [Figure 3] This figure shows the functional configuration of the prediction server according to exemplary embodiment 1. [Figure 4] This figure shows an example of the first training data set used in Exemplary Embodiment 1. [Figure 5]This figure shows an example of the first medical data used in Exemplary Embodiment 1. [Figure 6] This figure shows an example of the frequency distribution of the correct labels created in Exemplary Embodiment 1. [Figure 7] This figure shows an example of a merger rule for feature vectors generated in Exemplary Embodiment 1. [Figure 8] This figure shows an example of the second training data set generated in Exemplary Embodiment 1. [Figure 9] This figure shows an example of the second medical data generated in Exemplary Embodiment 1. [Figure 10] This is a flowchart illustrating the operation of the prediction server as a data merging rule generation device according to exemplary embodiment 1. [Figure 11] This figure shows an example of a frequency distribution considering the combination of items created by a modified example of Embodiment 1. [Figure 12] This figure shows the functional configuration of the prediction server according to exemplary embodiment 2. [Figure 13] This is a flowchart illustrating the processing performed by the identification unit of the prediction server according to exemplary embodiment 2. [Figure 14] This figure shows the detailed configuration of the provisional model generated in Exemplary Embodiment 2. [Figure 15] This figure shows an example of a pattern of feature vector combinations generated in Exemplary Embodiment 2. [Figure 16] This figure shows the functional configuration of the prediction server according to exemplary embodiment 3. [Figure 17] This figure shows an example of a training data set used in Exemplary Embodiment 3. [Figure 18] This figure shows the detailed configuration of the machine learning model according to exemplary embodiment 3. [Figure 19] This figure illustrates the calculations in the merged layer and embedding layer of the machine learning model according to exemplary embodiment 3. [Figure 20]This is a flowchart illustrating the machine learning model training process performed by the learning control unit of the prediction server according to Exemplary Embodiment 3. [Figure 21] This figure shows an example of a score table created in Exemplary Embodiment 3. [Figure 22] This is a flowchart illustrating the score calculation process performed by the learning control unit in the exemplary embodiment 3. [Figure 23] This figure illustrates the provisional merging of the second feature vector in the merged layer of the machine learning model according to Exemplary Embodiment 3. [Figure 24] This figure shows another example of a score table created in Exemplary Embodiment 3. [Figure 25] This figure shows an example of a score table recreated in Exemplary Embodiment 3. [Figure 26] This figure shows the functional configuration of the prediction server according to exemplary embodiment 4. [Figure 27] This is a flowchart illustrating the machine learning model training process performed by the learning control unit of the prediction server according to exemplary embodiment 4. [Figure 28] This figure shows a list of combinations of second feature vectors and corresponding embedding vectors in exemplary embodiment 4. [Figure 29] This figure illustrates the merging of the second feature vector in the merged layer of the machine learning model according to exemplary embodiment 4. [Figure 30] This figure shows the functional configuration of the prediction server according to exemplary embodiment 5. [Figure 31] This is a flowchart illustrating the machine learning model training process performed by the learning control unit of the prediction server according to Exemplary Embodiment 5. [Figure 32] This figure shows the functional configuration of the prediction server according to exemplary embodiment 6. [Figure 33] This is a flowchart illustrating the machine learning model training process performed by the learning control unit of the prediction server according to exemplary embodiment 6. [Modes for carrying out the invention]

[0035] Hereinafter, with reference to the attached drawings, exemplary embodiments of the present disclosure will be described based on an example in which the technical idea of ​​the present disclosure is applied to a hospital stay prediction system that predicts the length of hospital stay for a patient based on medical data at the time of admission. However, the scope of applicability of the technical idea of ​​the present disclosure is not limited thereto. In addition to the exemplary embodiments disclosed, various forms that can be implemented by those skilled in the art are included in the claims.

[0036] [Exemplary Embodiment 1] Figure 1 is a diagram showing the schematic configuration of a hospital stay prediction system according to Exemplary Embodiment 1 of the present disclosure. The hospital stay prediction system includes a prediction server 100, a user terminal 101, and a communication line 102 that connects the prediction server 100 and the user terminal 101 so that they can communicate with each other.

[0037] The prediction server 100 predicts the length of hospital stay for a patient based on the patient's medical data transmitted from the user terminal 101 via the communication line 102. The prediction server 100 then returns the predicted length of hospital stay to the user terminal 101 via the communication line 102.

[0038] The user terminal 101 is a well-known personal computer. The communication line 102 is the internet or an intranet, etc. The communication line 102 may be a wired line or a wireless line. Furthermore, the communication line 102 may be a dedicated line or a public line.

[0039] Figure 2 is a block diagram showing the hardware configuration of the prediction server 100. The prediction server 100 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, storage 14, input unit 15, display unit 16, and communication interface 17. Each hardware element is connected to each other via a bus 19 so that they can communicate with one another.

[0040] The CPU 11 is a central processing unit. The CPU 11 reads a program stored in the ROM 12 or storage 14 and executes the program using the RAM 13 as a workspace. In this exemplary embodiment 1, the ROM 12 or storage 14 stores a program that predicts the length of hospital stay for a patient based on the patient's medical data.

[0041] ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. Storage14 consists of a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Disk), or flash memory, and stores various programs and data, including the operating system.

[0042] The input unit 15 consists of a mouse and keyboard, and is used when the user provides input to the prediction server 100.

[0043] The display unit 16 is, for example, a liquid crystal display panel, and is used when the prediction server 100 presents information to the user. Alternatively, the display unit 16 and the input unit 15 may be made common by using a touch panel type liquid crystal display panel.

[0044] The communication interface 17 allows the prediction server 100 to communicate with other devices such as the user terminal 101. It is an interface for communication. The communication interface 17 can use standards such as Ethernet (registered trademark), FDDI (Fiber Distributed Data Interface), or Wi-Fi (registered trademark).

[0045] (Functional configuration of prediction server 100) Figure 3 shows the functional configuration of the prediction server 100 according to this exemplary embodiment 1. The prediction server 100 includes, as a functional configuration, a machine learning model 110, a specific unit 120, a rule generation unit 121, a merging unit 123, a model generation unit 130, a learning control unit 140, and a prediction control unit 150. These functional configurations are realized by the CPU 11 of the prediction server 100 reading and executing a program stored in the ROM 12 or storage 14.

[0046] The prediction server 100 receives a first training data set 160 and a first medical data set 170 as input. The first training data set 160 is a set of training data created from past medical data of hospitalized patients and is used in the training phase to train the machine learning model 110. The first medical data set 170 is medical data of patients whose length of hospitalization is to be predicted and is used in the operational phase to have the trained machine learning model 110 make predictions.

[0047] The first learning data set 160 is stored in the storage 14 or provided from an external device (not shown) via the communication line 102. The first medical treatment data 170 is provided from the user terminal 101 via the communication line 102.

[0048] Figure 4 shows an example of the first training data set 160 used in this exemplary embodiment 1. The first training data set 160 is a set of training data created from the medical data of multiple past hospitalized patients. The first training data set 160 includes 80% training data, 10% validation data, and 10% test data. Each training data includes a data ID (Identifier), two items, and one correct label. The first item is the patient's "age," the second item is the patient's "gender," and the correct label is the patient's "length of hospital stay."

[0049] In this exemplary embodiment 1, there are three types of patient "age groups": "20s," "40s," and "60s," and the feature vectors representing these are defined as three-dimensional one-hot vectors. Specifically, the feature vector representing "20s" is (1,0,0), the feature vector representing "40s" is (0,1,0), and the feature vector representing "60s" is (0,0,1).

[0050] Furthermore, there are two types of patient "gender": "male" and "female," and the feature vectors representing these are defined as two-dimensional one-hot vectors. Specifically, the feature vector representing "male" is (1,0), and the feature vector representing "female" is (0,1).

[0051] Furthermore, the correct label for a patient's "length of hospital stay" is either "less than 7 days" or "7 days or more," and the feature vectors representing these are defined as two-dimensional one-hot vectors. Specifically, the feature vector representing "less than 7 days" is (1,0), and the feature vector representing "7 days or more" is (0,1).

[0052] For example, the training data in the first row of Figure 4 with data ID "00001" means that the patient was in their "20s," was "female," and had a hospital stay of "less than 7 days."

[0053] Figure 5 shows an example of the first medical data 170 used in this exemplary embodiment 1. The first medical data 170 is medical data of a patient whose length of hospitalization is to be predicted, and includes a medical data ID and two items. The two items are in the same format as the first training data set 160. That is, the first item is the patient's "age" and the second item is the patient's "sex".

[0054] (Specific section 120) Returning to Figure 3, the identification unit 120 identifies the combinations of mergeable feature vectors included in the first training data set 160. A combination of mergeable feature vectors is a combination of feature vectors that have the same or similar meanings, and which, when input into the machine learning model 110 described later, will produce the same or similar prediction results.

[0055] The identification unit 120 creates a frequency distribution of the correct labels for each feature vector of each item included in the first training data set 160 in order to identify combinations of feature vectors that can be merged.

[0056] For example, if we create a frequency distribution of the correct labels for each feature vector of "20s," "40s," and "60s" in the "age group" item included in the first training data set 160, and represent it as a histogram, it will look like Figure 6.

[0057] Next, the identification unit 120 identifies, for each of the possible feature vector combinations in Figure 6—namely, the combination of "20s" and "40s," the combination of "40s" and "60s," and the combination of "60s" and "20s"—combinations whose frequency distribution similarity is equal to or greater than a predetermined first threshold, as feature vector combinations that can be merged. The frequency distribution similarity can be calculated using measures such as the KL (Kullback-Leibler) divergence or the JS (Jensen-Shannon) divergence.

[0058] For example, in the example in Figure 6, the length of hospital stay is relatively more often "less than 7 days" for those in their "20s" and "40s," while the length of hospital stay for those in their "60s" is relatively more often "7 days or more." Therefore, the similarity between those in their "20s" and "40s" is high. Accordingly, in Figure 6, if the similarity of the frequency distribution of the feature vector combinations of those in their "20s" and "40s" satisfies the condition that it is equal to or greater than the first threshold mentioned above, the identification unit 120 identifies the combination of feature vectors of those in their "20s" and "40s" as a combination of feature vectors that can be merged.

[0059] (Rule generation unit 121) The rule generation unit 121 generates a feature vector merge rule 122 based on the combination of mergeable feature vectors identified by the identification unit 120. For example, if the identification unit 120 identifies the combination of feature vectors "20s" and "40s" in the "age group" item as a combination of mergeable feature vectors, the rule generation unit 121 generates a merge rule 122 as shown in Figure 7. The rule generation unit 121 stores the generated merge rule 122 in a readable format, for example, in the storage 14.

[0060] (Merged section 123) The merging unit 123 reads the merging rule 122 generated by the rule generation unit 121 from the storage unit 14. Then, based on the read merging rule 122, the merging unit 123 generates a second training data set 161 by merging the combinations of mergible feature vectors contained in the first training data set 160. For example, based on the merging rule 122 shown in Figure 7, the merging unit 123 generates a second training data set 161 as shown in Figure 8 from the first training data set 160 as shown in Figure 4.

[0061] Let's compare the first training data set 160 in Figure 4 with the second training data set 161 in Figure 8. In the first training data set 160 in Figure 4, the feature vector for the "age group" item is 3-dimensional. In contrast, in the second training data set 161 in Figure 8, the feature vector for the "age group" item is 2-dimensional. This is because, in the process of generating the second training data set 161 from the first training data set 160, the combination of feature vectors for "20s" and "40s" in the "age group" item was merged, reducing the dimension of the feature vector for the "age group" item from 3 dimensions to 2 dimensions.

[0062] The second training data set 161 contains 80% training data, 10% validation data, and 10% test data. The training data is used to train the machine learning model 110.

[0063] Furthermore, the merging unit 123 generates second medical data 171 by merging the merging-compatible feature vectors contained in the first medical data 170 based on the merging rule 122 described above. For example, the merging unit 123 generates second medical data 171, as shown in Figure 9, from the first medical data 170, as shown in Figure 5, based on the merging rule 122 shown in Figure 7.

[0064] Here again, in the process of generating the second medical data 171 from the first medical data 170, the combination of feature vectors for "20s" and "40s" in the "age group" item is merged, thereby reducing the dimension of the feature vector for the "age group" item from 3 dimensions to 2 dimensions.

[0065] By using a second training data set 161 and a second clinical data set 171 with reduced dimensionality, the prediction accuracy of the machine learning model 110 can be improved compared to using the first training data set 160 and the first clinical data set 170.

[0066] (Model generation unit 130) Returning to Figure 3, the model generation unit 130 generates a machine learning model 110 based on the second training data set 161 generated by the merging unit 123.

[0067] (Machine learning model 110) The machine learning model 110 takes a feature vector representing the patient's "age" and a feature vector representing the patient's "gender" as input and predicts whether the patient's hospital stay will be "less than 7 days" or "7 days or longer". The machine learning model 110 is a deep learning model based on a neural network and includes an input layer 111, an intermediate layer 112, and an output layer 113.

[0068] (Input layer 111) The number of neurons in the input layer 111 is equal to the sum of the dimensions of the feature vectors for each item in the second training data set 161. Specifically, in the second training data set 161, the feature vector representing "age" has a dimension of 2, and the feature vector representing "gender" also has a dimension of 2. Therefore, the number of neurons in the input layer 111 is 2 + 2 = 4.

[0069] There are no special conditions regarding the number of neurons in the hidden layer 112. Furthermore, multiple hidden layers may be used instead of a single one. Each neuron in the hidden layer 112 outputs a value obtained by adding a bias to the weighted sum of the outputs of each neuron in the input layer 111 and applying an activation function. The activation function can be a sigmoid function, a ReLU function, or the like. Each neuron in the input layer 111 is connected to all of the neurons in the hidden layer 112. In other words, the input layer 111 and the hidden layer 112 are fully connected.

[0070] (Output layer 113) The number of neurons in the output layer 113 is equal to the number of correct labels in the second training data set 161. In the second training data set 161, there are two types of correct labels: "less than 7 days" and "7 days or more". Therefore, the output layer 113 contains two neurons. Each neuron in the output layer 113 outputs a value obtained by adding a bias to the weighted sum of the outputs of each neuron in the hidden layer 112 and applying an activation function. For example, the Softmax function can be used as the activation function. The Softmax function is a function such that the sum of the output values ​​of each neuron in the output layer 113 is 1. By using the Softmax function, the output value of each neuron in the output layer 113 can be considered as a probability.

[0071] One neuron in output layer 113 outputs the probability P1 that the patient's hospital stay will be "less than 7 days". The other neuron in output layer 113 outputs the probability P2 that the patient's hospital stay will be "7 days or longer". The hidden layer 112 and output layer 113 are fully connected.

[0072] (Learning control unit 140) The learning control unit 140 uses the training data included in the second learning data set 161 to train the machine learning model 110 so that it can predict the length of hospital stay for patients. During the learning process of the machine learning model 110, the weights and biases of each neuron included in the hidden layer 112 and output layer 113 of the machine learning model 110 are optimized.

[0073] In detail, the learning control unit 140 optimizes the weights and biases of each neuron by backpropagation using a loss function L defined according to the following equation based on cross-entropy error.

[0074]

number

[0075] However, the above formula assumes that the correct label is given in the form of a one-hot vector. Also, Pi(n) is the probability corresponding to the correct label of the nth training data output from the output layer 113 of the machine learning model 110, and is either P1 or P2. Specifically, when the correct label of the nth training data is "less than 7 days", Pi(n) = P1, and when the correct label of the nth training data is "7 days or more", Pi(n) = P2. Also, N is the total number of training data, for example, N = 100.

[0076] (Prediction control unit 150) The prediction control unit 150 inputs the second medical data 171 about the patient whose hospitalization period is to be predicted into the input layer 111 of the machine learning model 110 after learning is performed by the learning control unit 140, that is, the learned machine learning model 110.

[0077] The prediction control unit 150 causes the display unit 16 to display, as the predicted hospitalization period, the hospitalization period corresponding to the larger of the probabilities P1 and P2 output from the output layer 113 of the machine learning model 110. Specifically, when P1 > P2, the prediction control unit 150 causes the display unit 16 to display "less than 7 days". On the other hand, when P1 < P2, the prediction control unit 150 causes the display unit 16 to display "7 days or more".

[0078] (Operation as a device for generating a data merging rule of the prediction server 100) Next, the operation as a device for generating a data merging rule of the prediction server 100 according to the present exemplary embodiment 1 will be described.

[0079] As described above, the prediction server 100 according to the present exemplary embodiment 1 includes a specific unit 120 and a rule generation unit 121 as functional configurations. With these functional configurations, the prediction server 100 functions as a device for generating a merging rule for merging combinations of mergeable feature vectors included in input data to generate input data with reduced dimensionality.

[0080] Figure 10 is a flowchart illustrating the operation of the prediction server 100 as a data merging rule generator. In detail, these processes are performed by the identification unit 120 and the rule generation unit 121 of the prediction server 100.

[0081] In step S101 of Figure 10, the identification unit 120 creates a frequency distribution of the correct labels for each feature vector of each item included in the first training data set 160. For example, the frequency distribution of the correct labels is as shown in Figure 6 above.

[0082] In step S102, the identification unit 120 identifies combinations of feature vectors that can be merged if the similarity of the frequency distributions is equal to or greater than a predetermined first threshold for each possible combination of feature vectors. For example, if the frequency distribution is as shown in Figure 6, the identification unit 120 identifies the combination of feature vectors for "20s" and "40s" in the "age group" item as a combination of feature vectors that can be merged.

[0083] In step S103, the rule generation unit 121 generates a feature vector merge rule 122 based on the combination of mergeable feature vectors identified in step S102. For example, the feature vector merge rule 122 is as shown in Figure 7 above.

[0084] With the above steps completed, the process of generating the data merging rules is finished. Subsequently, in the training phase in which the machine learning model 110 is trained, the merging unit 123 generates a second training data set 161 by merging the merging rules 122 generated in step S103 above, for each item in the first training data set 160, with respect to the merging combinations of feature vectors that can be merged. For example, the second training data set 161 is as shown in Figure 8.

[0085] Furthermore, in the operational phase in which the machine learning model 110 is made to make predictions, the merging unit 123 generates second medical data 171 by merging combinations of mergible feature vectors for each item included in the first medical data 170, based on the feature vector merging rule 122 generated in step S103 above. For example, the second medical data 171 will look like Figure 9 mentioned above.

[0086] As described above, the prediction server 100 according to this exemplary embodiment 1 functions as a data merging rule generator that generates input data with reduced dimensionality by merging combinations of mergible feature vectors included in the input data.

[0087] As mentioned above, a combination of mergeable feature vectors is a combination of feature vectors that have the same or similar meanings, and more specifically, a combination of feature vectors that, when input into the machine learning model 110, produce the same or similar prediction results.

[0088] The data merging rule generator generates mergeable features included in the first training data set 160. The system identifies combinations of feature vectors and generates a feature vector merging rule 122 based on the merging combinations of feature vectors. This improves the prediction accuracy of the machine learning model 110 compared to when the feature vectors are not merged to reduce the dimensionality.

[0089] In other words, as shown in this example, even if the input data contains data from different age groups, such as "20s" and "40s," when this input data is input to the machine learning model 110, it may yield the same or similar prediction results. By merging the input data as in this example, even input data from different age groups, such as "20s" and "40s," can be input to the machine learning model 110 as input data of the same category with the same meaning. As a result, the number of input data of the same category in the machine learning model 110 increases. This increases the amount of training data of the same category in the training phase, improving the learning effect of the machine learning model 110, and consequently, an improvement in the prediction accuracy of the machine learning model 110 in the operational phase can be expected.

[0090] In the above-described exemplary embodiment 1, the identification unit 120 may further create a frequency distribution that takes into account the combination of items for the combination identified as a combination of feature vectors that can be merged in step S102 of Figure 10, and if the similarity of the frequency distribution that takes into account the combination of items is less than a predetermined second threshold, the combination may be excluded from the combination of feature vectors that can be merged.

[0091] Specifically, in step S102 of Figure 10, if, for example, the combination of feature vectors for "20s" and "40s" is identified as a combination that can be merged, the identification unit 120 may further create a frequency distribution that takes into account the combination of "age group" and "gender" as shown in Figure 11.

[0092] In Figure 11, the frequency distributions for "men in their 20s" and "men in their 40s" are not very similar. Similarly, the frequency distributions for "women in their 20s" and "women in their 40s" are not very similar. This is because the similarity of the frequency distributions of the feature vector combinations of "20s" and "40s" when gender is not distinguished, as shown in Figure 6, differs from the similarity of the frequency distributions of the feature vector combinations of "20s" and "40s" when gender is distinguished, as shown in Figure 11. This suggests that when gender is distinguished, "20s" and "40s" should not be merged.

[0093] In such cases, the identification unit 120 may exclude the combination of feature vectors "20s" and "40s," which was initially identified as a combination of feature vectors that can be merged in step S102 of Figure 10, from the list of combinations of feature vectors that can be merged.

[0094] In the above exemplary embodiment 1, after identifying the combinations of feature vectors that can be merged for each single item based on the similarity of the frequency distributions of the ground truth labels for each combination of feature vectors for each single item, these combinations are excluded from the list of feature vectors that can be merged based on the similarity of the frequency distributions of the ground truth labels for combinations of multiple items. However, the method for identifying the combinations of feature vectors that can be merged for combinations of multiple items is not limited to this.

[0095] A combination of feature vectors in which the similarity of the frequency distributions of the correct labels for combinations of multiple items is greater than or equal to a predetermined seventh threshold may be identified as a combination of feature vectors that can be merged. For example, instead of the frequency distribution of the correct label for "gender" only in Figure 6, a frequency distribution of the correct labels for "men in their 20s," "women in their 20s," "men in their 40s," and "women in their 40s" combining "gender" and "age group" may be created, and if the similarity of the frequency distributions of "men in their 20s" and "men in their 40s," and "women in their 20s" and "women in their 40s" is greater than or equal to the seventh threshold, then the combination of "20s" and "40s" may be identified as a combination of feature vectors that can be merged. In this embodiment, "symptoms," "age group," and "gender" are given as examples of items, but the items are not limited to these, and any item stored as medical data is acceptable, including information such as "disease" and "medical department."

[0096] Furthermore, in the above exemplary embodiment 1, the rule generation unit 121 may terminate the generation of the merge rule 122 in step S103 of Figure 10 when the number of mergeable feature vector combinations included in the merge rule 122 exceeds a predetermined third threshold. By appropriately determining the third threshold, it is possible to adjust the extent to which feature vector combinations are merged. In addition, the rule generation unit 121 may terminate the generation of the merge rule 122 in step S103 of Figure 10 when the total number of feature vectors reduced by merging with the merge rule 122 exceeds a predetermined third threshold. For example, if the rule generation unit 121 determines that the combination of item A, item B, and item C and the combination of item D and item E are mergeable feature vector combinations based on the merge rule 122, the total number of feature vectors reduced by merging is 3, and it determines whether this total of 3 is greater than or equal to the third threshold.

[0097] Furthermore, in Embodiment 1 described above, age groups such as "20s" or "40s" were used as examples of items to be merged. However, of course, strings containing words that describe a patient's symptoms, such as "cough" and "seki" or "high fever" and "fever," could also be used. "Cough" and "seki" have the same meaning, differing only in whether they are written in kanji or hiragana. Similarly, "high fever" and "fever" are similar. Therefore, the feature vectors of these items can be combined in a way that allows for merging.

[0098] Furthermore, in the above-described exemplary embodiment 1, when the identification unit 120 identifies a combination of feature vectors that can be merged in step S102 of Figure 10, it may narrow down the candidates for a combination of feature vectors that can be merged based on the edit distance between items represented by the feature vectors, distributed representations, or related information, etc.

[0099] In the example above, age groups such as "20s" and "40s" are used as examples of items to be merged. However, if the items to be merged are strings, the edit distance is defined as the minimum number of steps required to transform one string into the other by inserting, deleting, or replacing one character at a time. The fewer steps required for transformation, the closer the edit distance between the strings is. A close edit distance between strings indicates a high probability of similar meanings. Therefore, the identification unit 120 can narrow down the candidate combinations of feature vectors that can be merged based on the edit distance.

[0100] Furthermore, distributed representation is a technique for representing words as high-dimensional real-valued vectors, and words with similar meanings will have similar vector values. If the items to be merged are words represented in distributed representation, the identification unit 120 can narrow down the candidate combinations of feature vectors that can be merged by identifying words with similar meanings based on the distributed representation. In addition, related information is information that indicates the relationship between the meanings of each item to be merged. The identification unit 120 can narrow down the candidate combinations of feature vectors that can be merged based on the related information.

[0101] Furthermore, in the above exemplary embodiment 1, the identification unit 120 may present to the user a list of mergeable feature vector combinations identified in step S102 of Figure 10 by displaying them on the display unit 16. The rule generation unit 121 may receive from the user via the input unit 15 whether or not each mergeable feature vector combination displayed on the display unit 16 is mergeable, and create a merge rule 122 based on the received result.

[0102] Furthermore, the prediction server 100 according to this exemplary embodiment 1 also functions as a learning device that trains a machine learning model using a set of training data merged according to merge rules generated by the data merge rule generation device according to this disclosure.

[0103] Furthermore, the prediction server 100 according to this exemplary embodiment 1 also functions as a prediction device that takes data merged according to merge rules generated by the data merge rule generation device according to this disclosure as input and causes a machine learning model to make predictions.

[0104] [Exemplary Embodiment 2] Next, a prediction server 200 according to Exemplary Embodiment 2 of this disclosure will be described. In the following description, components that are the same as or similar to those in Exemplary Embodiment 1 will be given the same reference numerals and detailed descriptions will be omitted.

[0105] (Functional configuration of prediction server 200) Figure 12 shows the functional configuration of the prediction server 200 according to this exemplary embodiment 2. In the prediction server 200, the specific unit 120 included in exemplary embodiment 1 has been replaced with a specific unit 220. The specific unit 220 generates a provisional model 280.

[0106] (Processing performed by the specific unit 220) Figure 13 is a flowchart illustrating the processing performed by the specific unit 220 of the prediction server 200 according to this exemplary embodiment 2. At the start of the flowchart in Figure 13, the first learning data set 160 is divided into 80% training data, 10% validation data, and 10% test data.

[0107] In step S201 of Figure 13, the identification unit 220 generates a provisional model 280 using the feature vectors included in the first training data set 160 as input.

[0108] Figure 14 shows the detailed configuration of the provisional model 280. The provisional model 280 has a similar configuration to the machine learning model 110 and includes an input layer 281, a hidden layer 282, and an output layer 283. The configuration and coupling relationship of the hidden layer 282 and the output layer 283 of the provisional model 280 are identical to the configuration and coupling relationship of the hidden layer 112 and the output layer 113 of the machine learning model 110.

[0109] The number of neurons in the input layer 281 of the provisional model 280 is equal to the sum of the dimensions of the feature vectors for each item in the first training data set 160. Specifically, in the first training data set 160 in Figure 4, the feature vector representing "age" has a dimension of 3, and the feature vector representing "gender" has a dimension of 2. Therefore, the number of neurons in the input layer 281 is 3 + 2 = 5.

[0110] In step S202, the identification unit 220 trains the provisional model 280 using the training data included in the first training data set 160. Specifically, the identification unit 220 optimizes the weights and biases of each neuron in the hidden layer 282 and output layer 283 of the provisional model 280 by backpropagation using the cross-entropy error-based loss function L described in Exemplary Embodiment 1.

[0111] In step S203, the identification unit 220 enumerates the combinations of feature vectors for each item included in the first training data set 160, and generates a pattern of feature vector combinations as shown in the left column of Figure 15.

[0112] In step S204, the identification unit 220 sequentially selects one combination of feature vectors at a time from the patterns in Figure 15, and calculates the change in the prediction result of the provisional model 280 when the selected combination of feature vectors is swapped, according to the following formula. The right-hand column of Figure 15 shows the change in the prediction result calculated for each combination of feature vectors.

[0113]

number

[0114] However, in the above formula, P1(m) is the probability that the hospital stay will be "less than 7 days" when the selected combination of feature vectors for the m-th validation data point is input into the provisional model 280 without changing them. Also, P1_swap(m) is the probability that the hospital stay will be "less than 7 days" when the selected combination of feature vectors for the m-th validation data point is changed and input into the provisional model 280. Furthermore, M is the total number of validation data points.

[0115] Alternatively, instead of using the above formula, the change in the prediction result may be calculated according to the following formula.

[0116]

number

[0117] However, in the above formula, P2(m) is the probability that the hospital stay will be "7 days or longer" when the selected combination of feature vectors for the m-th validation data point is input into the provisional model 280 without changing them. Also, P2_swap(m) is the probability that the hospital stay will be "7 days or longer" when the selected combination of feature vectors for the m-th validation data point is changed and input into the provisional model 280. Furthermore, M is the total number of validation data points.

[0118] In step S205, the identification unit 220 identifies combinations of feature vectors in the pattern shown in Figure 15 whose change value in the prediction result is less than a predetermined fourth threshold as combinations of feature vectors that can be merged. For example, if the fourth threshold = 10 percent, the identification unit 220 identifies the combination of feature vectors for "20s" and "40s" as a combination of feature vectors that can be merged.

[0119] With the above steps completed, the processing performed by the identification unit 220 is finished. The operation of the prediction server 200 after the identification unit 220 has identified the combination of feature vectors that can be merged is the same as in the exemplary embodiment 1.

[0120] As described above, the identification unit 220 of the prediction server 200 according to this exemplary embodiment 2 generates and trains a provisional model 280 using the feature vectors included in the first training data set 160 as input. The identification unit 220 selects a combination of feature vectors from the first training data set 160, and identifies the combination of feature vectors as a combination of feature vectors that can be merged if the change in the prediction result of the provisional model 280 when the selected combination of feature vectors is swapped is less than a predetermined fourth threshold.

[0121] Due to the above features, the prediction server 200 according to this exemplary embodiment 2 merges combinations of feature vectors while confirming that identical or similar prediction results are obtained when input to a provisional model 280 having a similar configuration to the machine learning model 110. This makes it possible to more reliably improve the prediction accuracy of the machine learning model 110.

[0122] In the above-described exemplary embodiment 2, when the specific unit 220 selects one combination of feature vectors at a time and attempts to swap them in step S204 of Figure 13, it may narrow down the combination of feature vectors to be swapped based on the edit distance of the feature vectors, distributed representation, or related information, etc.

[0123] Furthermore, in the exemplary embodiment 2 described above, the identification unit 220 may display on the display unit 16 a list of mergeable feature vector combinations identified in step S205 of Figure 13, sorted in ascending order of the change in the prediction result. The rule generation unit 121 may receive from the user via the input unit 15 whether or not each mergeable feature vector combination displayed on the display unit 16 is mergeable, and generate a merge rule 122 based on the received result.

[0124] [Exemplary Embodiment 3] Next, the prediction server 300 according to Exemplary Embodiment 3 of this Disclosure will be described. In Exemplary Embodiments 1 and 2 described above, feature vector merging was performed prior to training the machine learning model 110. In contrast, in Exemplary Embodiment 3, feature vector merging is performed simultaneously during the process of training the machine learning model.

[0125] (Functional configuration of prediction server 300) Figure 16 shows the functional configuration of the prediction server 300 according to this exemplary embodiment 3. The prediction server 300 comprises a machine learning model 310, a learning control unit 340, and a prediction control unit 350 as its functional configuration. These functional configurations are realized by the CPU 11 of the prediction server 300 reading and executing a program stored in the ROM 12 or storage 14.

[0126] The prediction server 300 receives the training data set 360 and the medical data 370 as input. In the training phase, during which the machine learning model 310 is trained, the training data set 360, created from past medical data of hospitalized patients, is input. The training data set 360 is stored in the storage 14 or provided via the communication line 102 from an external device (not shown). On the other hand, in the operation phase, during which the trained machine learning model 310 is made to make predictions, the medical data 370 of patients whose length of hospitalization is to be predicted is input. The medical data 370 is provided from the user terminal 101 via the communication line 102.

[0127] Figure 17 shows an example of the training data set 360 used in this exemplary embodiment 3. The training data set 360 is a collection of training data created from the medical data of multiple past hospitalized patients. Each training data includes a data ID, a patient's "symptoms" item, and "length of hospital stay" as the correct label.

[0128] In this exemplary embodiment 3, there are three types of "symptoms" for the patient: "cough," "fever," and "high fever," and the first feature vectors representing these are defined as three-dimensional one-hot vectors. Specifically, the first feature vector representing "cough" is (1,0,0), the first feature vector representing "fever" is (0,1,0), and the first feature vector representing "high fever" is (0,0,1).

[0129] Furthermore, the correct label for hospitalization duration is either "less than 7 days" or "7 days or more," and the feature vectors representing these are defined as two-dimensional one-hot vectors. Specifically, the feature vector representing "less than 7 days" is (1,0), and the feature vector representing "7 days or more" is (0,1). For example, the training data in the first row of Figure 17 with data ID "00001" represents a patient whose symptom at the time of admission was "cough" = (1,0,0), and whose hospitalization duration was "less than 7 days."

[0130] The training data set 360 consists of 80% training data, 10% validation data, and 10% test data. The training data is used to train the machine learning model 310.

[0131] (Machine learning model 310) Returning to Figure 16, the machine learning model 310 is a deep learning model based on a neural network, and includes an input layer 311, a merge layer 312, an embedding layer 313, and a prediction unit 314.

[0132] Figure 18 shows the detailed configuration of the machine learning model 310. The machine learning model 310 is input with a first feature vector that represents the patient's "symptoms" as described above. From here on, the first feature vector will be C m =(x1,x2,x3)=(δ 1m ,δ 2m ,δ 3m This is denoted as ), where the subscripts m=1,2,3 and δ is the Kronecker delta. Specifically, C1=(1,0,0), C2=(0,1,0), and C3=(0,0,1).

[0133] (Input layer 311) The input layer 311 receives the first feature vector C. m = (x1, x2, x3) is output as is. In detail, the input layer 311 contains three neurons 311a, 311b, and 311c. Each neuron 311a, 311b, and 311c has a first feature vector C mEach element x1, x2, and x3 of it is input respectively. Each neuron 311a, 311b, and 311c outputs each element x1, x2, and x3 of the input first feature vector C m as it is.

[0134] Note that the number of neurons included in the input layer 311 is 3 because the dimensionality of the first feature vector C m considered in the exemplary embodiment 3 is 3. Generally, the input layer 311 includes the same number of neurons as the dimensionality of the first feature vector C m .

[0135] (Merge layer 312) The merge layer 312 converts the first feature vector C m output from the input layer 311 into the second feature vector D m and outputs it. Thereafter, the second feature vector is denoted as D m =(y1,y2,y3)=(δ 1m ,δ 2m ,δ 3m ). However, the subscript m = 1, 2, 3, and δ is the Kronecker delta. Specifically written, D1=(1,0,0), D2=(0,1,0), and D3=(0,0,1).

[0136] As described above, C1 = D1=(1,0,0), C2 = D2=(0,1,0), and C3 = D3=(0,0,1). Therefore, the set {C m} of the first feature vectors is equal to the set {Dm} of the second feature vectors. In other words, the merge layer 312 functions as a conversion table from the first feature vector C m to the second feature vector D m .

[0137] The merge layer 312 includes three neurons 312a, 312b, and 312c. Generally, the merge layer 312 includes the same number of neurons as the dimensionality of the first feature vector C m .

[0138] Each neuron 312a, 312b, and 312c in the merged layer 312 outputs the weighted sum of the outputs x1, x2, and x3 of each neuron 311a, 311b, and 311c in the input layer 311. Therefore, the outputs y1, y2, and y3 of each neuron 312a, 312b, and 311c in the merged layer 312 are weighted w (1) 11 From lol (1) 33 Using this, it can be written as follows:

[0139] y1 = x1·w (1) 11 +x2·w (1) 21 +x3·w (1) 31 y² = x¹·w (1) 12 +x2·w (1) 22 +x3·w (1) 32 y3 = x1·w (1) 13 +x2·w (1) 23 +x3·w (1) 33

[0140] The above operation performed in merged layer 312 can be written in the form of a matrix operation as follows:

[0141] D m =C m W (1)

[0142] However, in the above equation, D m =(y1,y2,y3) is the second feature vector output from merged layer 312, and C m =(x1,x2,x3) is the first feature vector input to merge layer 312. Also, matrix W (1) It is defined according to the following formula.

[0143] W (1) =(w (1) ij )

[0144] However, the indices i and j are 1, 2, and 3.

[0145] Furthermore, focusing on the function of the merged layer 312 as a transformation table, the second feature vector D output from the merged layer 312 m =(y1,y2,y3) is one of the following: D1=C1=(1,0,0), D2=C2=(0,1,0), or D3=C3=(0,0,1).

[0146] Furthermore, in the initial state before training the machine learning model 310, the merged layer 312 receives the first feature vector C from the input layer 311. m The second feature vector D having the same value m It is set to convert to, in other words, output as is. That is, it is set so that y1=x1, y2=x2, and y3=x3.

[0147] Therefore, in the initial state before training the machine learning model 310, the matrix of the merged layer 312 W (1) This is the identity matrix as follows:

[0148] W (1) =(w (1) ij )=(δ ij )

[0149] However, the indices i and j are 1, 2, and 3, respectively.

[0150] Furthermore, as will be described later, in the process of training the machine learning model 310, the matrix W of the merged layer 312 (1) The weights of are also changed accordingly. This is the first feature vector C in merged layer 312. m From the second feature vector D m This means that the transformation rules to multiple second feature vectors D are changed, specifically the rules for transforming to D. m The mergers are performed. This optimizes the transformation rules so that the prediction accuracy of the machine learning model 310 improves.

[0151] (Embedded layer 313) The embedding layer 313 outputs the second feature vector D from the merged layer 312. m Corresponding embedding vector E k Outputs.

[0152] In detail, the embedding layer 313 contains four neurons 313a, 313b, 313c, and 313d. However, the number of neurons in the embedding layer 313 is not necessarily four. It may contain two, three, or five or more neurons. Typically, the number of neurons in the embedding layer 313 corresponds to the first feature vector C m It is approximately 10 to 1000 times the number of dimensions.

[0153] Each neuron 313a, 313b, 313c, and 313d in the embedding layer 313 outputs the weighted sum of the outputs y1, y2, and y3 of each neuron 312a, 312b, and 312c in the merged layer 312. Therefore, the outputs z1, z2, z3, and z4 of each neuron 313a, 313b, 313c, and 313d in the embedding layer 313 are weighted w (2) 11 From lol (2) 34 Using this, it can be written as follows:

[0154] z1 = y1·w (2) 11 +y2·w (2) 21 +y3·w (2) 31 z² = y¹·w (2) 12 +y2·w (2) 22 +y3·w (2) 32 z3 = y1·w (2) 13 +y2·w (2) 23 +y3·w (2)33 z4 = y1·w (2) 13 + y2·w (2) 23 + y3·w (2) 33

[0155] The above operation performed in the embedding layer 313 can be written in the form of matrix operation as follows.

[0156] E k = Y m W (2)

[0157] However, in the above formula, E k = (z1, z2, z3, z4) is the embedding vector output from the embedding layer 313, and D m = (y1, y2, y3) is the second feature vector output from the merging layer 312. Also, the matrix W (2) is defined according to the following formula.

[0158] W (2) = (w (2) ij )

[0159] However, the subscripts are i = 1, 2, 3 and j = 1, 2, 3, 4.

[0160] Summarizing the above results, in the initial state before learning of the machine learning model 310, the operations performed in the merging layer 312 and the embedding layer 313 can be summarized as follows. Also, please refer to FIG. 19 together. Also, hereinafter, consider "cough", "fever", and "high fever" as examples of candidates for merging.

[0161] When the first feature vector C1 = (1, 0, 0) representing "cough" is input to the merging layer 312, the merging layer 312 converts it into a second feature vector D1 = (1, 0, 0) with the same content and outputs it. When the second feature vector D1 = (1, 0, 0) is input to the embedding layer 313, the embedding layer 313 outputs the corresponding embedding vector E1 = (w(2) 11 ,w (2) 12 ,w (2) 13 ,w (2) 14 Outputs ).

[0162] When the first feature vector C2=(0,1,0) representing "heat generation" is input to the merger layer 312, the merger layer 312 converts it into a second feature vector D2=(0,1,0) with the same content and outputs it. When the second feature vector D2=(0,1,0) is input to the embedding layer 313, the embedding layer 313 converts it into the corresponding embedding vector E2=(w (2) 21 ,w (2) 22 ,w (2) 23 ,w (2) 24 Outputs ).

[0163] When the first feature vector C3=(0,0,1) representing "high heat" is input to the merger layer 312, the merger layer 312 converts it into a second feature vector D3=(0,0,1) with the same content and outputs it. When the second feature vector D3=(0,0,1) is input to the embedding layer 313, the embedding layer 313 outputs the corresponding embedding vector E3=(w (2) 31 ,w (2) 32 ,w (2) 33 ,w (2) 34 Outputs ).

[0164] From the above results, it can be interpreted that the second feature vector D1 is associated with the embedding vector E1. Similarly, the second feature vector D2 is associated with the embedding vector E2, and the second feature vector D3 is associated with the embedding vector E3.

[0165] (Prediction unit 314) Returning to Figure 18, the prediction unit 314 receives the embedding vector E output from the embedding layer 313. k In other words, the output z1, z2, z3, and z4 of each neuron 313a, 313b, 313c, and 313d in the implanted layer 313 are used as inputs to predict the patient's length of hospital stay. More specifically, the prediction unit 314 includes an input layer 315, a hidden layer 316, and an output layer 317.

[0166] (Input layer 315) The input layer 315 contains four neurons 315a, 315b, 315c, and 315d. Each neuron 315a, 315b, 315c, and 315d transmits the outputs z1, z2, z3, and z4 of each neuron 313a, 313b, 313c, and 313d in the embedding layer 313 directly to the hidden layer 316. Generally, the input layer 315 contains the same number of neurons as the embedding layer 313.

[0167] (Middle class 316) Hidden layer 316 contains four neurons 316a, 316b, 316c, and 316d. Each neuron 316a, 316b, 316c, and 316d in hidden layer 316 outputs a value obtained by adding a bias to the weighted sum of the outputs of each neuron 315a, 315b, 315c, and 315d in input layer 315, and then applying an activation function. The activation function can be a sigmoid function, a ReLU function, or the like. Input layer 315 and hidden layer 316 are fully connected.

[0168] Furthermore, the number of neurons in the hidden layer 316 is not limited to four. The number of neurons in the hidden layer 316 may be two, three, or five or more. In addition, there may be multiple hidden layers instead of a single one.

[0169] (Output layer 317) The output layer 317 contains two neurons, 317a and 317b. Each neuron 317a and 317b in the output layer 317 outputs a value obtained by adding a bias to the weighted sum of the outputs of each neuron 316a, 316b, 316c, and 316d in the hidden layer 316, and then applying an activation function. The Softmax function can be used as the activation function. As a result, the upper neuron 317a outputs the probability P1 that the patient's hospital stay will be "less than 7 days". The lower neuron 317b outputs the probability P2 that the patient's hospital stay will be "7 days or longer". The hidden layer 316 and the output layer 317 are fully connected.

[0170] Note that the output layer 317 contains two neurons because there are only two types of correct labels: "less than 7 days" and "7 days or more." Generally, the output layer 317 contains a number of neurons equal to the number of types of correct labels.

[0171] Furthermore, as will be described later, during the training process of the machine learning model 310, the weights and biases of each neuron in the hidden layer 316 and output layer 317 within the prediction unit 314 are optimized.

[0172] (Learning control unit 340) Returning to Figure 16, the learning control unit 340 uses the training data included in the aforementioned learning data set 360 to train the machine learning model 310 so that it can predict the length of hospital stay for patients. During the learning process of the machine learning model 310, the weights and biases of each neuron included in the embedding layer 313 and prediction unit 314 of the machine learning model 310 are optimized.

[0173] Furthermore, in the process of training the machine learning model 310, the learning control unit 340 processes the first feature vector C in the merged layer 312. m From the second feature vector D m By changing the transformation rule to , the second feature vector D output from merged layer 312 is obtained. m To merge.

[0174] Specifically, the learning control unit 340 uses an algorithm in which a score is given based on the value of the loss function used for learning the machine learning model 310, and changes the conversion rule from the first feature vector C m to the second feature vector D m in the merging layer 312. By doing so, the second feature vector D m output from the merging layer 312 is merged. As a result, the same effect as merging the first feature vector C m generated from the patient's medical data to reduce the dimensionality is obtained.

[0175] (Prediction control unit 350) The prediction control unit 350 inputs the medical data 370 of the patient for whom the hospitalization period is to be predicted into the input layer 311 of the above-described machine learning model 310 after learning, that is, the learned machine learning model 310. The medical data 370 of the patient is provided from the user terminal 101 via the communication line 102.

[0176] The prediction control unit 350 causes the display unit 16 to display, as the predicted hospitalization period, the hospitalization period corresponding to the larger probability among the probabilities P1 and P2 output from the output layer 317 in the prediction unit 314 of the machine learning model 310. Specifically, when P1 > P2, the prediction control unit 350 causes the display unit 16 to display "less than 7 days". On the other hand, when P1 < P2, the prediction control unit 3150 causes the display unit 16 to display "7 days or more".

[0177] (Operation during learning of the machine learning model 310 in the prediction server 300) Next, the operation during learning of the machine learning model 310 in the prediction server 300 according to the third exemplary embodiment will be described.

[0178] FIG. 20 is a flowchart for explaining the learning process of the machine learning model 310 executed by the learning control unit 340 of the prediction server 300.

[0179] In step 301 of Figure 20, the learning control unit 340 defines a set S that includes all second feature vectors. In this exemplary embodiment 3, there are three types of second feature vectors: D1, D2, and D3. Therefore, a set S = {D1, D2, D3} is defined that includes all second feature vectors.

[0180] In step S302, the learning control unit 340 enumerates all patterns of subsets containing two or more elements of the second feature vector set S={D1,D2,D3} and creates a score table as shown in Figure 21. In the score table in Figure 21, for example, the first subset {D1,D2} contains the second feature vectors D1 and D2. Also, the initial value of each score in the score table is 0.

[0181] In step S303, the learning control unit 340 optimizes the weights and biases of each neuron in the embedding layer 313 and prediction unit 314 of the machine learning model 310 using the training data included in the learning data set 360.

[0182] In detail, the learning control unit 340 optimizes the weights and biases of each neuron by backpropagation using a loss function L defined according to the following equation based on cross-entropy error.

[0183]

number

[0184] However, the above formula assumes that the correct labels are given in the form of one-hot vectors. In the above formula, Pi(n) is the probability that corresponds to the correct label of the nth training data, output from the output layer 317 of the machine learning model 310, and is either P1 or P2. Specifically, if the correct label of the nth training data is "less than 7 days", then Pi(n) = P1, and if the correct label of the nth training data is "7 days or more", then Pi(n) = P2. Also, N is the total number of training data, for example, N = 100.

[0185] In step S304, the learning control unit 340 calculates the score for each subset included in the score table in Figure 21. Specifically, the learning control unit 340 performs the score calculation process shown in the flowchart in Figure 22.

[0186] In step S401 of Figure 22, the learning control unit 340 inputs N training data into the machine learning model 310 and calculates the value of the loss function described above. Let this value of the loss function be L1.

[0187] In step S402, the learning control unit 340 selects one subset from the score table in Figure 21. For example, the learning control unit 340 selects the subset {D2, D3}.

[0188] In step S403, the learning control unit 340 provisionally merges the second feature vectors included in the subset selected in step S402. Specifically, the learning control unit 340 is matrix W of merged layer 312. (1) By rewriting the weights, the transformation rule from the first feature vector to the second feature vector in merged layer 312 is provisionally changed.

[0189] For example, when provisionally merging the second feature vectors D2 and D3, the learning control unit 340 processes the matrix W of the merging layer 312 as shown in Figure 23. (1) The elements in the third row are temporarily rewritten to (0,1,0). As a result, when the first feature vector C3=(0,0,1) is input to the merged layer 312, the merged layer 312 will output the second feature vector D2=(0,1,0).

[0190] This means that the second feature vectors D2 and D3 output from the merged layer 312 were merged by changing the transformation rule from the first feature vector to the second feature vector in the merged layer 312.

[0191] Furthermore, when provisionally merging the second feature vectors D2 and D3, the matrix W of the merging layer 312 (1) You can temporarily rewrite each element in the second row to (0,0,1). In this case, when the first feature vector C2=(0,1,0) is input to the merged layer 312, the second feature vector D3=(0,0,1) will be output from the merged layer 312.

[0192] In step S404, the learning control unit 340 re-inputs the N training data into the machine learning model 310 with the second feature vectors provisionally merged, and recalculates the value of the loss function described above. Let this value of the loss function be L2.

[0193] In step S405, the learning control unit 340 calculates a score for the subset containing the provisionally merged second feature vector according to the following formula, and adds the calculated score to the score of that subset in the score table in Figure 21.

[0194] Score = L1-L2

[0195] However, in the above equation, L1 is the value of the loss function calculated earlier in step S401, and L2 is the value of the loss function recalculated in step S404 above.

[0196] For example, if the score calculated when the second feature vectors D2 and D3 are provisionally merged is 0.7, the learning control unit 340 adds 0.7 to the score of the second subset {D2, D3} in the score table in Figure 21.

[0197] In step S406, the learning control unit 340 unmerges the provisionally merged second feature vectors. Specifically, the learning control unit 340 unmerges the matrix W of the merged layer 312. (1) By rewriting the weights again, the transformation rule from the first feature vector to the second feature vector in merged layer 312 is restored.

[0198] In step S407, the learning control unit 340 determines whether all subsets in the score table in Figure 21 have been selected and whether the processes from steps S402 to S406 have been executed.

[0199] If not all subsets in the score table in Figure 21 are selected, the learning control unit 340 returns to step S402 to select the unselected subsets.

[0200] On the other hand, if all subsets in the score table in Figure 21 have been selected and the processes in steps S402 to S406 above have been executed, the learning control unit 340 proceeds to the process in step S305 in Figure 20.

[0201] In step S305 of Figure 20, the learning control unit 340 determines whether or not it is possible to merge the second feature vectors. More specifically, the learning control unit 340 determines whether or not there exists a subset where the number of second feature vectors that have already been merged is less than a predetermined fifth threshold, and the score in the score table is greater than or equal to a predetermined sixth threshold.

[0202] If it is determined in step S305 above that merging of the second feature vector is not possible, that is, if step S305 = NO, the learning control unit 340 proceeds to the process in step S309, which will be described later.

[0203] On the other hand, if it is determined in step S305 that the second feature vector can be merged, that is, if step S305 = YES, the learning control unit 340 proceeds to the next step S306.

[0204] For example, if the fifth threshold is 2, the sixth threshold is 20, and the score table is in the state shown in Figure 24, then it is determined that the second feature vectors D2 and D3, which are included in the subset {D2, D3}, are mergeable.

[0205] In step S306, the learning control unit 340 merges the second feature vector determined to be mergable in step S305 above. Specifically, the learning control unit 340 changes the conversion rule from the first feature vector to the second feature vector in the merging layer 312 by rewriting the weight of the matrix W (1) in the merging layer 312.

[0206] In step S307, the learning control unit 340 redefines the set S defined in step S301 previously. For example, when the second feature vectors D2 and D3 are merged in step S306 above, the set S is redefined as S = {D1, D2}.

[0207] In step S308, the learning control unit 340 recreates the score table created in step S302 previously. For example, when the set S is redefined as S = {D1, D2} in step S307 above, the score table becomes as shown in FIG. 25.

[0208] In step S309, the learning control unit 340 determines whether the processes of steps S303 to S308 above have been executed the specified number of times in advance. For example, the specified number of times in advance = 10,000 times.

[0209] If the processes of steps S303 to S308 above have not been executed the specified number of times, the learning control unit 340 returns to the process of step S303.

[0210] On the other hand, if the processes of steps S303 to S308 above have been executed the specified number of times, the learning control unit 340 ends the process of the flowchart in FIG. 20.

[0211] Once the above process is complete, the training of the machine learning model 310 is finished. The merged layer 312 of the trained machine learning model 310 outputs a second feature vector that has been merged in a way that improves the prediction accuracy of the machine learning model 310. The embedding layer 313 of the trained machine learning model 310 outputs an embedding vector that accurately captures the meaning of the merged second feature vector. The prediction unit 314 of the trained machine learning model 310 outputs the probability of the length of hospital stay predicted from the patient's medical data.

[0212] As described above, the machine learning model 310 of the prediction server 300 according to this exemplary embodiment 3 includes a merger layer 312 that converts a first feature vector into a second feature vector and outputs it. In the process of training the machine learning model 310, the learning control unit 340 of the prediction server 300 merges the second feature vectors output from the merger layer 312 by changing the conversion rule from the first feature vector to the second feature vector in the merger layer 312.

[0213] In detail, the learning control unit 340 of the prediction server 300 merges the second feature vectors output from the merging layer 312 using an algorithm that assigns a score based on the value of the loss function used to train the machine learning model 310.

[0214] The above characteristics yield a similar effect to merging the first feature vector generated from the patient's medical data to reduce its dimensionality. As a result, the prediction accuracy of machine learning model 310 improves compared to when the first feature vector is not merged to reduce its dimensionality. The reason why creating dimensionality in the feature vector improves prediction accuracy is as explained above.

[0215] Furthermore, the number of second feature vectors merged in the merged layer 312 may be included in the score of the algorithm used to optimize the transformation rules of the merged layer 312. For example, by increasing the score in proportion to the number of second feature vectors merged, the merging of second feature vectors will be performed more aggressively.

[0216] Furthermore, although the initial value of the algorithm's score was 0 in the score table in Figure 21, the initial value of the score may be determined based on the edit distance, distributed representation, or related information of the first feature vector input to the merged layer 312. Providing an initial value in this way will allow the optimization to proceed more quickly.

[0217] Furthermore, the algorithm used to modify the transformation rules of merged layer 312 is not limited to the algorithm described above. Various algorithms can be used to modify the transformation rules of merged layer 312, including reinforcement learning algorithms such as REINFORCE, Q-learning, or DQN.

[0218] [Exemplary Embodiment 4] Next, a prediction server 400 according to Exemplary Embodiment 4 of this Disclosure will be described. In the following description, components that are the same as or similar to those in Exemplary Embodiment 3 will be given the same reference numerals and detailed descriptions will be omitted.

[0219] In this exemplary embodiment 4 and the exemplary embodiments 5 and 6 described later, during the process of training the machine learning model 310, operations are performed to further similarize combinations of similar embedding vectors. Subsequently, combinations of second feature vectors corresponding to combinations of highly similar embedding vectors are merged.

[0220] (Functional configuration of prediction server 400) Figure 26 shows the functional configuration of the prediction server 400 according to this exemplary embodiment 4. In the prediction server 400, the learning control unit 340 included in exemplary embodiment 3 has been replaced with a learning control unit 440.

[0221] (Learning control unit 440) In the process of training the machine learning model 310 to predict the length of hospital stay for patients, the learning control unit 440 merges the second feature vector output from the merger layer 312 by changing the transformation rule from the first feature vector to the second feature vector in the merger layer 312.

[0222] In detail, the learning control unit 440 introduces a term in the loss function used to train the machine learning model 310 that forces similar combinations of embedding vectors to become even more similar. This ensures that the machine learning model 310 is trained under the constraint that similar combinations of embedding vectors are forced to become even more similar. Then, the learning control unit 440 merges the second combinations of feature vectors that correspond to highly similar combinations of embedding vectors. This achieves an effect similar to merging the first feature vectors generated from patient medical data to reduce the dimensionality.

[0223] (Operation of machine learning model 310 during training on prediction server 400) Figure 27 is a flowchart illustrating the training process of the machine learning model 310, which is performed by the learning control unit 440 of the prediction server 400.

[0224] In step S501 of Figure 27, the learning control unit 440 uses the training data included in the learning data set 360 to optimize the weights and biases of each neuron in the embedding layer 313 and prediction unit 314 of the machine learning model 310.

[0225] In detail, the learning control unit 440 optimizes the weights and biases of each neuron by backpropagation using a loss function L defined according to the following equation.

[0226]

number

[0227] However, in the above formula, Pi(n) is the probability corresponding to the correct label of the n-th training data output from the output layer 317 of the machine learning model 310, and is either P1 or P2. Specifically, when the correct label of the n-th training data is "less than 7 days", Pi(n) = P1, and when the correct label of the n-th training data is "7 days or more", Pi(n) = P2. Also, N is the total number of training data, for example, N = 100.

[0228] Also, in the above formula, γ is a parameter for scale adjustment. Also, σ ij is the similarity of combinations of embedding vectors where the similarity Sim is greater than or equal to a predetermined threshold TH, and is defined according to the following formula.

[0229]

Equation

Equation

[0230] In the above formula, the value of the threshold TH is, for example, 0.8.

[0231] In this exemplary embodiment 4, in the initial state before the learning of the machine learning model 310, there are three embedding vectors E1, E2, and E3. Therefore, combinations of the three embedding vectors {E1, E2}, {E2, E3}, and {E3, E1} exist. In this case, the above-mentioned σ ij is the similarity of combinations in which the similarity Sim is greater than or equal to the threshold TH among these three combinations of embedding vectors.

[0232] As described above, in the loss function L, by introducing a term that forces combinations of similar embedding vectors to become even more similar, as the learning of the machine learning model 310 progresses, combinations of similar embedding vectors will become even more similar.

[0233] In step S502, the learning control unit 440 determines whether or not it is possible to merge the second feature vectors. More specifically, the learning control unit 440 determines whether or not there exists a combination of second feature vectors that corresponds to a combination of embedding vectors whose cosine similarity is equal to or greater than a predetermined first similarity. However, the cosine similarity is defined according to the following formula, where A is one embedding vector and B is the other embedding vector.

[0234]

number

[0235] If it is determined in step S502 that merging of the second feature vector is not possible, that is, if step S502 = NO, the learning control unit 440 proceeds to the process of step S504, which will be described later.

[0236] On the other hand, if it is determined in step S502 that the second feature vector can be merged, that is, if step S502 = YES, the learning control unit 440 proceeds to the next step S503.

[0237] For example, if the first similarity is 0.8 and there are combinations of second feature vectors and embedding vectors as shown in Figure 28, then it is determined that the second feature vectors D2 and D3, which have a cosine similarity of 0.9, can be merged.

[0238] In step S503, the learning control unit 440 merges the combination of second feature vectors that were determined to be mergeable in step S502. Specifically, as shown in Figure 29, the learning control unit 440 merges the matrix W of the merging layer 312. (1) By rewriting the weights in the third row, the second feature vectors D2 and D3 output from merge layer 312 are merged.

[0239] In step S504, the learning control unit 440 determines whether the processes in steps S501 to S503 have been executed a predetermined number of times. For example, the predetermined number of times is 10,000.

[0240] If the processes from steps S501 to S503 described above have not been executed the predetermined number of times, the learning control unit 440 returns to the process of step 501.

[0241] On the other hand, if the processes from steps S501 to S503 described above have been executed a predetermined number of times, the learning control unit 440 terminates the process shown in the flowchart of Figure 27.

[0242] Once the above process is complete, the training of the machine learning model 310 is finished. The merged layer 312 of the trained machine learning model 310 outputs a second feature vector that has been merged in a way that improves the prediction accuracy of the machine learning model 310. The embedding layer 313 of the trained machine learning model 310 outputs an embedding vector that accurately captures the meaning of the merged second feature vector and has improved similarity. The prediction unit 314 of the trained machine learning model 310 outputs the probability of the length of hospital stay predicted from the patient's medical data.

[0243] As described above, the learning control unit 440 of the prediction server 400 in this exemplary embodiment 4 introduces a term in the loss function L used for training the machine learning model 310 that forces similar combinations of embedding vectors to become even more similar. This has the same effect as merging the first feature vectors generated from patient medical data to reduce the number of dimensions. As a result, the prediction accuracy of the machine learning model 310 is improved compared to when the number of dimensions is not reduced by merging the first feature vectors.

[0244] In addition, in the above-described exemplary embodiment 4, as an alternative method for determining whether or not the second feature vectors can be merged in step S502 of the flowchart in Figure 27, similar to the exemplary embodiment 2 described above, if the change in the prediction result of the machine learning model 310 when the combination of embedding vectors is swapped is less than a predetermined seventh threshold, the combination of second feature vectors corresponding to the combination of embedding vectors may be identified as a combination of second feature vectors that can be merged.

[0245] [Exemplary Embodiment 5] Next, a prediction server 500 according to exemplary embodiment 5 of this disclosure will be described.

[0246] (Functional configuration of Prediction Server 500) Figure 30 shows the functional configuration of the prediction server 500 according to this exemplary embodiment 5. In the prediction server 500, the learning control unit 340 included in exemplary embodiment 3 has been replaced with a learning control unit 540.

[0247] (Learning control unit 540) In the process of training the machine learning model 310 to predict the length of hospital stay for patients, the learning control unit 540 merges the second feature vectors output from the merger layer 312 by changing the transformation rule from the first feature vector to the second feature vector in the merger layer 312.

[0248] In detail, the learning control unit 540, during the training process of the machine learning model 310, swaps combinations of embedding vectors having a similarity of a predetermined second similarity or higher with a predetermined probability. As a result, the machine learning model 310 is trained under conditions where similar combinations of embedding vectors are swapped with a certain probability. Then, the learning control unit 540 merges the combinations of second feature vectors corresponding to the combinations of embedding vectors that are highly similar. This achieves an effect similar to merging the first feature vectors generated from patient medical data to reduce the number of dimensions.

[0249] (Operation of machine learning model 310 during training on prediction server 500) Figure 31 is a flowchart illustrating the learning process of the machine learning model 310, which is performed by the learning control unit 540 of the prediction server 500.

[0250] In step S601 of Figure 31, the learning control unit 540 uses the training data included in the learning data set 360 to optimize the weights and biases of each neuron included in the embedding layer 313 and prediction unit 314 of the machine learning model 310.

[0251] In step S602, the learning control unit 540 swaps combinations of embedding vectors having a similarity of at least a predetermined second similarity with a predetermined probability. The cosine similarity mentioned above can be used as the similarity. For example, the predetermined second similarity is 0.6, and the predetermined probability is 1 / 2.

[0252] In this exemplary embodiment 5, in the initial state before training the machine learning model 310, there are three combinations of embedding vectors: {E1,E2}, {E2,E3}, and {E3,E1}. During the training process of the machine learning model 310, if there is a combination among these three with a cosine similarity of 0.6 or higher, that combination is swapped with a probability of 1 / 2.

[0253] As described above, during the learning process of the machine learning model 310, by swapping similar combinations of embedding vectors with a certain probability, the combinations of similar embedding vectors become even more similar as the learning of the machine learning model 310 progresses.

[0254] In detail, as the machine learning model 310 progresses, similar embedding vector combinations are swapped with a certain probability. If this were the only factor, the swapped combinations would result in a larger loss because they would be different from the originally optimized embedding vectors. However, by reducing the distance between similar embedding vectors, even when the embedding vector combinations are swapped, the input will be identical to the originally optimized embedding vectors, thus reducing the loss. As the machine learning model 310 learns this, similar embedding vector combinations become even more similar.

[0255] The subsequent steps S603 to S605 are the same as steps S502 to S504 of the exemplary embodiment 4 described above.

[0256] As described above, in the learning control unit 540 of the prediction server 500 in this exemplary embodiment 5, during the process of training the machine learning model 310, swaps combinations of embedding vectors having a similarity of a predetermined second similarity or higher with a predetermined probability. This achieves an effect similar to reducing the number of dimensions by merging the first feature vectors generated from patient medical data. As a result, the prediction accuracy of the machine learning model 310 is improved compared to the case where the number of dimensions is not reduced by merging the first feature vectors.

[0257] [Exemplary Embodiment 6] Next, a prediction server 600 according to exemplary embodiment 6 of this disclosure will be described.

[0258] (Functional configuration of Prediction Server 600) Figure 32 shows the functional configuration of the prediction server 600 according to this exemplary embodiment 6. In the prediction server 600, the learning control unit 340 included in exemplary embodiment 3 has been replaced with a learning control unit 640.

[0259] (Learning control unit 640) In the process of training the machine learning model 310 to predict the length of hospital stay for patients, the learning control unit 640 merges the second feature vector output from the merger layer 312 by changing the transformation rule from the first feature vector to the second feature vector in the merger layer 312.

[0260] In detail, during the process of training the machine learning model 310, the learning control unit 640 adds a correction value to at least one of the combinations of embedding vectors that have a similarity of a predetermined third similarity or higher, thereby further increasing the similarity of the embedding vector combination.

[0261] Specifically, when one of the embedding vector combinations is A and the other is B, a correction value is added to the embedding vector A according to the following formula.

[0262]

number

[0263] However, in the above equation, γ is a predetermined coefficient, and 0 < γ < 1.

[0264] Through the above operation, the machine learning model 310 is trained under conditions in which disturbances are applied that make similar combinations of embedding vectors even more similar. Then, the learning control unit 640 merges the second combination of feature vectors corresponding to the highly similar combinations of embedding vectors. This achieves an effect similar to merging the first feature vectors generated from patient medical data to reduce the dimensionality.

[0265] (Operation of machine learning model 310 during training on prediction server 600) Figure 33 is a flowchart illustrating the training process of the machine learning model 310, which is performed by the learning control unit 640 of the prediction server 600.

[0266] In step S701 of Figure 33, the learning control unit 640 optimizes the weights and biases of each neuron in the embedding layer 313 and prediction unit 314 of the machine learning model 310 using the training data included in the learning data set 360.

[0267] In step S702, the learning control unit 640 adds a correction value to at least one of the combinations of embedding vectors that have a similarity of at least a predetermined third similarity to further enhance the similarity of the embedding vector combinations. Here again, cosine similarity is used as the similarity. For example, the predetermined third similarity is 0.6.

[0268] As described above, during the training process of the machine learning model 310, by adding disturbances that make similar combinations of embedding vectors even more similar, the combinations of similar embedding vectors become even more similar as the machine learning model 310 progresses.

[0269] The subsequent steps S703 to S705 are the same as steps S502 to S504 of the exemplary embodiment 4 described above.

[0270] As described above, in the learning control unit 640 of the prediction server 600 according to this exemplary embodiment 6, during the process of learning the machine learning model 310, it adds a correction value to at least one of the combinations of embedding vectors having a similarity of a predetermined third similarity or higher to further make the combination of embedding vectors similar. This achieves an effect similar to reducing the number of dimensions by merging the first feature vectors generated from the patient's medical data. As a result, the prediction accuracy of the machine learning model 310 is improved compared to the case where the number of dimensions is not reduced by merging the first feature vectors.

[0271] In the above exemplary embodiment 2, the identification unit 220 identifies combinations of feature vectors in the pattern of Figure 15 whose change value of the prediction result is less than a predetermined fourth threshold as a combination of feature vectors that can be merged, but it is not limited to this. As a method to show that the difference in the prediction results of the provisional model 280 when the feature vectors are swapped and when they are not swapped is small, for example, instead of the change value of the prediction result, combinations of feature vectors whose similarity of the prediction result is greater than or equal to a predetermined fourth similarity may be identified as a combination of feature vectors that can be merged. More specifically, the prediction result is treated as a vector, and the similarity between the prediction result vector obtained by vectorizing the prediction result when the selected combination of feature vectors is input to the provisional model 280 without swapping them and the prediction result vector obtained by vectorizing the prediction result when the selected combination of feature vectors is swapped and input to the provisional model 280 is derived, and if the similarity of the derived prediction result vector is greater than or equal to the fourth similarity, it is identified as a combination of feature vectors that can be merged. The similarity between prediction result vectors can be shown, for example, by cosine similarity.

[0272] Similarly, in the exemplary embodiment 6 described above, if the similarity of the prediction results of the machine learning model 310 when the combination of embedding vectors is swapped is greater than or equal to a predetermined fifth similarity, the combination of second feature vectors corresponding to the combination of embedding vectors may be identified as a combination of second feature vectors that can be merged. The similarity of the prediction results refers to the similarity between the prediction result vector obtained by vectorizing the prediction results output by the machine learning model 310 without swapping the combination of embedding vectors, and the prediction result vector obtained by vectorizing the prediction results output by the machine learning model 310 with the combination of embedding vectors swapped. The similarity between prediction result vectors is indicated, for example, by cosine similarity.

[0273] Furthermore, while the above exemplary embodiment exemplified a case where two items, such as "age group" and "gender," were used as the mergible feature vectors, the embodiment is not limited to this. Three or more items, such as "age group," "gender," and "medical department," may also be specified as the mergible feature vector combinations.

[0274] Furthermore, in the above exemplary embodiment, the hardware structure of the Processing Unit, which executes various processes such as a specific unit, a rule generation unit, a merging unit, a model generation unit, a learning control unit, and a prediction control unit, can be any of the following types of processors. These processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a Programmable Logic Device (PLD) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacturing, and a dedicated electrical circuit, which is a processor with a circuit configuration specifically designed to execute a particular process, such as an Application Specific Integrated Circuit (ASIC).

[0275] Furthermore, the above various processes may be executed on one of these various processors, or on a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). Alternatively, multiple processing units may be configured on a single processor. An example of configuring multiple processing units on a single processor is the use of a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, such as a System on a Chip (SOC).

[0276] Thus, each processing unit is configured, in terms of its hardware structure, using one or more of the above-mentioned processors.

[0277] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits (Circuitry) that combine circuit elements such as semiconductor devices.

[0278] Furthermore, the technology disclosed herein extends not only to the operating programs for the data merging rule generator, the learning device, and the imaging device, but also to a computer-readable storage medium (such as a USB memory stick or DVD (Digital Versatile Disc)-ROM (Read Only Memory)) that non-temporarily stores the imaging device's operating program.

[0279] The disclosure of Japanese Patent Application No. 2021-137517, dated August 25, 2021, is incorporated herein by reference in its entirety.

[0280] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

Claims

1. A generator for generating data merging rules for machine learning models, The system comprises a processor and memory connected to or built into the processor, The aforementioned processor, A process to identify combinations of mergible feature vectors contained in a data set with correct labels, A rule generation process that generates a rule for merging the feature vectors based on the combination of mergeable feature vectors, Execute, In the specified process, the processor further creates an evaluation index that takes into account combinations of multiple items for the combination identified as the combination of mergeable feature vectors, and if the evaluation index that takes into account combinations of items falls within a predetermined range, the processor excludes the combination from the combination of mergeable feature vectors. A generator for generating data merging rules.

2. The data merging rule generating apparatus according to Claim 1, wherein the evaluation index is the similarity of the frequency distribution of the correct labels for each feature vector, and the predetermined range is a range less than a predetermined second threshold.

3. The data merging rule generation apparatus according to claim 1, wherein in the specified processing, the processor creates a frequency distribution of the correct label for each feature vector included in the data set, and identifies a combination of feature vectors whose similarity of the frequency distribution of the correct label is equal to or greater than a predetermined first threshold as the combination of feature vectors that can be merged.

4. The data merging rule generation apparatus according to claim 3, wherein in the specific processing, the processor creates a frequency distribution of the correct labels for a single item feature vector.

5. The data merging rule generating apparatus according to claim 1, wherein in the specified processing, the processor creates a frequency distribution of the correct level considering combinations of multiple items for each feature vector included in the data set, and identifies combinations of feature vectors whose similarity of the frequency distribution of the correct labels is equal to or greater than a predetermined seventh threshold as the combination of feature vectors that can be merged.

6. The data merging rule generation apparatus according to claim 1, wherein in the rule generation process, the processor terminates the generation of the merging rule when the number of combinations of mergeable feature vectors included in the merging rule exceeds a predetermined third threshold.

7. In the aforementioned specific process, the processor: A provisional model is generated and trained using the feature vectors included in the aforementioned data set as input. A data merging rule generating apparatus according to claim 1, comprising: selecting a combination of feature vectors from the data set; and identifying the selected combination of feature vectors as the merging-compatible combination of feature vectors when the change in the prediction result of the provisional model when the selected combination of feature vectors is swapped is less than a predetermined fourth threshold.

8. In the aforementioned specific process, the processor: A provisional model is generated and trained using the feature vectors included in the aforementioned data set as input. A data merging rule generating apparatus according to claim 1, comprising: selecting a combination of feature vectors from the data set; and identifying the selected combination of feature vectors as the merging-compatible combination of feature vectors when the similarity of the prediction results of the provisional model after swapping the selected combination of feature vectors is equal to or greater than a predetermined fourth similarity.

9. The data merging rule generating apparatus according to claim 1, wherein in the specified processing, the candidate for the mergable feature vector is determined based on at least one of the edit distance, distributed representation, or related information of the feature vector.

10. A display process that displays the combination of mergeable feature vectors on the display unit, A receiving process that receives from the user whether or not the aforementioned combination of mergeable feature vectors can be merged, A further device for generating data merging rules according to claim 1.

11. A learning device that trains a machine learning model using a set of training data merged according to a merge rule generated by the data merge rule generation device described in claim 1.

12. A prediction device that uses data merged according to a merge rule generated by the data merge rule generation device described in claim 1 as input to cause a machine learning model to make predictions.

13. A method for operating a device for generating data merging rules for machine learning models, A step of identifying a combination of mergible feature vectors contained in a data set having correct labels, A step of generating a merger rule for the feature vectors based on the aforementioned combination of mergeable feature vectors, Includes, The aforementioned identification step includes, for the combination identified as the combination of mergeable feature vectors, further creating an evaluation index that takes into account combinations of multiple items, and if the evaluation index that takes into account combinations of items falls within a predetermined range, excluding the combination from the combination of mergeable feature vectors. How to operate the data merging rule generation device.

14. A program that generates data merging rules for machine learning models, A step of identifying a combination of mergible feature vectors contained in a data set having correct labels, A step of generating a merger rule for the feature vectors based on the aforementioned combination of mergeable feature vectors, Have the computer run it, The aforementioned identification step includes, for the combination identified as the combination of mergeable feature vectors, further creating an evaluation index that takes into account combinations of multiple items, and if the evaluation index that takes into account combinations of items falls within a predetermined range, excluding the combination from the combination of mergeable feature vectors. program.

15. A learning device for machine learning models, The system comprises a processor and memory connected to or built into the processor, The aforementioned machine learning model includes a merger layer that transforms a first feature vector into a second feature vector and outputs it. The aforementioned processor, A learning process is executed to train the machine learning model using the second feature vector as input. The processor, in the learning process, merges the second feature vectors output from the merger layer by changing the transformation rule from the first feature vector to the second feature vector in the merger layer. The machine learning model further includes an embedding layer that outputs an embedding vector corresponding to the second feature vector, The processor further makes similar combinations of embedding vectors similar in the learning process. Learning device.

16. The learning apparatus according to claim 15, wherein the processor modifies the transformation rule in the merged layer in the learning process using an algorithm that assigns a score based on the value of the loss function used to train the machine learning model.

17. The learning device according to claim 16, wherein the score of the algorithm includes the number of the second feature vectors merged in the merge layer.

18. The learning device according to claim 16, wherein the initial value of the score of the algorithm is determined based on at least one of the edit distance, distributed representation, or related information of the first feature vector input to the merged layer.

19. The learning device according to claim 15, wherein the processor, in the learning process, introduces a term to the loss function used for learning the machine learning model that forces similar combinations of embedding vectors to become even more similar.

20. The learning device according to claim 15, wherein the processor, in the learning process, swaps combinations of embedding vectors having a similarity of a predetermined second similarity or higher with a predetermined probability.

21. The processor, in the learning process, uses a similarity of a predetermined third similarity or higher. The learning device according to claim 15, wherein a correction value is added to at least one of the combinations of embedding vectors having the following characteristics to further make the combination of embedding vectors similar.

22. The learning device according to claim 15, wherein the processor, in the learning process, merges the combination of second feature vectors that correspond to the combination of embedding vectors having a similarity of a predetermined first similarity or higher.

23. The learning device according to claim 15, wherein the processor, in the learning process, merges the combination of second feature vectors corresponding to the combination of embedding vectors when the change in the prediction result of the machine learning model when the combination of embedding vectors is swapped is less than a predetermined seventh threshold.

24. The learning device according to claim 15, wherein the processor, in the learning process, merges the second feature vector combinations corresponding to the embedding vector combinations when the similarity of the prediction results of the machine learning model when the combination of embedding vectors is swapped is equal to or greater than a predetermined fifth similarity.

25. A method for operating a machine learning model learning device, The aforementioned machine learning model includes a merger layer that transforms a first feature vector into a second feature vector and outputs it. The process includes the step of training the machine learning model using the second feature vector, The training step includes a step of merging the second feature vectors output from the merger layer by changing the transformation rule from the first feature vector to the second feature vector in the merger layer, The machine learning model further includes an embedding layer that outputs an embedding vector corresponding to the second feature vector, The learning step includes a step of further making similar combinations of embedding vectors similar, How to operate the learning device.

26. This is a program for training machine learning models. The aforementioned machine learning model includes a merger layer that transforms a first feature vector into a second feature vector and outputs it. The computer is made to perform the step of training the machine learning model using the second feature vector described above. The aforementioned learning step involves causing the computer to perform the step of merging the second feature vectors output from the merger layer by changing the transformation rule from the first feature vector to the second feature vector in the merger layer, The machine learning model further includes an embedding layer that outputs an embedding vector corresponding to the second feature vector, The learning step includes a step of further making similar combinations of embedding vectors similar, program.

Images