Ai-based event class discrimination method and discrimination system

Abstract

To provide a new determination method and system that enable AI processing in which a determination is made in line with viewpoints for experts to make determination, and can provide effective answers even with a small amount of learning data by mutually adjusting errors in expert answers and errors in AI answers.SOLUTION: A system is configured to cause teacher data for AI learning in which target events identified by identification codes are classified, and data related to targets for which an event class determination is required to be AI processed, and to receive the class determination data obtained by AI processing. The system is configured to input the teacher data for learning to a plurality of pieces of AI processing at random for learning, to input data related to the targets to each of the plurality of pieces of AI processing, to receive class determination data corresponding to each learning from each AI processing, and to store it in a storage device, thereby determining the class of the target event identified by the identification code based on each piece of class determination data corresponding to each identification code.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a method and an apparatus for classifying events to be classified based on an individual, an object, or information from data related to the individual, the object, or the information.

Background Art

[0002] It is not uncommon to classify certain events such as the rank, grade, risk, importance, etc. related to an individual, an object, or information from various data related to the individual, the object, or information.

[0003] The classification is generally determined based on a large number of data, and the credibility of the classification also increases by the judgment of experts on the types of data related to the events of objects such as individuals, objects, or information.

[0004] For example, in the medical field, based on the medical data of each individual as the object, the severity and criticality of the disease that each individual has as an event are comprehensively judged by medical practitioners such as doctors who are experts, thereby improving the accuracy.

[0005] In addition, the events of objects such as individuals, objects, or information change over time, and the classes to be classified also change according to the changes in the events.

[0006] Therefore, if it is possible to infer the future change of an event based on current or past data as a change in the class, it can contribute to better social activities such as prior risk avoidance and future preparation.

[0007] On the other hand, due to the improvement of AI technology this year, the improvement of the prediction and judgment system based on various data is being planned.

[0008] However, since the causal relationship between the inference for result output and the result of that AI is not clear, it has been an obstacle to its use in fields that require clarification of the causal relationship.

[0009] Therefore, at least, AI processing that has been judged from the perspective for an expert to make a judgment is required.

[0010] Furthermore, there are errors and fluctuations that subtly change according to the environment at that time even in the judgment of an expert. Moreover, when performing a certain classification, the judgment slightly differs among experts, and this becomes the individuality of each expert, resulting in differences also in the risk classification that is the answer result based on the individuality. Although there are points to respect that individuality, if a specific classification is made due to that individuality, it may rather reduce the prediction accuracy.

[0011] On the other hand, also in the judgment of AI, compared with a large amount of data called big data, the data for an expert to judge is relatively small, and thus the accuracy of the answer of the AI itself also has a range.

[0012] Therefore, a new method for adjusting the mutual relationship between the accuracy of an expert's answer and the accuracy of a judgment by AI while maintaining the individuality of an expert's judgment has been required.

Prior Art Documents

Patent Documents

[0013]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0014] The present invention provides a new judgment method and its system from such a perspective, enabling AI processing that is judged along the perspective for an expert to make a judgment, and further providing a new judgment method and system that can adjust the mutual error between the expert's answer and the AI's answer to enable an effective answer even with less learning data.

Means for Solving the Problem

[0015] According to the present invention, there is provided a system configured to classify teacher data for AI (Artificial Intelligence) learning that classifies target events specified by identification codes, perform AI processing on data related to a target for which class judgment of an event is required, and receive class judgment data obtained by the AI processing.

[0016] The system is configured to randomly input the teacher data for learning into a plurality of AI processes for learning respectively, input the data related to the target into each of the plurality of AI processes respectively, and receive class judgment data corresponding to each learning from each of the AI processes. Thus, based on each class judgment data corresponding to each identification code, the class of the target event specified by the identification code is determined.

[0017] Here, the target can be a person, the event is the severity related to the health of the person, and the severity is classified into a plurality of classes.

[0018] Furthermore, the determination of the class of the target event specified by the identification code can be the class to which the largest number of the plurality of class judgment data obtained from each AI process belongs.

[0019] Furthermore, it is determined whether a plurality of class judgment data obtained from each AI process meet a predetermined condition, and the teacher data corresponding to the plurality of class judgment data that meet the predetermined condition is deleted from the overall teacher data to be used as the teacher data for AI learning.

[0020] The above-mentioned predetermined conditions may be based at least on the variance among a plurality of class determination data obtained from each AI process. For example, it may be conditioned that the variance among a plurality of class determination data obtained from each AI process is equal to or less than a predetermined degree.

[0021] Furthermore, the above-mentioned predetermined conditions may be added with the condition that the number of mismatches between the class indicated by the teacher data and the class indicated by the class determination data obtained by a plurality of AI processes corresponding to the teacher data is equal to or more than a predetermined number.

[0022] Here, the class of the target event is a class of an event that changes according to the passage of a temporal period, and the class determination of the event obtained by the AI process can also be the class of the event expected after the passage of a temporal period. For example, in the medical field, it can be used to predict the class of severity that changes annually.

[0023] Furthermore, the present invention provides a method, the method comprising: randomly inputting teacher data for AI learning that classifies a target event specified by an identification code stored in a storage device into a plurality of AI processes for learning respectively; inputting data regarding a target for which a class determination of an event is required into each of the plurality of AI processes by an information processing device, and receiving class determination data corresponding to each learning from each of the AI processes; and determining the class of the target event specified by the identification code based on each class determination data corresponding to each identification code.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0025] Figure 1 shows system 1 according to an embodiment of the present invention. As an embodiment, the information processing apparatus 100 is connected to the network 20. A user can access the information processing apparatus 100 via the information terminal 30 and receive the services provided by the information processing apparatus 100. The access is made via a wired or wireless network 20 such as the Internet, a dedicated line, or an in-house line. There is no particular limitation on the number of accessible information terminals, and they can be appropriately used according to the purpose of use. The information processing apparatus 100 is configured to receive input data from the information terminal 30 via the network 20 and provide corresponding services.

[0026] The source data server 10 is configured to provide the information processing apparatus 100 with the necessary data via the network 20 according to the service content of the information processing apparatus 100. If necessary, mutual authentication and agreement formation regarding data handling may be performed in advance among the information processing apparatus 100, the source data server, and the information terminal 30.

[0027] The AI server or server engine 200 can be configured by combining hardware such as a CPU, a multi-processor equipped with a plurality of processor cores, a GPU (Graphics Processing Units), a DSP (Digital Signal Processors), and an FPGA (Field-Programmable Gate Arrays), and can be composed of algorithms or learned models that realize machine learning functions, language analysis functions, and further voice recognition functions. It is also possible to combine a quantum processor, and the machine learning function can be achieved, for example, by a neural network including deep learning (deep neural network) or reinforcement learning. Note that the server can also be composed of one or more servers, and each component such as the AI engine unit and the storage unit may be distributed.

[0028] Next, with reference to FIG. 2, the hardware configuration of the information processing apparatus 100 will be described.

[0029] An information processing apparatus 100 according to an example of this embodiment includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a storage unit, a communication unit including a network I / F (Interface), a display unit, an input unit, etc. Optionally, it may include other hardware configurations.

[0030] The CPU is an arithmetic unit that reads programs and data stored in the ROM, storage unit, etc., stores necessary data on the RAM, and executes processing to realize the control and functions of the information processing apparatus 100.

[0031] The storage unit is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) for storing an OS (Operating System), various application programs, etc.

[0032] The communication unit provides an interface between the information processing apparatus 100 and the network 20, and has a communication interface function for performing information communication with external devices such as an information terminal 30, a source data server 10, and an AI server 200 connected to the network 20.

[0033] The display unit is a display device such as a display, and displays the processing result by the information processing apparatus 100 to the user. The input unit can be an input device such as a keyboard, a mouse, a camera, a microphone, etc.

[0034] The bus transmits an address signal, a data signal, various control signals, etc. to connect each component in the information processing apparatus 100.

[0035] Note that the information processing apparatus 100 is not limited to the above configuration. It may be realized in a distributed manner by separate computers, or each component of the information processing apparatus 100 may be distributed and function integrally. Also, each element constituting the information processing apparatus 100 may be single or composed of a plurality of combinations. Furthermore, depending on the required service content, necessary applications may be distributed between the information terminal 30 and the information processing apparatus 100 so that services are realized in cooperation.

[0036] Also, the system is not limited to the hardware configuration shown in this embodiment, and may be in any form as long as it can implement the present invention. Also, an external AI processing function provided by a private entity or the like may be used.

[0037] FIG. 3 shows an example of the functions of the information processing apparatus 100. FIG. 4 shows various data stored in the storage unit of the information processing apparatus 100, and FIGS. 5 to 9 show a series of main processing flows according to the embodiment of the present invention.

[0038] The data collection unit 110 collects data necessary for classifying the target event (step S110 in FIG. 5) and stores it as ID-specific data D110 that can be managed by identification data (ID).

[0039] The main factor selection unit 120 enables a user who uses the method and system of the present invention to select factors used for classifying. Instead of the user, the system 1 may automatically determine and select effective factors. The main factor data extraction unit 130 extracts data for each ID related to the factor (hereinafter also referred to as "ID-specific factor data" or simply "factor data") from the factors selected by the main factor selection unit 120 from the data collected by the data collection unit 110 and stores it as ID-specific factor data D120 (step S120).

[0040] Note that the factors may be selected in advance, and based on the selected factors, the necessary factor data may be collected from the source data server 10.

[0041] Identification data is data that can uniquely identify targets such as individuals, companies, objects, and information. The collected data and the extracted ID-specific factor data D110 include various types of data that can be associated with the identification data.

[0042] For example, in the case of an individual, the identification data may be the name itself, or an identification code that can identify the name (for example, a patient ID when visiting a hospital, an account number during a bank transaction, an ID used when purchasing goods through online shopping, etc.). Of course, as long as it can identify the individual, company, etc., a number of IDs, accounts, etc. are acceptable, and the data associated with them may be collected and integrated as a whole.

[0043] The various types of data that can be associated with the identification data can be of any type as long as they can be used to classify certain events related to targets such as individuals, companies, objects, and information from those data.

[0044] For example, taking the target as a company, the events can be classified as the soundness of each company, and the various types of data can be any information related to the soundness of the company.

[0045] The classifying teacher determination unit 140 uses some of the ID-specific factor data D120 extracted from the ID-specific data D110 as the determination of the class (step S130). Alternatively, the ID-specific factor data may be directly extracted from the ID-specific data D110 and used as the determination of the class. In order to enable a reliable class determination, it is preferably performed by an expert or knowledgeable person who is proficient in the practice related to the data (hereinafter, also simply referred to as an "expert").

[0046] A professional in practice may input the class discrimination result into the information processing device 100 via the information terminal 30. For this purpose, the information processing device 100 displays the extracted factor data to the professional via the information terminal 30, and the professional can input the class discrimination result into the information terminal 30 based on the displayed data and their own experience and knowledge, and store it in the information processing device 100.

[0047] Preferably, there are multiple experts who perform class discrimination, and the information processing device 100 stores the class discrimination results by each of the multiple persons in the information processing device 100.

[0048] Therefore, step S130 performed by the class classification teacher determination unit 140 may be realized as a simple interface application that can provide factor data from the information processing device 100 to the information terminal 30 so that an expert can perform class discrimination, and can provide discrimination data from the information terminal 30 to the information processing device 100, or may be realized as a simple application that allows an expert to input discrimination data into the information processing device 100. The class discrimination may be independently performed repeatedly by different experts based on the same information based on the same identification code.

[0049] This class discrimination result can be input into the AI and used as learning data. Here, it can be called teacher determination data D130 as a generalized name.

[0050] The teacher determination data D130 may be discrimination data for classes that can be predicted in the future in addition to the discrimination of the current class based on the factor data. Future prediction includes predictions for the next fiscal year, the next month, etc., and there is no particular limitation on how long the future period is. Also, the future period may be set appropriately according to the period to which the collected data belongs.

[0051] The teacher judgment data noise removal unit 150 excludes data whose class discrimination results are treated as abnormal, such as specific data or abnormal data, from the teacher judgment data input for learning, for example, data having noise, from the entire teacher judgment data.

[0052] An embodiment of this exclusion method performed by the teacher judgment data noise removal unit 150 will be described with reference to FIGS. 6 and 7.

[0053] First, the teacher judgment data D130 is input to AI processing such as the AI server 200, and learning is performed according to the content of the teacher judgment data D130. In addition, the ID-specific factor data D120 is input to the AI processing as an object of class prediction, and is output to the information processing apparatus 100 as AI judgment data D140 accompanied by class discrimination according to the learning (step S140).

[0054] Here, as shown in FIG. 5, the ID-specific factor data used for the discrimination by experts as the teacher judgment data D130 can be data in which a part is extracted from the above ID-specific factor data D120. However, the main factor data extraction unit 130 may separately collect the ID-specific factor data D120 and the ID-specific factor data used for the discrimination by experts from an external server. That is, how and through what path the ID-specific factor data D120 used for determination by AI and the ID-specific factor data input to AI for learning are acquired and prepared is not particularly limited.

[0055] The AI judgment data D140 can accompany the class by teacher judgment for each individual identification code (ID), that is, the class discriminated by an expert and the class discriminated by AI.

[0056] The learning of this AI and the discrimination by the AI are preferably performed by a plurality of AI processes. Also, the input of data to a plurality of AIs is preferably performed randomly. The teacher determination data D150 and the ID-specific factor data D120 input randomly are individually processed by a plurality of AIs (step S140), and the individual AI determination data is received and stored in the information processing apparatus 100 as the output for each AI. Then, from those AI determination data, a plurality of AI determination data D140 for each identification code are created.

[0057] Next, for each identification code, teacher determination data determined as an outlier is deleted from the data of class classification by teacher determination and class classification by each AI. FIG. 7 shows an example of deleting teacher determination data determined as class 1 for ID 0002 from that determination (step S150 in FIG. 7).

[0058] The determination of an outlier is, for example, when a predetermined difference or distance occurs between the class by AI determination and the class by teacher determination, the teacher determination data is processed as noise and removed from the entire teacher determination data (step S150).

[0059] For example, when the classes are divided into 1 to 4 and numbered, if there is a predetermined value, for example, a difference of 3 or more between the class by AI determination and the class by teacher determination, it can be determined as an outlier. Of course, the method for making an abnormal class determination is not limited to this, and it goes without saying that other appropriate determination methods can be applied. By managing the classes as numbers in this way, the information processing apparatus 100 can execute it as a mathematical operation process.

[0060] When making predictions by a plurality of AIs, the collective knowledge theorem may be used. The collective knowledge theorem can be expressed by the following formula. Group error = Average individual error - Variance value

[0061] The group error is the difference between the average of the values estimated by the members of a certain group and the correct answer. The average individual error is the average value of the errors of each member, and the variance value is the variation of the estimated values of each member. Mathematically, Let the estimated value of member i be Xi (i = 1, 2,..., N) When the group estimated value is A and the true value is R, Group error = (A - R) 2 A = {X(1) + X(2) + ··· + X(N)} / N Average individual error = { (X(1) - R) 2 + (X(2) - R) 2 + ··· + (X(N) - R) 2} / N Variation (variance value) = { (X(1) - A) 2 + (X(2) - A) 2 + ··· + (X(N) - A) 2} / N It can be expressed as follows.

[0062] What this collective wisdom theorem shows is that the individual's estimation error (the first term) in the group is offset by the diversity (the second term), and the group can make an estimation closer to the correct answer.

[0063] When applied to the present invention, when AI(1) to AI(n) (n is an integer), which are AI servers or AI engines, are used as each member, and the ranks predicted and determined by each member are used as the estimated values, the determination results of the plurality of AIs should show a distribution centered on the correct answer. According to this collective wisdom theorem, the larger the variance value of the determination results of each member, the higher the accuracy of the solution shown by the average of the group becoming the correct answer compared to the determination results of individual AIs.

[0064] In order to delete the data of abnormal class determination in the training data and reduce the overall error, based on this collective wisdom theorem, since it can be conditioned that the error of the AI is small and the variance value is large, the data to be deleted can be limited to the data with a large error and a small variance value or standard deviation.

[0065] Therefore, in this embodiment, as one condition, when the dispersion of the ranks determined by the AI is within a predetermined range, for example, when its standard deviation is equal to or less than a certain threshold, the teacher data corresponding to the corresponding AI determination is deleted from the entire teacher data. Or, in addition to that condition, when a predetermined number or more of AIs have a difference from the class of the teacher data among the class predictions of multiple AIs and the class of the teacher data, the teacher data may be deleted from the entire teacher data on the condition that the teacher data is an outlier.

[0066] For example, assuming the number of AIs is 10, when the standard deviation of the determinations among the AIs is 0.5 or less and the number of AIs with a discrepancy between the AI's judgment and the teacher data is 7 or more, the teacher data may be removed on the grounds that it has outliers due to unacceptable fluctuations or blurs.

[0067] It should be understood that the terms "abnormal" or "outlier" used here are for convenience in removing teacher judgment data under specific conditions, and the term "abnormal" is used to indicate the target for excluding teacher data that is generally judged to be unsuitable for AI processing.

[0068] As shown in FIG. 7, the teacher judgment data from which the teacher judgment data determined to be abnormal has been removed is stored as the adjusted teacher judgment data D150.

[0069] Referring to FIG. 8, the post-removal data class determination unit 160 inputs the adjusted teacher determination data D150, which includes a plurality of factor data related to a plurality of identification codes and from which abnormal data has been removed, as learning data to the aforementioned AI server 200. Further, a group of factor data D120 by ID related to one or a plurality of identification codes for which classification is desired is also input to the AI server 200. By AI processing, based on that group of factor data by ID, an AI discrimination for classification corresponding to each identification code is executed (step S160). The post-removal data class determination unit 160 receives the discrimination result by the AI and stores it as AI determination data D160.

[0070] At this time, it is preferable to input the adjusted teacher determination data D150 and the factor data D120 by ID to a plurality of AIs (1) to AI(n) for prediction and discrimination. Also, it is preferable to input the data to the plurality of AIs randomly. The randomly input adjusted teacher determination data D150 and the factor data D120 by ID are individually processed by the plurality of AIs, and the adjusted individual AI determination data is stored in the information processing apparatus 100 as the output for each AI. Then, from those AI determination data, the final class AI determination data D160 by identification code is created.

[0071] As shown in FIG. 9, the final class determination unit 170 performs the final class determination (step S170). The final discrimination class data D170 is created from the class determinations of AIs (1) to AI(n) for each identification code listed in the final class AI determination data D160. This class determination criterion may be determined from the central average of the predicted classes of AIs (1) to AI(n), or the most frequent class may be determined as the final class, or other appropriate determination methods may be used.

[0072] Next, an example of applying the present invention to the medical field will be described.

Example

[0073] (Preliminary confirmation of the effectiveness of AI Part 1) Before implementing the present invention, a preliminary check was conducted to determine whether the AI utilization method based on the basic idea of the present invention is effective.

[0074] Figure 10 shows the state of people who have undergone a health check-up, divided into four layers as classes, and outlines the relationship between the number of people in each layer and the medical expenses.

[0075] Starting from the lower layer, class numbers are assigned as Class 1, Class 2, Class 3, and Class 4. Class 1 is expected to continue to improve health in the future, Class 2 is mild, Class 3 is moderate, and Class 4 is severe.

[0076] These four layers can also be used to assess the risk of illness for individuals and can be understood as levels of progression to severity. The top layer has the highest level of progression to severity, and the bottom layer can be judged to have almost no level of progression to severity. Therefore, this classification is also a risk classification, and the present invention can be used for risk management.

[0077] When this classification is performed, depending on the method of classification, the proportion of the number of people in each layer in the total number of people forms a roughly pyramid shape of 50%, 30%, 15%, and 5% from the bottom up. On the other hand, the proportion of medical expenses of each layer in the total medical expenses forms an inverted pyramid distribution of 40% for Class 4 in the topmost layer, 35%, 20%, and 5% as going down.

[0078] Therefore, if the occurrence of Class 4 and Class 3 with a population composition of 5% and 15% can be predicted and prevented, more appropriate care can be provided to people who may become severe, and the total medical expenses can also be reduced.

[0079] Therefore, first, in order to verify the effectiveness of the present invention in advance, 250 health check and medical data (hereinafter simply referred to as "medical data") were obtained from medical institutions, and the medical data from 2014 was visually examined by doctors and other experts (also simply referred to as "experts") engaged in medicine, and the risk level of progression to severity of each individual at the time of 2015 in the following year was predicted.

[0080] Then, the risk numbers were assigned numbers from 1 to 4, and the medical data of each person with risk numbers 1 to 4 were input into the AI as learning data. Furthermore, for the learned AI, the medical data of 50 people in 2014 were input from among 250 people, and the risk of deterioration in 2015 was predicted.

[0081] As a result, among the 50 people, there were 8 differences between the prediction by experts for 2015 and the prediction by the AI for 2015.

[0082] Furthermore, the severity in 2015 was judged by experts from the medical data of 50 people in 2015, and when the actual results were compared with the prediction data of both experts and the AI, among the 8 differences, the severity of 2 cases predicted by the AI matched the actual results.

[0083] From this, although it is small-scale data, the possibility of the effectiveness of the AI was confirmed.

Example

[0084] (Preliminary Confirmation of the Effectiveness of the AI Part 2) The data obtained from medical institutions cover a wide variety of types. However, if factors affecting each disease that can be obtained through individual health guidance, and items that can be obtained from receipt data and health examination data are narrowed down to a certain number and identified as major factors, predictions can be made for a wide range of general medical recipients, the scope of use will expand, and the prediction accuracy can also be improved.

[0085] From this perspective, although the process will not be described in detail here, 49 items of factor data were used.

[0086] First, the information processing device 100 collected data of "Medical Treatment Information Details", "Nursing Care Benefit Details", "Specific Health Examination Results", and "Insured Person Master" from the data servers 10 (Figure 1) of various medical institutions to construct ID-specific data D110, and created and stored 2,699 pieces of ID-specific factor data D120 having the factor data of 49 items for 2014 that were selected.

[0087] Still, FIG. 11 shows the corresponding year of the data collected from each medical institution.

[0088] Next, 2,699 pieces of factor data D120 by ID stored in the information processing apparatus 100 were read out, and the contents of the factor data D120 by ID were displayed via the information terminal 30 used by 25 experts, and clinical inferences by the experts based on each of the 49 items of data in 2014 were performed.

[0089] The risk prediction for 2015 obtained by the clinical inference was input to the information terminal 30 and stored in the information processing apparatus 100 as teacher determination data D130 for each of the 2,699 pieces of ID having the risk prediction.

[0090] Furthermore, as shown in FIG. 13, the information processing apparatus 100 input the 2,699 pieces of teacher determination data D130 as AI learning data to the AI for learning.

[0091] Also, 6,707 pieces of factor data D120 by ID, which are the actual data in 2014 collected and extracted by the information processing apparatus 100, were input to the AI server 200 as the data used for the risk prediction in 2015.

[0092] As a result, the information processing apparatus 100 received 6,707 pieces of prediction data (class prediction data for 2015) predicted by the AI server 200 from the AI server 200.

[0093] In order to confirm the accuracy of the result predicted by the AI server 200, the experts performed risk classification by clinical inference based on the 6,707 pieces of actual data in 2015, and compared the result (teacher class actual determination based on the 2015 actual data) with the AI prediction (2015 class prediction).

[0094] The result was that the matching rate of the classes was 58.90%, and the ratio of each class was as shown in FIG. 13.

[0095] From the above, it was understood that it was possible to predict the severity of the next year from the data of a certain year, but a prediction with higher accuracy was also desired.

Example

[0096] (Elimination of unnecessary fluctuations and blurs by multiple AI judgments and improvement of prediction)

[0097] As a result of investigating the content of the teacher data, it was found that some experts had variations in the severity prediction or made severity predictions far from others, which, as fluctuations in the teacher data, also affected the accuracy of the AI prediction.

[0098] Therefore, as described in the flowcharts of FIGS. 6 to 9, multiple AI prediction results by multiple AIs were utilized, compared with the expert prediction, and the final severity prediction was made from the re-AI prediction based on the data excluding the severity predictions determined to be abnormal by the comparison.

[0099] Using the teacher judgment data D130 from the 2,699 ID-specific factor data D120 described in Example 2 and the 6,707 ID-specific factor data D120 collected and extracted by the information processing apparatus 100, the order of the data in the teacher judgment data D130 was randomly input from the information processing apparatus 100 to multiple AIs, and AI predictions were made. The severities by the multiple AIs obtained as a result were compared with the predicted severities of the corresponding experts for each ID.

[0100] The comparison was made not only between the expert judgment data and the judgment data of each of the AIs (1) to AI(n), but also between the AIs (1) to AI(n). As shown in the list of FIG. 14, based on the set knowledge theorem already described, when the number of cases where the expert judgment data did not match the AI judgment data was 7 or more and the deviation between the judgments of the AIs was less than 0.5 in standard deviation, the expert judgment data was deleted.

[0101] Note that the numbers shown here are examples, and any value can be determined as an outlier.

[0102] In this way, following the steps shown in FIG. 7, teacher data determined to have outliers was deleted from the entire 2,699 pieces of teacher data, and 2,228 pieces of new teacher data D150 were created.

[0103] From the new teacher data D150 and the 6,707 pieces of factor data by ID for the year 2014, as in the steps shown in FIG. 8, the information processing apparatus 100 randomly caused a plurality of AIs to learn and received determination results predicted from the AI server.

[0104] As shown in FIG. 15, as a result of comparing the 6,707 pieces of class prediction data for the year 2015 output from the AI server with the class actual determination by an expert based on the actual data for the year 2015, for all AIs (1) to AI(n) (n = 10), a significant improvement of 83% or more was observed.

[0105] This indicates that AI prediction from the perspective of general medical staff is possible, and shows the basis for being widely applicable to the medical field.

[0106] In addition, since the four layers described can also be used to measure the risk of illness to a person, this classification corresponds to a risk classification, and each layer can also be recognized as a risk layer. Therefore, it can be understood that the AI prediction based on the classification according to the present invention is also effective for risk management.

[0107] Although the embodiments of the present invention have been described as above, various alternative examples, modifications, or variations are possible for those skilled in the art based on the above description, and the present invention includes the various alternative examples, modifications, or variations described above without departing from the gist thereof.

Explanation of Reference Numerals

[0108] 10 Provider data server 10 20 Network 30 Information terminal 100 Information processing apparatus 200 AI Servers or Server Engines

Claims

1. A system for performing AI (Artificial Intelligence) learning on teacher data in which an event of a target identified by an identification code is classified, and AI processing on data on a target for which a class judgment of an event is required, and receiving class judgment data obtained by the AI ​​processing, The learning teacher data is randomly input to each of a plurality of AI processes to learn the data, and the data on the target is input to each of the plurality of AI processes to receive class judgment data corresponding to each learning from each of the AI ​​processes. Thus, the class of the target event identified by each identification code is determined based on the respective class judgment data corresponding to each identification code.

2. The system according to claim 1, wherein teacher data corresponding to a plurality of class judgment data that meet a predetermined condition is deleted from the entire teacher data to be used as teacher data for AI learning.

3. The system of claim 2, wherein the predetermined condition is based at least on a variance between the multiple class judgment data obtained from each AI process, or at least on whether the variance between the multiple class judgment data obtained from each AI process is less than a predetermined degree.

4. The system according to claim 3, further comprising a condition that the number of mismatches between the class indicated by the teacher data and the class indicated by the class judgment data obtained by the plurality of AI processes corresponding to the teacher data is equal to or greater than a predetermined number.

5. The system according to claim 1, wherein the class of the target event is a class of event that changes over a time period, and the class judgment of the event obtained by the AI ​​processing is also a class of event predicted after the time period has passed.

6. The system according to claim 1 or 2, wherein the class of the target event identified by the identification code is determined as the class to which the plurality of class judgment data obtained from each AI process most frequently belongs.

7. The system of claim 1 , wherein the subject is a person and the event is a severity of the person.

8. Randomly inputting training data for AI learning, which is obtained by classifying target events identified by the identification codes stored in the storage device, into a plurality of AI processes by the information processing device, and causing the AI ​​processes to learn the training data; Further, data on a target for which a class judgment of an event is required by the information processing device is input to each of the plurality of AI processes, and class judgment data corresponding to each learning is received from each of the AI ​​processes, A method in which a class of the target event identified by each identification code is determined based on respective class judgment data corresponding to each identification code.

9. The method according to claim 8, further comprising deleting teacher data corresponding to a plurality of class judgment data that meet a predetermined condition from the entire teacher data to be used as teacher data for learning the AI.

10. The method according to claim 9, wherein the predetermined condition is based at least on a variance between the multiple class judgment data obtained from each AI process, or at least on whether the variance between the multiple class judgment data obtained from each AI process is equal to or less than a predetermined degree.

11. The method according to claim 10, further comprising the condition that the number of mismatches between the class indicated by the teacher data and the class indicated by the class judgment data obtained by the plurality of AI processes corresponding to the teacher data is equal to or greater than a predetermined number.

12. The method according to claim 8, wherein the class of the target event is a class of an event that changes over a time period, and the class judgment of the event obtained by the AI ​​processing is also a class of an event predicted after the time period has passed.

13. The method according to claim 8 or 9, wherein the class of the target event identified by the identification code is determined as the class to which the plurality of class judgment data obtained from each AI process most frequently belongs.

14. The method of claim 8, wherein the subject is a human and the event is a severity of the human.

Images