Repair recommendation system and repair recommendation method

Abstract

To provide a repair recommendation system and a repair recommendation method that can present information relating to validity of recommendation information to a user.SOLUTION: A repair recommendation system 1 according to one embodiment of the present invention includes: a repair recommendation engine unit 22 that uses a learning model 12 that receives input of failure information concerning an apparatus and outputs recommendation countermeasure contents for dealing with a failure to infer recommendation information including a plurality of countermeasure contents for the input failure information and a certainty degree of each countermeasure information; and an input / output processing unit 21 that classifies, on the basis of the certainty degree, the countermeasure contents into one of a plurality of determination groups that are provided according to a level of the certainty degree, identifies a case resolution rate that is, in advance, associated with the classified determination group, and outputs information relating to magnitude of the identified case resolution rate as validity information of the recommendation information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a repair recommendation system and a repair recommendation method. [Background technology]

[0002] In recent years, AI (Artificial Intelligence) has been put to practical use in various fields, and for example, in the industrial field, a maintenance support system for facilities and equipment that utilizes AI technology has appeared. Patent Document 1 discloses a repair recommendation system that uses a machine learning model to infer recommendation information including multiple recommended repairs and a confidence level indicating the degree of accuracy of each repair for input information related to an event, and outputs the inference results as recommendation information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-22205 Summary of the Invention [Problem to be solved by the invention]

[0004] In systems that output information on recommended repairs, such as the repair recommendation system of Patent Document 1, information such as the TopN accuracy rate in the recommendation information is generally used as an index of the validity of the learning model. The TopN accuracy rate is the probability that the countermeasure content output as the recommended information is included in the top N of the countermeasure contents actually implemented in past cases (test data) input to the learning model as test cases. As the TopN accuracy rate, the accuracy rate of the entire test data input to the learning model is often used.

[0005] However, in the actual operation of a repair recommendation system, there are cases where low-quality input information is input. In principle, if the quality of the input information is low, that is, if the input information does not contain sufficient information to grasp the symptoms of the failure, it is expected that the validity of the recommended repair information will be low. However, in a learning model, validity is judged based on the Top N accuracy rate for the entire input test data, so in cases where the accuracy of inference is low due to the low quality of the input information, the validity cannot be correctly grasped.

[0006] Even when low-validity recommendation information is presented in this way, if the user trusts the Top N accuracy rate information of the learning model, the user will order the replacement parts, etc., output as recommended information. As a result, the failure will not be handled appropriately and repairs will not be completed. Furthermore, when a recommendation system outputs low-accuracy (low-validity) recommendation information in this way, the user's trust in the recommendation system and customer satisfaction will also decrease.

[0007] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to make it possible to present information regarding the validity of recommended information to a user. [Means for solving the problem]

[0008] A repair recommendation system according to one embodiment of the present invention includes a recommendation calculation unit that uses a learning model in which failure information on a broken device is input and recommended countermeasures for dealing with the failure as output to infer recommendation information including a plurality of countermeasures for the input failure information and a confidence level indicating the accuracy of each countermeasure, and an input / output processing unit that classifies the countermeasures into one of a plurality of judgment groups established according to the confidence level based on the confidence level associated with the countermeasure information, identifies a case resolution rate based on the countermeasures that is pre-associated with the classified judgment group, and outputs information regarding the magnitude of the identified case resolution rate as validity information for the recommendation information. Effect of the Invention

[0009] According to at least one aspect of the present invention, it becomes possible to present information on the validity of recommended information to a user. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a schematic configuration example of a repair recommendation system according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram showing an example of the hardware configuration of a repair recommendation server and a terminal device constituting a repair recommendation system according to an embodiment of the present invention; [Diagram 3] FIG. 13 is a diagram illustrating an example of a configuration of replacement part candidates and certainty factors according to an embodiment of the present invention. [Figure 4] FIG. 1 is a diagram illustrating an overview of a method for calculating a TopN case resolution rate according to an embodiment of the present invention. [Diagram 5] 1 is a diagram showing, in list form, an overview of a method for calculating a judgment coefficient that is a classification criterion for judgment groups according to an embodiment of the present invention, and an example of classification into judgment groups. FIG. [Figure 6] FIG. 13 is a diagram showing an example of the configuration of a judgment group table according to an embodiment of the present invention. [Figure 7] FIG. 13 is a diagram illustrating an example of a configuration of a case resolution rate table according to an embodiment of the present invention. [Figure 8] FIG. 13 is a diagram showing an overview of an extraction process of a case resolution rate based on recommendation information in one embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing an example of display of recommendation information displayed on the screen of a terminal device 30 when the TopN case resolution rate is low according to an embodiment of the present invention. [Figure 10] 10 is a sequence chart showing an example of a procedure of a repair recommendation process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, examples of modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. The present invention is not limited to the embodiments, and various numerical values ​​in the embodiments are merely examples. In addition, in this specification and drawings, the same components or components having substantially the same functions are given the same reference numerals, and duplicated explanations will be omitted.

[0012] <Configuration of repair recommendation system> First, the configuration of a repair recommendation system according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the schematic configuration of a repair recommendation system 1 according to an embodiment of the present invention.

[0013] As shown in FIG. 1, the repair recommendation system 1 includes an offline learning unit 10, a repair recommendation server 20, and a terminal device 30. The offline learning unit 10 and the repair recommendation server 20 are devices operated and managed by a company that provides the repair recommendation service. The terminal device 30 is a device operated by a user U that receives the repair recommendation service. The repair recommendation service is a service that provides recommended countermeasures for a malfunction based on device malfunction information input by the user U. In FIG. 1, the solid arrows indicate the flow of data, control signals, etc., and the dashed arrows indicate the direction of information reference.

[0014] The offline learning unit 10 has a learning model (recommendation model) 12 constructed by supervised learning (machine learning) based on learning data 11. The learning data 11 is composed of input information and output information selected from equipment failure history information data. The input information is composed of failure information indicating the equipment failure status, and the output information is composed of information such as replacement parts for failed parts and countermeasures.

[0015] The learning model 12 compares the output recommendation information for each case of a past event input as input information with the content of the action actually taken in that event to determine whether the recommendation information is correct or incorrect. Based on such learning, the learning model 12 infers recommendation information based on the input information. For example, various methods such as a Bayesian network, a support vector machine, and deep learning can be used for learning by the learning model 12.

[0016] The recommendation information is composed of information on a plurality of countermeasures recommended for the input information, i.e., the failure information, and information on the certainty of each countermeasure. In the following explanation, an example will be given in which the countermeasure is a replacement part candidate for a faulty part. The certainty of the replacement part candidate as the countermeasure is an index showing whether sufficient information necessary for the recommendation is provided when the learning model recommends the countermeasure from the input information to the repair recommendation system, and takes a high value when a word that is strongly linked to a specific faulty part, countermeasure part, or replacement part is provided. Examples of such words include "pressure drop" and "x-digit error code", and from a human perspective, the word corresponds to a word that can be determined as the part if it is a keyword.

[0017] The repair recommendation server 20 is a so-called application server, and includes an input / output processing unit 21 and a repair recommendation engine unit 22. The repair recommendation engine unit 22 corresponds to a so-called recommendation application, and a trained learning model 12 is deployed in the repair recommendation engine unit 22.

[0018] The repair recommendation engine unit 22 (an example of a recommendation calculation unit) outputs output information Ir (recommendation information) based on input information Ii transmitted from the terminal device 30 and received by the input / output processing unit 21.

[0019] The input / output processing unit 21 performs processing to transmit the input information Ii transmitted from the terminal device 30 to the repair recommendation engine unit 22 via the network N. Furthermore, based on information on the confidence levels of a plurality of replacement part candidates output from the repair recommendation engine unit 22, the input / output processing unit 21 refers to the judgment group table 210 and the case resolution rate table 211 to acquire information on the case resolution rate corresponding to the recommendation information.

[0020] Then, the input / output processing unit 21 generates recommendation validity information according to the degree of the case resolution rate, and transmits the generated recommendation validity information together with the recommendation information to the terminal device 30. The recommendation validity information is information indicating the degree of validity of the recommendation information. The recommendation validity information will be described in detail with reference to FIG. 9 described later.

[0021] The case resolution rate is the ratio of the number of cases in which the repair has been completed, i.e., the event of a failure has been resolved, to all test data (past failure cases) input to the learning model 12. Specifically, the case resolution rate is indicated by the proportion (%) of M replacement parts that are correct solutions to equipment failures among replacement part candidates with the highest confidence levels within the top N out of all replacement part candidates output as recommended information.

[0022] The judgment group table 210 (referred to as the judgment Gr table in the figure) stores information on a plurality of groups established according to the level of certainty. The case resolution rate table 211 stores information on the case resolution rate for each judgment group. The judgment group table 210 will be described in detail with reference to FIG. 6, and the case resolution rate table 211 will be described in detail with reference to FIG. 7, which will be described later.

[0023] The input information of the learning data 11 includes natural language written in free format (hereinafter referred to as "free text"). The learning model 12 has a function of analyzing the text included in the input information Ii and breaking it down into words.

[0024] Furthermore, the input / output processing unit 21 performs processing to transmit the replacement part candidates, the recommendation information, and the recommendation validity information to the terminal device 30 as output information Ir.

[0025] The terminal device 30 is a terminal operated by a user U, and is composed of, for example, a tablet terminal, a laptop PC (Personal Computer), a desktop PC, etc. The user U is an operation person, a worker, a maintenance person who maintains the on-site equipment at the site where the equipment including the device is located, etc. The terminal device 30 and the repair recommendation server 20 are connected to each other so as to be able to communicate with each other via a network N composed of an Internet line, a dedicated line, a mobile communication network, etc.

[0026] When equipment failure or the like occurs at the site, the user U inputs failure information on the failed equipment to the terminal device 30. The failure information is assumed to include information on the failed equipment and an explanation of the failure situation. Then, the terminal device 30 transmits this information to the repair recommendation server 20 as input information Ii. Based on the input information Ii from the terminal device 30, the recommendation information output as output information Ir from the repair recommendation server 20, or the recommendation information and recommendation validity information, are displayed on the screen of the terminal device 30. The user U refers to the recommendation information displayed on the screen of the terminal device 30, or the recommendation information and recommendation validity information, to determine the countermeasure (repair) to be implemented. Hereinafter, providing the recommendation information to the user U is also referred to as "recommending".

[0027] The number of replacement part candidates recommended by the repair recommendation server 20 is determined in advance by the top N of all replacement part candidates output from the learning model 12 and by a contract with the user U. The "N" in the top N corresponds to the "N" in the TopN correct answer rate. The value of N is determined, for example, by the high (target) recommendation accuracy that the user U expects from the repair recommendation system 1.

[0028] The target recommendation accuracy is determined between the user U and the company providing the repair recommendation service by an index such as "the probability that the repair will be completed is X% (e.g., 80%) by using the top N (e.g., 3) replacement part candidates output as recommended information." Here, "repair is completed" means that the repair is properly performed using the correct replacement part. The values ​​of N and X used in the index are set as target values, and the company providing the repair recommendation service using the repair recommendation system 1 constructs a learning model 12 based on the target value.

[0029] <Example of computer hardware configuration> Next, the hardware configuration of each device for realizing the functions of the control system of the repair recommendation system 1 shown in FIG. 1 will be described with reference to FIG.

[0030] Fig. 2 is a block diagram showing an example of the hardware configuration of the repair recommendation server 20 and the terminal device 30 that constitute the repair recommendation system 1. The calculator 100 shown in Fig. 2 is hardware used as a so-called computer.

[0031] The computer 100 includes a control unit 110, a non-volatile storage 120, a display unit 130, an operation input unit 140, and a communication I / F (Interface) 150, which are all connected to a bus B.

[0032] The control unit 110 includes a central processing unit (CPU) 111, a read only memory (ROM) 112, and a random access memory (RAM) 113. The CPU 111 reads out program code of software (applications) for implementing each function according to this embodiment from the ROM 112, expands it in the RAM 113, and executes it. The computer 100 may include a processing device such as an MPU (Micro-Processing Unit) instead of the CPU 111. Variables, parameters, etc. generated during the arithmetic processing are temporarily written to the RAM 113.

[0033] The non-volatile storage 120 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a non-volatile memory card, etc. In addition to an operating system (OS) and various parameters, programs for operating the computer 100, etc. are recorded in the non-volatile storage 120. The programs may be stored in the ROM 112.

[0034] The program is stored in the form of a computer-readable program code, and the CPU 111 sequentially executes operations according to the program code. In other words, the ROM 112 or the non-volatile storage 120 is used as an example of a computer-readable non-transitory recording medium that stores a program executed by a computer.

[0035] The functions of the repair recommendation engine unit 22 and the input / output processing unit 21 of the repair recommendation server 20 are realized by the CPU 111 reading out the corresponding program code from the ROM 112 or the non-volatile storage 120, expanding it in the RAM 113, and executing it.

[0036] Moreover, the storage areas for the judgment group table 210 and the case resolution rate table 211 in the input / output processing unit 21 of the repair recommendation server 20 are configured by the non-volatile storage 120 .

[0037] The display unit 130 is, for example, a monitor configured with an LCD (Liquid Crystal Display) or the like, and displays the results of the processing performed by the computer 100, recommended information, and the like. The operation input unit 140 is configured with, for example, a keyboard, a mouse, a touch sensor, etc., and generates an operation signal according to an operation by a user and supplies the operation signal to the CPU 111.

[0038] The display unit 130 and the operation input unit 140 may be integrated into a single touch panel. The repair recommendation server 20 may be configured without the display unit 130 and the operation input unit 140.

[0039] The communication I / F 150 may be, for example, a network interface card (NIC), and may transmit and receive various types of data to and from other devices connected via the network N.

[0040] <Replacement part candidates and confidence level included in recommendation information> Next, a description will be given of information on replacement part candidates and certainty factors included in recommendation information output from the repair recommendation server 20 to the terminal device 30. Fig. 3 is a diagram showing an example of the configuration of replacement part candidates and certainty factors.

[0041] Fig. 3 shows in the form of a table how certainty information is associated with each replacement part candidate output from the repair recommendation engine unit 22. In the table shown in Fig. 3, pairs of part names and certainty of replacement part candidates are displayed from top to bottom in order of the associated replacement part candidate with the highest certainty. In the recommendation information, a recommendation rank (1st, 2nd, ...) is assigned to replacement part candidates in order of the highest certainty.

[0042] For example, the top record in the table shown in Figure 3 displays information about a replacement part candidate with the part name "Part A" which has the highest certainty value of "40", and the second record displays information about a replacement part candidate with the part name "Part B" which has the next highest certainty value of "30".

[0043] As described above, the confidence level is a value that indicates the Top N accuracy rate. The confidence level is a value that is assigned to all replacement part candidates output as recommendation information, and takes values ​​from "0" to "100." Therefore, the sum of all confidence levels associated with each replacement part candidate is "100."

[0044] The recommendation information is information resulting from inference made by the repair recommendation engine unit 22 using the failure information input by the user as input. Therefore, it is considered that the higher the quality of the failure information input by the user, that is, the more the failure information appropriately contains information useful for selecting a replacement part, the higher the certainty associated with the part. Therefore, it can be said that the certainty is also an index showing the high quality of the failure information input by the user.

[0045] <Calculation method for case resolution rate> Next, a calculation method of the case resolution rate stored in the case resolution rate table 211 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an outline of a calculation method of the TopN case resolution rate. Fig. 4 shows, in table format, the result of case resolution (repair completion) determination for each test case input as test data to the repair recommendation engine unit 22.

[0046] Each test case is composed of examples of failures that have occurred in the past, and the test data for a test case is a set of free-word failure information actually entered into the repair recommendation system and information on the replacement parts used for the repair.

[0047] The table shown in Fig. 4 shows the top 10 replacement part candidates output as recommended information for each test case, in association with information on whether each replacement part candidate was a correct part that was actually used in the repair. The information on whether a part was a correct part is indicated by "o" if it was a correct part, and by "x" if it was not a correct part. The table shown in Fig. 4 also shows the number of replacement parts required for repair ("number of required parts"), the number of correct parts, and the result of the repair completion judgment ("repair completion judgment") for each test case.

[0048] The number of correct answers indicates the number of correct replacement parts (correct parts) that were actually used for repairs and were included in the top N replacement part candidates output as recommended information. Figure 4 shows an example in which "N" in the top N is set to "6," and in Figure 4, the range up to the top 6 is surrounded by a dashed frame.

[0049] When judging whether a repair is complete, if all of the correct parts with the M points set as the "number of parts required" are included among the top N replacement part candidates in the recommended information ("6" in the example shown in Figure 4), the repair is judged to be complete ("OK"), and if all are not included, the repair is judged to be incomplete ("NG"). In cases where all of the correct parts with the points set as the "number of parts required" are included among the top N replacement part candidates in the recommended information, the correct parts were presented as replacement part candidates, so the repair was completed without any problems. Therefore, the repair completion judgment is judged to be complete ("OK").

[0050] On the other hand, in cases where the top N replacement part candidates in the recommended information do not include all of the correct parts with the score set as the "number of parts required," the repair based on the recommended information is not completed. Therefore, the repair completion judgment is determined to be incomplete ("NG"). The information shown as a table in Figure 4 is calculated for each type of equipment.

[0051] "Case 1-a" in Figure 4 shows an example where the number of parts (M) required for repair is one. In case 1-a, it is shown that the replacement part candidate included in the first recommended information out of the top six replacement part candidates presented as recommended information was the correct part. Therefore, the repair completion judgment in case 1-a is "OK".

[0052] "Case 1-b" shows an example where the number of parts (M pieces) required for repair is three. Even in case 1-b where the number of parts required is "3", it is shown that the replacement part candidates included in the first, third and fourth recommended information out of the top six replacement part candidates presented as recommended information, i.e., three replacement part candidates, were correct parts. Therefore, the repair completion judgment in case 1-b is also "OK".

[0053] On the other hand, in case 2, where the number of required parts is also "3," it is shown that the replacement part candidates included in the second and fourth recommended information out of the top six replacement part candidates presented as recommended information were correct parts. In other words, since the number of correct parts was "2," which did not reach the number of required parts of "3," the repair completion judgment in case 2 is "NG."

[0054] In case 3, since the number of required parts is "7", it is impossible for all of the correct parts corresponding to the number of required parts to be included in the top 6 recommended information. Therefore, such a case is excluded from the target of repair completion judgment.

[0055] The TopN case resolution rate is shown by the ratio of the number of repair completion judgments to the number of all test cases that were subject to repair completion judgments. In other words, in the example shown in Figure 4, the number of cases that were subject to repair completion judgments was 3, and the number of repair completion judgments was 2, so the TopN (=6) case resolution rate for all test cases shown in Figure 4 is 66.66...%.

[0056] The information shown in Fig. 4 is calculated for each device for which repairs have been performed. Therefore, the TopN case resolution rate is also calculated for each device. Also, while Fig. 4 shows an example in which the TopN (=6) case resolution rate is calculated, the input / output processing unit 21 calculates the case resolution rates in the Top1st to Top10th places as the TopN case resolution rates.

[0057] <Classification of judgment groups> Next, an example of classification of judgment groups established according to the level of certainty will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing an overview of a calculation method of judgment coefficients that are classification criteria for judgment groups, and a classification example into judgment groups in list form, and Fig. 6 is a diagram showing a configuration example of a judgment group table.

[0058] In Fig. 5, parts output from the repair recommendation engine unit 22 as replacement part candidates required for repairing a certain broken device are arranged in descending order of confidence (highest recommendation rank). Each rank (recommendation rank) shown in Fig. 5 is associated with information on "part name", "confidence", "determination coefficient", and "determination group".

[0059] The "determination coefficient" is expressed by the sum of the confidence levels for the recommended ranks from 1 to i. For example, the confidence level of "Part A" with the number 1 recommended rank is "40", and when the recommended rank is 1, the rank "i" = 1, so the judgment coefficient for the part with the number 1 recommended rank (replacement part candidate) is entered as is as the confidence level value of "40".

[0060] Since the confidence level of "Part B" ranked second in the recommendation order is "30", the judgment coefficient associated with Part B is set to "70", which is the sum of the confidence level of the part ranked first in the recommendation order (Part A) of "40" and Part B's own confidence level of "30". Similar processing is performed for all parts output from the repair recommendation engine unit 22 as replacement part candidates for the corresponding device. Since the confidence level ranges from "1" to "100", the judgment coefficient value of the part ranked lowest in the recommendation order (with the smallest confidence level) is "100".

[0061] The confidence level increases as the number of times that the replacement part candidate output as the recommended information has actually been used as a repair part increases. Therefore, it can be said that the greater the value of the judgment coefficient in the top N recommended positions, the higher the accuracy (validity) of the recommendations that propose the replacement part candidates up to the Nth position. Therefore, in this embodiment, the judgment coefficient is used as an index showing the validity of the recommended information.

[0062] The "judgment group" is a group corresponding to each section obtained by dividing the judgment coefficient calculated in this way into an arbitrary range. Information on the judgment groups is stored in the input / output processing unit 21 as a judgment group table 210.

[0063] Fig. 6 is a diagram showing an example of the configuration of the judgment group table 210. In the judgment group table 210 shown in Fig. 6, judgment coefficients "0" to "49" are classified into judgment group G1, "50" to "79" are classified into judgment group G2, "80" to "89" are classified into judgment group G3, and "90" to "100" are classified into judgment group G4. In other words, the judgment group G1, which is the group with the smallest value of the classified judgment coefficient, is the group with the smallest cumulative value of the confidence level and the lowest validity of the recommendation. On the other hand, the judgment group G1, which is the group with the largest value of the classified judgment coefficient, is the group with the largest cumulative value of the confidence level and the highest validity of the recommendation.

[0064] The number of judgment groups to be divided, the value of the judgment coefficient as the criterion for dividing the judgment groups, etc. may be changed flexibly according to the situation. For example, if a target value is set in advance for the TopN case resolution rate and it is determined that the target value has not been achieved in the operation stage, the input / output processing unit 21 may change the number of judgment groups to be divided, the division of the judgment coefficient value, etc.

[0065] For example, suppose that the target value of the TopN case resolution rate is set to "80%" while the actual TopN case resolution rate is "75%". In this case, the input / output processing unit 21 takes measures such as changing the value of the lower limit judgment coefficient of the judgment group G3 to a value equal to or less than 75, increasing the number of judgment groups, and changing the correspondence between the judgment groups and the case resolution rate. This allows the TopN case resolution rate to reach the target value. By performing such processing, when there is a large difference between the target set by the user U and the actual situation, i.e., the quality of the input information, the repair recommendation server 30 becomes able to output recommendation information with an accuracy according to the actual situation of the user U.

[0066] <Configuration of case resolution rate table> Next, the configuration of the case resolution rate table 211 will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the configuration of the case resolution rate table 211. The case resolution rate table 211 is provided for each device. The device is a device that includes a fault candidate part included in the recommendation information output from the repair recommendation system 1.

[0067] 7 shows the case resolution rate of each replacement part candidate included in the 1st to 10th place recommendation ranking in each judgment group. The case resolution rate stored in the case resolution rate table 211 is the TopN case resolution rate. In other words, "1" to "10" in the "1st place" to "10th place" shown in the row direction of the case resolution rate table 211 correspond to "N" in the TopN case resolution rate.

[0068] For example, the case resolution rate of the top one in the judgment group G4 with the highest confidence level is "95%", whereas the case resolution rate of the top one in the judgment group G1 with the lowest confidence level is "30%". In other words, the case resolution rate table 211 shows the TopN case resolution rates according to the confidence level given to the replacement part candidates, that is, the quality of the input information (fault information) that has a high correlation with the confidence level.

[0069] <Outline of extraction process of case resolution rate based on recommendation information> Next, referring to Fig. 8, an outline of the process in which the input / output processing unit 21 extracts the case resolution rate for the corresponding case based on the recommendation information output from the repair recommendation engine unit 22 will be described. Fig. 8 is a diagram showing an outline of the extraction process of the case resolution rate based on the recommendation information.

[0070] The upper part of Fig. 8 shows the confidence level, judgment coefficient, and judgment group classification results for each of the replacement part candidates ranked 1st to 10th in the recommendation ranking included in the recommendation information. The recommendation information shown in the upper part of Fig. 8 is recommended information output from the repair recommendation engine unit 22 when failure information is input from a user who uses the repair recommendation system 1. For ease of explanation, the part name, confidence level, and judgment coefficient of each replacement part candidate shown in the upper part of Fig. 8 are set to the same values ​​as those in the output when test data is input as shown in Fig. 4.

[0071] When the repair recommendation engine unit 22 outputs recommendation information corresponding to the failure information input by the user, the input / output processing unit 21 extracts the top N case resolution rates predetermined between the user and the repair recommendation engine unit 22 by referring to a case resolution rate table 211 shown in the lower part of Fig. 8. For example, when it is decided that the top six faulty part candidates are to be presented as recommendation information, the input / output processing unit 21 extracts information on the judgment group associated with the sixth replacement part candidate. In the example shown in Fig. 8, the judgment group associated with the sixth replacement part candidate is group G4.

[0072] Next, the input / output processing unit 21 extracts the case resolution rate for which the judgment group is group G4 and the recommendation ranking is 6th, from the case resolution rate table 211 shown in the lower part of Fig. 8. In the example shown in the lower part of Fig. 8, the input / output processing unit 21 can extract the case resolution rate of "99%".

[0073] The input / output processing unit 21 sets a threshold value for the TopN case resolution rate in advance, and when the extracted TopN case resolution rate is equal to or lower than the threshold value, sends a message or the like indicating that the validity of the recommendation is low, included in the recommendation information, to the terminal device 30. The input / output processing unit 21 can set criteria in advance, such as a TopN case resolution rate of 80 or more as a "pass", 60 to 79 as a "conditional pass", and 59 or less as a "fail".

[0074] In addition, the input / output processing unit 21 associates a message such as "The validity of the recommended information is 60 to 80%" with a "conditional pass" and a message such as "No valid recommended information" and "Please enter all the information necessary for diagnosing the fault" with a "fail" result.

[0075] For example, if the case resolution rate corresponding to the top three recommended items is a value of 80 or higher, the input / output processing unit 21 transmits the top three replacement part candidates and information on the confidence level of each replacement part candidate as recommendation information to the terminal device 30.

[0076] In addition, if the case resolution rate corresponding to the top three recommended rankings is a value within the range of 60 to 79, the input / output processing unit 21 transmits the top three replacement part candidates, information on the confidence level of each replacement part candidate, and a message stating that "the validity of the recommended information is 60% to 80%" as recommendation information to the terminal device 30.

[0077] Furthermore, if the case resolution rate corresponding to the top three recommendation rankings is 59 or less, the input / output processing unit 21 sends a message such as "There is no highly valid recommended information" and "Please enter all the information necessary to diagnose the fault" as recommendation information to the terminal device 30.

[0078] FIG. 9 is a diagram showing an example of recommended information displayed on the screen of the terminal device 30 when the TopN case resolution rate is low. A fault information input area Ar1 is provided above the screen Sc shown in Fig. 9, and fault information such as "XX fault. Please deal with it as soon as possible" is input in the fault information input area Ar1. A recommended information display area Ar2 is provided below the fault information input area Ar1. In the example shown in Fig. 9, a message "There is no highly appropriate recommended information" and a message "※Please enter all information necessary for diagnosing the fault" are displayed in the recommended information display area Ar2.

[0079] By displaying such a message, the user can understand that the quality of the failure information he or she inputted may be low from the viewpoint of the learning model 12 in the repair recommendation system 1 inferring replacement part candidates. According to the present embodiment, it is possible to expect an improvement in the quality of failure information inputted by a user who has seen such a message. Furthermore, by continuously accumulating high-quality input information in the repair recommendation system 1, it is possible to expect an improvement in the accuracy of inference by the learning model 12 in the repair recommendation system 1, and an improvement in the case resolution rate based on recommendation information.

[0080] Note that the message prompting the input of accurate fault information is not limited to the example shown in Fig. 9. For example, the message prompting the input of accurate fault information may be configured as a message notifying that the input fault information contains little information related to the fault symptoms, or a message notifying that the input fault information contains a lot of unnecessary information not required for fault diagnosis.

[0081] <Repair recommendation processing by repair recommendation system> Next, a repair recommendation process by the repair recommendation system 1 will be described with reference to Fig. 10. Fig. 10 is a sequence chart showing an example of the procedure of the repair recommendation process. First, when a target device breaks down, a user U inputs information (failure information) explaining the failure status of the target device to a terminal device 30 (see FIG. 1). The terminal device 30 transmits the input failure information to the repair recommendation server 20 via the network N (step S1). Next, the input / output processing unit 21 of the repair recommendation server 20 accepts input of the failure information from the terminal device 30 (step S2). Then, the input / output processing unit 21 outputs the input information to the repair recommendation engine unit 22 (step S3).

[0082] Next, the repair recommendation engine unit 22 inputs the failure information input as input information from the input / output processing unit 21 to the learning model 12, and infers recommendation information including a plurality of replacement part candidates recommended for the failure information and the certainty factor of each replacement part candidate (step S4). Next, the repair recommendation engine unit 22 outputs the inferred recommendation information to the input / output processing unit 21 (step S5).

[0083] Next, the input / output processing unit 21 calculates a judgment coefficient for the top N replacement part candidates based on the confidence factor included in the recommendation information (step S6). Next, the input / output processing unit 21 identifies a judgment group based on the judgment coefficient calculated in step S6 (step S7). Next, the input / output processing unit 21 acquires a TopN case resolution rate for the judgment group identified in step S7 (step S8).

[0084] Next, the input / output processing unit 21 judges whether the TopN case resolution rate acquired in step S8 is equal to or greater than a predetermined threshold (step S9). The threshold may be, for example, "80" set as the above-mentioned standard for "pass". If it is judged in step S9 that the case resolution rate is equal to or greater than the predetermined threshold (if the judgment in step S9 is YES), the input / output processing unit 21 transmits the recommendation information output from the repair recommendation engine unit 22 to the terminal device 30 (step S10).

[0085] On the other hand, if it is determined in step S9 that the case resolution rate is less than the predetermined threshold, the input / output processing unit 21 outputs recommendation validity information (step S11) together with the recommendation information output from the repair recommendation engine unit 22. More specifically, the input / output processing unit 21 outputs a message according to the level of the case resolution rate as the recommendation validity information, as described with reference to FIG.

[0086] In the present embodiment, an example has been given in which the input / output processing unit 21 does not output recommendation validity information to the terminal device 30 when the case resolution rate is equal to or higher than a predetermined threshold, but the present invention is not limited to this. Even when the case resolution rate is equal to or higher than a predetermined threshold, the input / output processing unit 21 may transmit a message indicating that the validity of the recommendation information is high to the terminal device 30 as recommendation validity information.

[0087] The terminal device 30 displays the recommendation information or the recommendation information and the recommendation validity information transmitted from the repair recommendation server 20 on a display device. Then, the user U refers to the recommendation validity information and orders the replacement part candidate proposed as the recommendation information. Then, a maintenance worker or the like repairs the failure using the ordered replacement part.

[0088] In the above-described embodiment, the input / output processing unit 21 of the repair recommendation server 20 classifies replacement part candidates included in the recommendation information output from the repair recommendation engine unit 22 into one of a plurality of judgment groups provided according to the degree of certainty. Then, the input / output processing unit 21 identifies a TopN case resolution rate that is previously associated with the classified judgment group, and outputs information regarding the magnitude of the identified TopN case resolution rate as validity information of the recommendation information.

[0089] For example, if the TopN case resolution rate acquired by the input / output processing unit 21 is low based on the recommendation information output as a result of inference by the repair recommendation engine unit 22, it is assumed that the quality of the failure information as input information was low. In this embodiment, in such a case, a message (recommendation validity information) indicating that the validity of the recommendation information is low is transmitted to the terminal device 30. Therefore, according to this embodiment, even if the confidence level associated with the replacement part candidate is high, the user U can understand that the validity of the recommendation information is low based on the message.

[0090] This also improves the user U's trust in the repair recommendation system 1, and therefore it is possible to expect the user U to continue using the repair recommendation system 1. Furthermore, as the user U continues to use the repair recommendation system 1, the number of test cases also increases, and the accuracy of inference by the repair recommendation engine unit 22 can also be improved.

[0091] Furthermore, according to the above-described embodiment, if the TopN case resolution rate extracted based on the recommendation information output from the repair recommendation engine unit 22 is low, a message such as "Please enter all the information necessary for fault diagnosis" is displayed on the screen of the terminal device 30. This allows the user U to understand that the fault information he or she input does not contain sufficient information necessary for fault diagnosis (inference). By notifying the user U of such a message, it is possible to improve the quality of the fault information input by the user U, and as a result, the accuracy of inference by the repair recommendation engine unit 22 can also be improved.

[0092] <Modification> In the above embodiment, the fault handling content included in the recommendation information is an example of a replacement part candidate for a part that constitutes a faulty device, but the present invention is not limited to this. The fault handling content may be a method of handling the fault or information on the part where the fault occurs. Examples of methods of handling the fault include adjustment, cleaning, removal of foreign matter, waiting and seeing, and part replacement, and examples of information on the part include a bill insertion part, a bankbook insertion part, a display, and the like.

[0093] In the above embodiment, the repair recommendation server 20 includes the input / output processing unit 21 and the repair recommendation engine unit 22, but the present invention is not limited to this. For example, the input / output processing unit 21 and the repair recommendation engine unit 22 may be included in separate servers. Alternatively, some or all of the functions of the repair recommendation server 20 may be implemented in the terminal device 30.

[0094] Furthermore, the above-described embodiments provide detailed and specific descriptions of the configurations of the devices and systems in order to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described.

[0095] In addition, the control lines and information lines indicated by solid lines and arrows in Figures 1 and 2 are those considered necessary for explanation, and do not necessarily show all control lines and information lines in the product. In reality, it can be considered that almost all components are connected to each other.

[0096] In addition, in this specification, the processing steps describing chronological processing include not only processing that is performed chronologically in the order described, but also processing that is not necessarily performed chronologically but is performed in parallel or individually (for example, parallel processing or processing by objects). [Explanation of symbols]

[0097] 1... repair recommendation system, 12... learning model, 20... repair recommendation server, 21... input / output processing unit, 22... repair recommendation engine unit, 30... terminal device, 210... judgment group table, 211... case resolution rate table

Claims

1. a recommendation calculation unit that uses a learning model in which fault information on a broken device is input and countermeasures recommended for dealing with the fault are output to infer recommendation information including a plurality of countermeasures for the input fault information and a confidence level indicating the accuracy of each countermeasure; and an input / output processing unit that classifies the handling content into one of a plurality of judgment groups that are set according to the level of the certainty based on the certainty level associated with the handling information, identifies a case resolution rate that is previously associated with the classified judgment group, and outputs information regarding the magnitude of the identified case resolution rate as validity information of the recommendation information. Repair recommendation system.

2. The measures included in the recommendation information are ranked in order from highest to lowest associated certainty; The case resolution rate is represented by the probability that M countermeasure contents that have actually been implemented are included in the top N countermeasure contents. The repair recommendation system of claim 1 .

3. The judgment groups are groups obtained by dividing a judgment coefficient obtained by accumulating the certainty factors associated with the countermeasure contents in order from the highest rank into a predetermined number within a predetermined range. The repair recommendation system of claim 2 .

4. When the case resolution rate determined based on the degree of certainty corresponding to the countermeasure content is equal to or lower than a predetermined threshold, the input / output processing unit outputs, together with the recommendation information, a message prompting the input of information necessary and sufficient for inferring the countermeasure content. The repair recommendation system of claim 3 .

5. When the case resolution rate associated with the judgment group does not reach a predetermined target value, the input / output processing unit performs an adjustment so that the case resolution rate coincides with the target value by changing the range of the judgment coefficient that defines each judgment group and / or the number of divisions of the judgment group. The repair recommendation system of claim 4.

6. The fault information includes a natural language description in a free format; The countermeasures included in the recommendation information are replacement part candidates to be used for repairing the broken device. The repair recommendation system according to any one of claims 1 to 5.

7. A step of inferring recommendation information including a plurality of countermeasures for the inputted failure information and a degree of certainty indicating the accuracy of each countermeasure, using a learning model in which failure information on a broken device is input and countermeasures recommended for dealing with the failure are output; based on the degree of certainty associated with the handling information, classifying the handling content into one of a plurality of judgment groups provided according to the degree of certainty, specifying a case resolution rate previously associated with the classified judgment group, and outputting information relating to the magnitude of the specified case resolution rate as validity information of the recommendation information. Repair recommendation method.

Images