Electronic device evaluation system and electronic device evaluation program

Abstract

Information will be provided regarding the status of electronic devices being tested, and measures will be taken to bring them back to an appropriate state. [Solution] This embodiment includes a model access means that accesses a status information derivation model using data measured from the electronic device 200 to obtain status information of the electronic device 200 and the measurement system, and accesses a countermeasure information derivation model using data measured from the electronic device 200 to obtain countermeasure information to be applied to the electronic device 200 and the measurement system; a display means for displaying the information; and a display control means that controls the display of the status information of the electronic device 200 and the measurement system and the countermeasure information to be applied to the electronic device and the measurement system obtained by the model access means on the display means.

Description

Technical Field

[0001] This invention relates to an evaluation system for an electronic device including a semiconductor device (hereinafter, the concept including a semiconductor device is abbreviated) and a program for evaluating an electronic device.

Background Art

[0002] Conventionally, the evaluation of an electronic device has been performed by obtaining numerical data or waveform data such as an oscilloscope from the electronic device and visually observing and considering it by a person.

[0003] Patent Document 1 discloses an observation condition determination support device capable of improving the classification accuracy of defects. This observation condition determination support device has means for obtaining a plurality of defect images obtained by photographing the same defect under a plurality of observation conditions preset in an observation device based on inspection data related to defects of a semiconductor device detected by an inspection device. Further, based on each defect image, classification of a plurality of the same defects is performed, and means for determining a first category to which the same defects belong for each observation condition as a result of the classification is provided. Furthermore, based on the ratio of the first category matching the second category determined by the user of the observation device by classifying the same defects, it has means for determining an observation condition to be used during the manufacture of the semiconductor device from among a plurality of observation conditions.

[0004] Also, Patent Document 2 discloses an IC tester debug support method for quickly identifying the location of a problem. This method discloses the debugging of an IC tester that tests a device under test using setting parameters set by a PC using a test program. During this debugging process, a single button press triggers a comparison of the test specifications and set parameters that the test program should adhere to, and displays them on the display device. Similarly, a single press triggers a change in the set parameters, and the PASS / FAIL status of the changed parameters is displayed on the display device via Schmoo. Furthermore, a single press triggers a test of the device under test using an IC tester, and the output of the device under test, representing the test result, is displayed on the display device. These processes help determine whether the problem lies in the test program, test specifications, or the device under test.

[0005] Patent Document 3 discloses a method and system for correcting defects in semiconductor manufacturing systems. The method and system for correcting defects in this semiconductor manufacturing system is a method for correcting defects in process tools for semiconductor manufacturing, which involves collecting old service activity data for old defects in the process tools and receiving new service activity data for new defects in the process tools. Furthermore, the new service activity data is compared with the old service activity data, and the changes that need to be adapted to the service activity data are identified from the comparison. Then, corrective actions are taken based on the adapted service activity data. Furthermore, one or more tests are performed using the above-mentioned process tool to generate new adapted service activity data using the adapted service activity data. Additionally, the new adapted service activity data is used to narrow down the adapted service activity data. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] International Publication No. 2010 / 061771 [Patent Document 2] Japanese Patent Publication No. 2008-70294 [Patent Document 3] Special Publication No. 2007-529898 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, none of the conventional evaluation methods have provided information on the status of the electronic devices by analyzing the obtained data, nor have they provided information on countermeasures to bring them into an appropriate state.

[0008] This invention has been made in view of the current situation, and its purpose is to provide an evaluation system for electronic devices and an evaluation program for electronic devices that informs the user of the status of the electronic device being tested, etc., and provides countermeasures for bringing it to an appropriate state. [Means for solving the problem]

[0009] The evaluation system for the electronic device of this embodiment includes: a status information derivation model generated by machine learning using training data in which data measured from the electronic device is used as explanatory variables and the status of the electronic device and the measurement system when the data of the explanatory variables is measured is used as the objective variable; a countermeasure information derivation model generated by machine learning using training data in which data measured from the electronic device is used as explanatory variables and the countermeasure to be taken on the electronic device and the measurement system when the data of the explanatory variables is measured is used as the objective variable; a model access means that accesses the status information derivation model using data measured from the electronic device to obtain status information of the electronic device and the measurement system, and accesses the countermeasure information derivation model using data measured from the electronic device to obtain countermeasure information to be taken on the electronic device and the measurement system; a display means for displaying the information; and a display control means that controls the display of the status information of the electronic device and the measurement system and the countermeasure information to be taken on the electronic device and the measurement system obtained by the model access means on the display means. It is equipped with, The aforementioned situational information derivation model is generated by machine learning using the same data measured from the electronic device as explanatory variables, and multiple possible situations of the electronic device and the measurement system at the time the data for these explanatory variables was measured as the dependent variable. By accessing the situational information derivation model using one set of data measured from the electronic device, it is possible to obtain multiple situational information for the electronic device and the measurement system, along with their probabilities. The display control means displays multiple status information of the electronic device and the measurement system along with probabilities on the display means. It is characterized by the following.

[0010] The evaluation program for the electronic device of this embodiment includes: a computer, a situation information derivation model generated by machine learning using training data in which the data measured from the electronic device is used as explanatory variables and the status of the electronic device and the measurement system when the data of the explanatory variables is measured is used as the objective variable; a countermeasure information derivation model generated by machine learning using training data in which the data measured from the electronic device is used as explanatory variables and the countermeasures to be taken on the electronic device and the measurement system when the data of the explanatory variables is measured is used as the objective variable; a model access means that accesses the situation information derivation model using the data measured from the electronic device to obtain situation information of the electronic device and the measurement system, and accesses the countermeasure information derivation model using the data measured from the electronic device to obtain countermeasure information to be taken on the electronic device and the measurement system; and a display control means that controls the display of the situation information of the electronic device and the measurement system and the countermeasure information to be taken on the electronic device and the measurement system obtained by the model access means on a display means table for displaying information. Make it work, The aforementioned situational information derivation model is generated by machine learning using the same data measured from the electronic device as explanatory variables, and the multiple possible situations of the electronic device and the measurement system at the time the data for these explanatory variables was measured as the dependent variable. It functions so that when the situational information derivation model is accessed using one set of data measured from the electronic device, multiple situational information of the electronic device and the measurement system, along with their probabilities, can be obtained. The computer functions as a model access means to obtain probabilities along with multiple status information of the measurement system, and the computer functions as an electronic device and a display control means to display probabilities along with multiple status information of the electronic device and the measurement system on the display means. It is characterized by the following: [Brief explanation of the drawing]

[0011] [Figure 1] A block diagram of an evaluation system for an electronic device according to the first embodiment of the present invention. [Figure 2] A block diagram showing a computer-based evaluation system for an electronic device according to an embodiment of the present invention. [Figure 3] A functional block diagram of an evaluation system for an electronic device according to an embodiment of the present invention. [Figure 4] This figure shows an example of situation information and action content information obtained by a situation information derivation model and an action content information derivation model for an evaluation system of an electronic device according to an embodiment of the present invention. [Figure 5] In the evaluation system of an electronic device according to an embodiment of the present invention, a diagram showing evaluation (waveform) data obtained from the electronic device. [Figure 6] A flowchart showing the operation of the evaluation system of an electronic device according to the first embodiment of the present invention. [Figure 7] A flowchart showing the operation of the evaluation system of an electronic device according to the second embodiment of the present invention. [Figure 8] A diagram showing an example of situation information and countermeasure content information obtained by the situation information derivation model and the countermeasure content information derivation model of the evaluation system of an electronic device according to the third embodiment of the present invention. [Figure 9] A flowchart showing the operation of the evaluation system of an electronic device according to the third embodiment of the present invention. [Figure 10] A diagram showing an example of display of situation information and countermeasure content information by the evaluation system of an electronic device according to the third embodiment of the present invention. [Figure 11] A flowchart showing the operation of the evaluation system of an electronic device according to the fourth embodiment of the present invention. [Figure 12] A flowchart showing the operation of the evaluation system of an electronic device according to the fifth embodiment of the present invention. [Figure 13] A diagram showing an example of display of situation information and countermeasure content information by the evaluation system of an electronic device according to the fifth embodiment of the present invention. [Figure 14] A flowchart showing the operation of the evaluation system of an electronic device according to the sixth embodiment of the present invention. [Figure 15] A diagram showing an example of situation information and countermeasure content information obtained by the situation information derivation model and the countermeasure content information derivation model of the evaluation system of an electronic device according to the seventh embodiment of the present invention. [Figure 16] A flowchart showing the operation of the evaluation system of an electronic device according to the seventh embodiment of the present invention. [Figure 17] A flowchart showing the operation of the evaluation system of an electronic device according to the eighth embodiment of the present invention. [Figure 18] A flowchart showing the operation of the evaluation system of an electronic device according to the eighth embodiment of the present invention. [Modes for carrying out the invention]

[0012] The evaluation system for an electronic device and the evaluation program for an electronic device according to the present invention will be described below with reference to the attached drawings. In each figure, the same components are denoted by the same reference numerals, and redundant explanations are omitted. The evaluation system 100 for an electronic device according to the first embodiment of the present invention is connected to a measuring device 300 that extracts signals from various parts of the electronic device 200 to determine whether the performance design and specification design are appropriate, as shown in Figure 1. Since the measuring device 300 extracts various signals to determine whether various performance designs and specification designs are appropriate, it is not a single device but can be a collection of multiple devices, and therefore the measuring device 300 is referred to as a measurement system. There can be multiple signal lines between the electronic device 200 and the measuring device 300, and there can be multiple circuits between the measuring device 300 and the evaluation system 100.

[0013] The evaluation system 100 can be configured using a computer, such as a server. As shown in Figure 2, the computer uses a CPU 10 to configure the evaluation system 100 using programs and data in the main memory 11. A storage interface 13, an input interface 14, a display interface 15, and a data input / output interface 16 are connected to the CPU 10 via a bus 12.

[0014] Storage interface 13 is connected to storage 23. Storage 23 includes external storage devices such as HDDs, auxiliary storage devices, and cloud storage. Storage 23 stores programs and data necessary for the operation of this evaluation system 100, which the CPU 10 can read and use from main memory 11 as needed. Therefore, storage 23 stores the programs that realize the evaluation system 100. Input interface 14 is connected to input devices 24 such as keyboards and touch panels and pointing devices 22 such as mice. Display interface 15 is connected to a display device 25 having a screen such as an LCD, and the display device 25 implements the display means. Data input / output interface 16 is connected to lines 26-1 to 26-m, which include lines for connecting measuring devices 300, and can be wired or wireless. The number of lines 26-1 to 26-m is arbitrary. The signals obtained via lines 26-1 to 26-m are input to the necessary components such as the CPU 10 and the display interface 15 via the data input / output interface 16.

[0015] As shown in the functional block diagram in Figure 3, the evaluation system 100 includes a status information derivation model 101, a response content information derivation model 102, a model access means 103, and a display control means 104. These means (including the models) can be implemented by the CPU 10 of the evaluation system 100 reading programs corresponding to these means stored in the storage 23 into the main memory 11.

[0016] The situation information derivation model 101 was generated by machine learning using training data in which the data measured from the electronic device 200 was used as explanatory variables and the status of the electronic device 200 and the measurement system when the explanatory variable data was measured was used as the objective variable. The countermeasure information derivation model 102 was generated by machine learning using training data in which the data measured from the electronic device 200 was used as explanatory variables and the countermeasures to be taken on the electronic device 200 and the measurement system when the explanatory variable data was measured was used as the objective variable.

[0017] The model access means 103 accesses the status information derivation model 101 using data measured from the electronic device 200 to obtain status information of the electronic device 200 and the measurement system, and also accesses the countermeasurement information derivation model 102 using data measured from the electronic device 200 to obtain countermeasurement information to be applied to the electronic device 200 and the measurement system.

[0018] The display control means 104 controls the display device 25, which is the display means, to display status information of the electronic device 200 and the measurement system obtained by the model access means 103, as well as information on the actions to be taken with the electronic device 200 and the measurement system.

[0019] The evaluation system 100 with the above configuration can obtain situation information and action content information as shown in Figure 4 using the situation information derivation model 101 and the action content information derivation model 102. By accessing the situation information derivation model 101 and the action content information derivation model 102 using the evaluation (waveform) data A to Q listed in the target data column of Figure 4, the situation of the electronic device 200 and the measurement system stored in the situation information column is derived as situation information, and the action content information to be applied to the electronic device 200 and the measurement system stored in the action content information column is derived. The evaluation (waveform) data A to Q can be waveforms or numerical values ​​as shown in Figures 5(A) to 5(Q). When several measurement items are instructed and measured using the electronic device 200 and the measurement device 300, some of the evaluation (waveform) data A to Q corresponding to the specified measurement items are sent from the measurement device 300 to the evaluation system 100, and the received evaluation (waveform) data are displayed on the display means of the evaluation system 100 along with their respective identification information. For example, by specifying evaluation (waveform) data G by, for example, identification information, accessing the status information derivation model 101 will derive status information for the electronic device 200, such as "Power supply noise is occurring," and by accessing the action content information derivation model 102 using evaluation (waveform) data G, action content information for the electronic device 200, such as "Please check the power supply settings and power supply environment," will be derived. Also, for example, by specifying evaluation (waveform) data K, accessing the status information derivation model 101 will derive status information for the measurement system, such as "The probe settings are incorrect," and by accessing the action content information derivation model 102 using evaluation (waveform) data K, action content information for the measurement system, such as "Please check the probe settings," will be derived.

[0020] Figure 6 shows a flowchart illustrating the operation of the evaluation system 100. The operation will be explained using this flowchart. The CPU 10 of the evaluation system 100 receives a measurement instruction from a measuring device that has specified (or input) the measurement items, and sends the measurement instruction to the measuring device (S11). Next, it receives evaluation (waveform) data corresponding to the measurement instruction and displays it on the display means (S12). Next, it waits for an instruction to access the status information derivation model 101 and the action content information derivation model 102 until it becomes YES (S13). If it becomes YES, it accesses the status information derivation model 101 and the action content information derivation model 102 according to the instruction and acquires the status information and action content information (S14). Furthermore, it displays the acquired status information and action content information on the display means (S15).

[0021] The second embodiment allows the user to choose between accessing both the situation information derivation model 101 and the action content information derivation model 102, accessing only the situation information derivation model 101, or accessing only the action content information derivation model 102, compared to the first embodiment. Figure 7 shows a flowchart illustrating the operation of this embodiment. Steps S12 onwards are the same as in Figure 6. In the next step S13A following step S12, it is detected whether there is an access instruction for the status information derivation model and / or the action content information derivation model.

[0022] In step S13A, if the answer is YES, in step S16, the system detects which model the instruction is for accessing. In step S16, if it is detected that the instruction is for accessing both the status information derivation model and the action content information derivation model, the system accesses the status information derivation model 101 and the action content information derivation model 102 according to the instruction and obtains the status information and action content information (S14). Furthermore, the obtained status information and action content information are displayed on the display means (S15).

[0023] In step S16, if it is detected that the instruction is to access only the status information derivation model 101, the system accesses only the status information derivation model 101 according to the instruction and obtains the status information and the action content information (S17). Furthermore, only the obtained status information is displayed on the display means (S18). In step S16, if it is detected that the instruction is to access only the action content information derivation model 102, the system accesses only the action content information derivation model 102 according to the instruction and obtains only the action content information (S19). Furthermore, only the obtained action content information is displayed on the display means (S20).

[0024] In the third embodiment, the situation information derivation model 101 is generated by machine learning using the same data measured from the electronic device 200 as explanatory variables and multiple possible situations of the electronic device 200 and the measurement system at the time the data of the explanatory variables was measured as the objective variables. The model is configured so that when the situation information derivation model is accessed using one set of data measured from the electronic device 200, multiple situation information of the electronic device 200 and the measurement system can be obtained along with their probabilities. The situation information derivation model 101 is accessed using one set of data measured from the electronic device 200, multiple situation information of the electronic device 200 and the measurement system can be obtained along with their probabilities, and the display control means 104 displays the multiple situation information of the electronic device 200 and the measurement system along with their probabilities on the display means.

[0025] Furthermore, the above-mentioned countermeasure information derivation model 102 is generated by machine learning using the same data measured from the electronic device 200 as the explanatory variable, and multiple possible countermeasure information that may be applied to the electronic device 200 and the measurement system when the data of this explanatory variable is measured as the target variable. The model is configured so that when the countermeasure information derivation model 102 is accessed using one set of data measured from the electronic device 200, multiple countermeasure information for the electronic device 200 and the measurement system can be obtained along with their probabilities. The countermeasure information derivation model 102 is accessed using one set of data measured from the electronic device 200, multiple countermeasure information for the electronic device 200 and the measurement system can be obtained along with their probabilities, and the display control means 104 displays the multiple countermeasure information for the electronic device 200 and the measurement system along with their probabilities on the display means.

[0026] The situation information derivation model 101 and the action content information derivation model 102 of this third embodiment allow us to obtain situation information and action content information as shown in Figure 8. By accessing the situation information derivation model 101 and the action content information derivation model 102 using the evaluation (waveform) data A to Q listed in the target data column of Figure 8, we can obtain the situation information and probabilities at the locations marked with a circle for situations 1 to 4 in the situation column, and the action content information and probabilities at the locations marked with a circle for action content 1 to action content 4. The probabilities obtained with the situation information and the probabilities obtained with the action content information are the same. Note that the evaluation (waveform) data A to Q in Figure 8 are not necessarily the same as the evaluation (waveform) data A to Q in Figure 4. Therefore, the situation information and action content information derived from the evaluation (waveform) data A to Q in Figure 8 are not identical to those in Figure 4. In this embodiment, for example, by accessing the situation information derivation model 101 using evaluation (waveform) data E, situation information for situation 1, situation 2, and situation 3 and their probabilities can be obtained. From situation 1, the situation information "short circuit with GND" and its probability can be obtained; from situation 2, the situation information "ground is floating" and its probability can be obtained; and from situation 3, the situation information "cable wiring error" and its probability can be obtained.

[0027] Furthermore, for example, by accessing the action content information derivation model 102 using evaluation (waveform) data E, it is possible to obtain action content information and probabilities for action content 1, action content 2, and action content 3. From action content 1, the action content information "Please check the connection point of GND" and its probability can be obtained; from action content 2, the action content information "Please check if GND is connected" and its probability can be obtained; and from action content 3, the action content information "Please review the cable wiring" and its probability can be obtained.

[0028] Figure 9 shows a flowchart illustrating the operation of the evaluation system 100 according to the third embodiment described above. The operation will be explained using this flowchart. Upon receiving a measurement instruction, the CPU 10 of the evaluation system 100 sends a measurement instruction to the measuring device that has specified (or input) the measurement item (S21). Next, it receives evaluation (waveform) data corresponding to the measurement instruction and displays it on the display means (S22). Next, it waits for a response to whether there is an instruction to access the situation information derivation model 101 and the action content information derivation model 102, for example, by identification information, until the response is YES (S23). If the response is YES, it accesses the situation information derivation model 101 and the action content information derivation model 102 according to the instruction and acquires multiple pieces of information, including the situation information and its probability, and the action content information and its probability, depending on the case (S24). Furthermore, it displays the acquired situation information and its probability in order of probability on the display means, and then displays the acquired action content information and its probability in order of probability on the display means (S25).

[0029] For example, if the situation information for situations 1, 2, and 3 and their probabilities are obtained by accessing the situation information derivation model 101 using evaluation (waveform) data E, and the action content information for action content 1, action content 2, and action content 3 and their probabilities are obtained by accessing the action content information derivation model 102 using evaluation (waveform) data E, the display will be as shown in Figure 10. The first line will display the situation information and action content information, along with a 50% probability, which will read "Situation 1: Shorted to GND. Action 1: Check the connection point of GND." The second line will display the situation information and action content information, along with a 30% probability, which will read "Situation 2: GND is floating. Action 2: Check if GND is connected." The third line will display the situation information and action content information, along with a 20% probability, which will read "Situation 3: Incorrect cable connection. Action 3: Review the cable connections."

[0030] The fourth embodiment, compared to the third embodiment, allows the user to select from the following options: an instruction to access both the situation information derivation model 101 and the action content information derivation model 102; an instruction to access only the situation information derivation model 101; or an instruction to access only the action content information derivation model 102. Figure 11 shows a flowchart illustrating the operation of this embodiment. Steps up to S22 are the same as in Figure 9. In the next step S23A after step S22, it is detected whether there is an access instruction for the status information derivation model and / or the action content information derivation model.

[0031] In step S23A, if the answer is YES, in step S26, the system detects which model the instruction is for accessing. In step S26, if it is detected that the instruction is for accessing both the situation information derivation model and the action content information derivation model, the system accesses the situation information derivation model 101 and the action content information derivation model 102 according to the instruction, and obtains multiple pieces of situation information and action content information and their probabilities, if possible (S24). Furthermore, the obtained situation information and its probabilities are displayed on the display means in order of probability, and the obtained action content information and its probabilities are also displayed on the display means in order of probability (S25).

[0032] In step S26, if it is detected that the instruction is to access only the situation information derivation model 101, the system accesses only the situation information derivation model according to the instruction and obtains the situation information and its probability (S27). Furthermore, the obtained situation information and its probability are displayed on the display means in order of probability (S28). In step S26, if it is detected that the instruction is to access only the action content information derivation model 102, the system accesses only the action content information derivation model 102 according to the instruction and obtains the action content information and its probability (S29). Furthermore, the obtained action content information and its probability are displayed on the display means in order of probability (S30).

[0033] In the fifth embodiment, the situation information derivation model 101 is the same as that of the third embodiment, and is generated by machine learning using the same data measured from the electronic device 200 as the explanatory variable and the multiple possible situations of the electronic device 200 and the measurement system at the time the data of this explanatory variable was measured as the objective variable. The system is configured so that when the situation information derivation model is accessed using one set of data measured from the electronic device 200, multiple situation information for the electronic device 200 and the measurement system can be obtained along with their probabilities. When the situation information derivation model 101 is accessed using one set of data measured from the electronic device 200, multiple situation information for the electronic device 200 and the measurement system can be obtained along with their probabilities, and the display control means 104 displays the top two probabilities for the multiple situation information for the electronic device 200 and the measurement system along with their probabilities on the display means. If only one situation information is obtained, one will be displayed. In this embodiment, the top two are displayed, but it is naturally possible to display the top n (where n is any positive integer) probabilities.

[0034] Furthermore, the above-mentioned countermeasure information derivation model 102 is the same as that of the third embodiment. It is generated by machine learning using the same data measured from the electronic device 200 as the explanatory variable, and multiple possible countermeasure information that may be applied to the electronic device 200 and the measurement system when the data of this explanatory variable is measured as the target variable. The model is configured so that when the countermeasure information derivation model 102 is accessed using one set of data measured from the electronic device 200, multiple countermeasure information for the electronic device 200 and the measurement system can be obtained along with their probabilities. When the countermeasure information derivation model 102 is accessed using one set of data measured from the electronic device 200, multiple countermeasure information for the electronic device 200 and the measurement system can be obtained along with their probabilities, and the display control means 104 displays the top two probabilities for the multiple countermeasure information for the electronic device 200 and the measurement system along with their probabilities on the display means. If only one countermeasure information is obtained, that one will be displayed. In this embodiment, we use the top two elements, but it is naturally possible to use the top n elements (where n is any positive integer). The operation flowchart of this embodiment is shown in Figure 12. Compared to the operation flowchart of the third embodiment shown in Figure 9, it differs from the third embodiment in that, in step S25A following step S24, the acquired situation information and its probability are displayed on the display means up to the second highest probability, and further, the acquired countermeasure information and its probability are also displayed on the display means up to the second highest probability. An example of the display is shown in Figure 13. It can be seen that, compared to the display in the third embodiment, the display means only displays up to the second highest probability.

[0035] In addition to the fifth embodiment described above, a sixth embodiment can be configured in which one of the following can be selected: an instruction to access both the situation information derivation model 101 and the action content information derivation model 102, an instruction to access only the situation information derivation model 101, or an instruction to access only the action content information derivation model 102. The operation flowchart of this sixth embodiment is shown in Figure 14, and differs from the operation flowchart of the fourth embodiment shown in Figure 11 in that the display in steps S25A, S28A, and S30A is limited to the second most probable result.

[0036] The situation information derivation model 101 of the seventh embodiment includes category information as an objective variable, which indicates the category of situation information to be derived. It specifies whether the category of situation information should be situation information relating to the electronic device 200 and the measurement system, situation information relating to the electronic device 200, situation information relating to the measurement system, or situation information relating to the surrounding environment. For example, the situation information "trigger setting error" falls under the category of situation information relating to the measurement system; the situation information "abnormal output voltage relative to input voltage" falls under the category of situation information relating to the electronic device 200; the situation information "control signal setting error" falls under the category of situation information relating to the electronic device 200 and the measurement system; and the situation information "the temperature (or humidity) of the measurement room is abnormally high (or low)" falls under the category of situation information relating to the surrounding environment.

[0037] Furthermore, the 7th embodiment's action content information derivation model 102 includes category information as a target variable, indicating the category of action content information to be derived. It specifies whether the category of action content information should be action content information relating to the electronic device 200 and the measurement system, action content information relating to the electronic device 200, action content information relating to the measurement system, or action content information relating to the surrounding environment. For example, action content information such as "Please set the trigger to ●●" falls under the category of action content information relating to the measurement system; action content information such as "Please check the device design" falls under the category of action content information relating to the electronic device 200; action content information such as "Please check the register settings" falls under the category of action content information relating to the electronic device 200 and the measurement system; and action content information such as "Please set the temperature (or humidity) of the measurement room to ●●" falls under the category of situational information relating to the surrounding environment.

[0038] The situation information derivation model 101 and the action content information derivation model 102 of this seventh embodiment allow us to obtain situation information and action content information as shown in Figure 15. By accessing the situation information derivation model 101 and the action content information derivation model 102 using the evaluation (waveform) data A to Q listed in the target data column of Figure 15, we can obtain the situation information and probabilities at the locations marked with a circle for situations 1 to 4 in the situation column, and the action content information and probabilities at the locations marked with a circle for action content 1 to action content 4. The probabilities obtained with the situation information and the probabilities obtained with the action content information are the same. Note that the evaluation (waveform) data A to Q in Figure 15 are not necessarily the same as the evaluation (waveform) data A to Q in Figure 8. Therefore, the situation information and action content information derived from the evaluation (waveform) data A to Q in Figure 15 are not identical to those in Figure 8.

[0039] In the Category X column of Figure 15, the category of situational information is described by one of a to d. By accessing the situational information derivation model 101 using evaluation (waveform) data A to evaluation (waveform) data Q, one of category information a to d can be obtained in addition to situational information and probability. There are four categories here, but it is not limited to four, and the content of the category information is not limited to the above. Category information a indicates that the category should be limited to situational information related to the measurement system, category information b indicates that the category should be limited to situational information related to the electronic device 200, category information c indicates that the category should be limited to situational information related to the electronic device 200 and the measurement system, and category information d indicates that the category should be limited to situational information related to ambient environment information.

[0040] In the Category Y column of Figure 15, the category of the countermeasure information is described by one of e to h. By accessing the countermeasure information derivation model 102 using evaluation (waveform) data A to evaluation (waveform) data Q, one of category information e to h can be obtained in addition to situation information and probability. There are four categories here, but it is not limited to four, and the content of the category information is not limited to the above. Category information e indicates that the category should be limited to countermeasure information related to the measurement system, category information f indicates that the category should be limited to countermeasure information related to the electronic device 200, category information g indicates that the category should be limited to countermeasure information related to the electronic device 200 and the measurement system, and category information h indicates that the category should be limited to countermeasure information related to surrounding environment information.

[0041] In the example in Figure 15, accessing the situation information derivation model 101 using evaluation (waveform) data B yields category information d and situation information such as situation 1, situation 2, situation 3, situation 4, and probabilities. Category information d indicates that the category should be limited to situation information related to ambient environment information. Suppose situation 1 is "abnormal output voltage relative to input voltage" with a probability of 30%, situation 2 is "data not acquired" with a probability of 30%, situation 3 is "the temperature (or humidity) of the measurement room is abnormally high (or low)" with a probability of 30%, and situation 4 is "no abnormality" with a probability of 10%. Of situations 1 to 4, only situation 3 falls under the category of ambient environment information, so only the information "Probability 30%: The temperature (or humidity) of the measurement room is abnormally high (or low)" is displayed on the display device.

[0042] In the example in Figure 15, accessing the action content information derivation model 102 using evaluation (waveform) data B yields category information h and action content information such as action content 1, action content 2, action content 3, action content 4, and probabilities. Category information h indicates that the category should be limited to action content information related to ambient environment information. Suppose action content 1 is "Check the device design" with a probability of 30%, action content 2 is "Check if it is being captured" with a probability of 30%, action content 3 is "Set the temperature (or humidity) of the measurement room to ●●" with a probability of 30%, and action content 4 is "This is normal data" with a probability of 10%. Of action content 1 to action content 4, only action content 3 falls under the category of ambient environment information, so only the information "Probability 30%: Set the temperature (or humidity) of the measurement room to ●●" is displayed on the display.

[0043] The operation flowchart of this embodiment is shown in Figure 16, and differs from the operation flowchart of the third embodiment shown in Figure 9 in that, in step S24B following step S23, the situation information derivation model 101 and the action content information derivation model 102 are accessed according to the instructions, and category information, situation information and its probability, and action content information and its probability are acquired, in some cases, multiple times. Furthermore, in step S25B following step S24B, the acquired situation information and its probability are limited by category information and displayed on the display means, and further, the acquired action content information and its probability are limited by category information and displayed on the display means.

[0044] Furthermore, the category information indicating the range of situational information to be derived may include category information that specifies the selection of situational information indicating a high risk of damage to electronic devices or measuring devices. For example, if the category information is kk, the situational information "There is a high risk of damage to electronic devices or measuring devices" will be set to one of Situation 1, Situation 2, Situation 3, or Situation 4. Although different from the illustration, by accessing the situational information derivation model 101 using evaluation (waveform) data KK, it is possible to obtain category information kk and situational information such as Situation 1, Situation 2, Situation 3, Situation 4, and probabilities. In this case, the situational information "There is a high risk of damage to electronic devices or measuring devices" is selected from one of Situation 1, Situation 2, Situation 3, or Situation 4 and displayed on the display means along with the probability.

[0045] Furthermore, the category information indicating the range of action information to be derived can include category information that specifies the selection of action information that instructs action to be taken in response to a high risk of damage to electronic devices or measuring devices. For example, if the category information is kk, the action information "There is a high risk of damage to the electronic device or measuring device, so please switch to a backup electronic device or backup measuring device." will be set to one of action 1, action 2, action 3, or action 4. Although it differs from the figure, when accessing the action information derivation model 102 using evaluation (waveform) data KK, it is possible to obtain the category information kk and the action information as action 1, action 2, action 3, action 4, and probabilities. In this case, the action information "There is a high risk of damage to the electronic device or measuring device, so please switch to a backup electronic device or backup measuring device." will be selected from one of action 1, action 2, action 3, or action 4 and displayed on the display means along with the probability.

[0046] By configuring the system as described above, it is possible to display information about situations requiring immediate attention and information about necessary actions. When displaying information about situations requiring immediate attention, it is possible to use special display methods, such as using sound or light to alert users, or using different colors or exaggerated fonts on the display screen than usual, to encourage the use of emergency measures.

[0047] In addition to the seventh embodiment described above, an eighth embodiment can be configured in which one of the following can be selected: an instruction to access both the situation information derivation model 101 and the action content information derivation model 102, an instruction to access only the situation information derivation model 101, or an instruction to access only the action content information derivation model 102.

[0048] The operation flowchart of this eighth embodiment is shown in Figure 17, and differs from the operation flowchart of the fourth embodiment shown in Figure 11 in that category information is obtained in steps S24B, S27B, and S29B, and further differs in that the display is limited by category information in steps S25B, S28B, and S30B. In the case of processing according to the flowchart in Figure 17, the category information indicating the category of situation information to be derived may include category information that specifies the selection of situation information indicating a high risk of damage to electronic devices or measuring devices, and the category information indicating the category of action content information to be derived may include category information that specifies the selection of action content information that instructs action corresponding to a high risk of damage to electronic devices or measuring devices.

[0049] In the ninth embodiment, the data measured from the electronic device 200 used by the model access means 103 and the status information of the electronic device 200 and the measurement system obtained by the model access means 103 are fed back to perform machine learning on the status information derivation model 101. In the ninth embodiment, the data measured from the electronic device 200 used by the model access means 103 and the information on the countermeasures to be applied to the electronic device 200 and the measurement system obtained by the model access means 103 are fed back to perform machine learning on the countermeasures information derivation model 102. If category information is obtained by the model access means 103, the category information is also fed back to perform machine learning on the status information derivation model 101 and the countermeasures information derivation model 102.

[0050] In the ninth embodiment, the operation is carried out according to the flowchart shown in Figure 18. That is, once the display of situation information, action content information, and probability is complete, the evaluation (waveform) data used (data measured from the electronic device 200) is used as the explanatory variable, and learning data with situation information, action content information, probability, and category information as the objective variables is stored in a learning data storage unit (not shown) (S31). Next, it is detected whether a predetermined amount of learning data has been stored or whether the storage for a predetermined period has ended (S32).

[0051] If the answer in step S32 is YES, machine learning is performed on the situation information derivation model 101 and the countermeasure information derivation model 102 using feedback from the training data stored in the training data storage unit (S33). In this way, the accuracy of the situation information derivation model 101 and the countermeasure information derivation model 102 can be improved.

[0052] Each embodiment described above is one of the display devices 25, although not explicitly shown as described in Figure 2. In each embodiment, two display means are used to display status information of the electronic device 200 and the measurement system on one display means, and information on the actions to be taken on the electronic device and the measurement system obtained by the model access means is displayed on the other display means. This allows the status information indicated by the data measured from the electronic device 200 (evaluation (waveform) data) and the actions to be taken on the electronic device 200 and the measurement system to be visually inspected on independent display means, enabling clear data evaluation. [Explanation of symbols]

[0053] 10 CPU 11 Main Memory 12 Bus 13 Storage Interface 14 Input Interface 15 Display Interface 16 Data input / output interface 22 Pointing device 23 Storage 24 Input device 25 Display device 26-1~26-m line 100 Evaluation System 101 Situation Information Derivation Model 102 Model for deriving information on countermeasures 103 Model access means 104 Display control means 200 Electronic device 300 measuring devices

Claims

1. A situation information derivation model is generated by machine learning using training data in which data measured from an electronic device is used as explanatory variables, and the state of the electronic device and measurement system when the data of the explanatory variables was measured is used as the objective variable. A model for deriving information on countermeasures is generated by machine learning using training data in which the data measured from the electronic device is used as explanatory variables and the countermeasures to be taken on the electronic device and the measurement system when the data of the explanatory variables is measured is used as the objective variable. A model access means that uses data measured from the electronic device to access the status information derivation model to obtain status information of the electronic device and the measurement system, and uses data measured from the electronic device to access the action content information derivation model to obtain action content information to be applied to the electronic device and the measurement system, A means of displaying information, A display control means that controls the display of status information of the electronic device and the measurement system obtained by the model access means and information on the actions to be taken on the electronic device and the measurement system on the display means. It is equipped with, The aforementioned situation information derivation model is generated by machine learning using the same data measured from the electronic device as explanatory variables, and multiple possible situations of the electronic device and the measurement system at the time the data for these explanatory variables was measured as the objective variable. By accessing the situation information derivation model using one set of data measured from the electronic device, it is possible to obtain multiple situational information of the electronic device and the measurement system along with their probabilities. An evaluation system for an electronic device, characterized in that multiple status information of the electronic device and the measurement system is displayed on the display means along with probabilities by the display control means.

2. In the aforementioned response content information derivation model, the same data measured from the electronic device is used as the explanatory variable, and multiple possible response content information that may be applied to the electronic device and the measurement system when the data for the explanatory variable is measured is generated by machine learning. When the response content information derivation model is accessed using one set of data measured from the electronic device, multiple possible response content information for the electronic device and the measurement system, along with their probabilities, can be obtained. The electronic device evaluation system according to claim 1, characterized in that the display control means displays information on multiple possible handling methods for the electronic device and the measurement system together with their probabilities on the display means.

3. In the aforementioned situational information derivation model, the same data measured from the electronic device is used as the explanatory variable, and multiple possible situations of the electronic device and the measurement system at the time the data for the explanatory variable was measured are used as the objective variable in machine learning to generate the model. By accessing the situational information derivation model using one set of data measured from the electronic device, it is possible to obtain multiple situational information of the electronic device and the measurement system along with their probabilities. The electronic device evaluation system according to claim 1, characterized in that when the display control means displays multiple status information of the electronic device and the measurement system together with the probabilities on the display means, the top n (where n is an arbitrary positive integer) probabilities are displayed.

4. In the aforementioned response content information derivation model, the same data measured from the electronic device is used as the explanatory variable, and multiple possible response content information that may be applied to the electronic device and the measurement system when the data for the explanatory variable is measured is generated by machine learning. When the response content information derivation model is accessed using one set of data measured from the electronic device, multiple possible response content information for the electronic device and the measurement system, along with their probabilities, can be obtained. The electronic device evaluation system according to claim 3, characterized in that when the display control means displays information on multiple possible solutions for the electronic device and the measurement system along with their probabilities on the display means, the top n (where n is an arbitrary positive integer) probabilities are displayed.

5. The electronic device evaluation system according to claim 1, characterized in that the situation information derivation model includes category information indicating the category of situation information to be derived as a target variable, and it is specified whether the category of situation information should be situation information relating to the electronic device and the measurement system, situation information relating to the electronic device, situation information relating to the measurement system, or situation information relating to the surrounding environment.

6. The electronic device evaluation system according to claim 1, characterized in that the aforementioned action content information derivation model includes category information indicating the category of action content information to be derived as an objective variable, and it is specified whether the category of action content information should be action content information relating to the electronic device and the measurement system, action content information relating to the electronic device, action content information relating to the measurement system, or action content information relating to the surrounding environment information.

7. The electronic device evaluation system according to claim 1, characterized in that the data measured from the electronic device used by the model access means and the status information of the electronic device and the measurement system obtained by the model access means are fed back to perform machine learning on the status information derivation model.

8. The electronic device evaluation system according to claim 1, characterized in that the model access means feeds back data measured from the electronic device and information on countermeasures to be applied to the electronic device and the measurement system obtained by the model access means to perform machine learning on the countermeasures information derivation model.

9. An electronic device evaluation system according to claim 1, characterized in that two of the display means are used to display status information of the electronic device and the measurement system on one of the display means, and information on the actions to be taken on the electronic device and the measurement system obtained by the model access means is displayed on the other display means.

10. Computers, A situation information derivation model generated by machine learning using training data in which data measured from an electronic device is used as explanatory variables, and the status of the electronic device and measurement system when the data for the explanatory variables was measured is used as the dependent variable. A model for deriving information on countermeasures is generated by machine learning using training data, where the data measured from the aforementioned electronic device is the explanatory variable, and the countermeasures to be taken on the electronic device and the measurement system when the data of the explanatory variable is measured are the target variable. A model access means that uses data measured from the electronic device to access the status information derivation model to obtain status information of the electronic device and the measurement system, and uses data measured from the electronic device to access the action content information derivation model to obtain action content information to be applied to the electronic device and the measurement system. A display control means that controls the display of status information of the electronic device and the measurement system obtained by the model access means, and information on the actions to be taken on the electronic device and the measurement system, on a display means for displaying the information. To make it function as, The aforementioned situational information derivation model is generated by machine learning using the same data measured from the electronic device as explanatory variables, and the multiple possible situations of the electronic device and the measurement system at the time the data for these explanatory variables was measured as the dependent variable. It functions so that when the situational information derivation model is accessed using one set of data measured from the electronic device, multiple situational information of the electronic device and the measurement system, along with their probabilities, can be obtained. An evaluation program for an electronic device, characterized in that the computer functions as a model access means to obtain probabilities along with multiple status information of the measurement system, and the computer functions as an electronic device and a display control means to display the probabilities along with multiple status information of the electronic device and the measurement system on the display means.

11. The aforementioned action content information derivation model is generated by machine learning using the same data measured from the electronic device as the explanatory variable, and multiple possible action content information that may be applied to the electronic device and the measurement system when the data of the explanatory variable is measured, as the target variable. It functions so that when the action content information derivation model is accessed using one set of data measured from the electronic device, multiple possible action content information for the electronic device and the measurement system, along with their probabilities, can be obtained. The evaluation program for an electronic device according to claim 10, characterized in that the computer functions as a model access means to obtain probability along with information on multiple possible handling scenarios for the electronic device and the measurement system, and the computer functions as a display control means to display on the display means.

12. The aforementioned situational information derivation model is generated by machine learning using the same data measured from the electronic device as explanatory variables, and the multiple possible situations of the electronic device and the measurement system at the time the data for these explanatory variables was measured as the dependent variable. It functions so that when the situational information derivation model is accessed using one set of data measured from the electronic device, multiple situational information for the electronic device and the measurement system, along with their probabilities, can be obtained. The electronic device evaluation program according to claim 10, characterized in that the computer functions as a model access means to obtain probabilities along with multiple status information of the electronic device and the measurement system, and the computer functions as a display control means to display the top n (where n is an arbitrary positive integer) probabilities when displayed on the display means.

13. The aforementioned action content information derivation model is generated by machine learning using the same data measured from the electronic device as the explanatory variable, and multiple possible action content information that may be applied to the electronic device and the measurement system when the data of the explanatory variable is measured, as the target variable. It functions to allow access to the action content information derivation model using one set of data measured from the electronic device, thereby obtaining the probability along with multiple possible action content information for the electronic device and the measurement system. The evaluation program for the electronic device according to claim 12, characterized in that the computer functions as a model access means to obtain probabilities along with information on multiple possible handling methods for the electronic device and the measurement system, and the computer functions as a display control means to display the top n (where n is any positive integer) probabilities when displayed on the display means.

14. The evaluation program for an electronic device according to claim 10, characterized in that the situation information derivation model includes category information indicating the category of situation information to be derived as a target variable, and it is specified whether the category of situation information should be situation information relating to the electronic device and the measurement system, situation information relating to the electronic device, situation information relating to the measurement system, or situation information relating to the surrounding environment.

15. The electronic device evaluation program according to claim 10, characterized in that the aforementioned action content information derivation model includes category information indicating the category of action content information to be derived as a target variable, and it is specified whether the category of action content information should be action content information relating to the electronic device and the measurement system, action content information relating to the electronic device, action content information relating to the measurement system, or action content information relating to the surrounding environment information.

16. The electronic device evaluation program according to claim 10, characterized in that the computer functions to perform machine learning of the status information derivation model by feeding back data measured from the electronic device used as the model access means and status information of the electronic device and the measurement system obtained by functioning as the model access means.

17. The electronic device evaluation program according to claim 10 is characterized in that the computer functions to perform machine learning of the countermeasure information derivation model by feeding back data measured from the electronic device used as the model access means and information on countermeasures to be applied to the electronic device and the measurement system obtained from the model access means.

18. The electronic device evaluation program according to claim 10, characterized in that the computer uses two display means to display status information of the electronic device and the measurement system on one display means, and to display information on the actions to be taken on the electronic device and the measurement system obtained by the model access means on the other display means.

Images