Analysis device, analysis method, and analysis program

Abstract

To provide an analysis device, a method, and a program, which improve efficiency of training a learning model and perform inference from image data obtained by capturing a graph pertaining to an analysis target.SOLUTION: In an analysis device comprising a processor for executing a program and a storage device for storing the program, the program causes the processor to execute: first generation processing for generating first image data of a graph pertaining to an analysis target from an arrangement region on a display screen on which the graph is arranged, as training data input to a learning model pertaining to the analysis target; acquisition processing for, when the first generation processing generates the first image data, acquiring target data pertaining to the analysis target corresponding to the graph as correct answer data used for training the learning model; and training processing for training the learning model on the basis of a training result output as a result of inputting the first image data to the learning model, and the target data acquired by the acquisition processing.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an analysis device, an analysis method, and an analysis program for generating a learning model and performing inference using the learning model. [Background technology]

[0002] The use of AI (Artificial Intelligence) for the purpose of improving productivity is beginning to increase in various fields such as manufacturing sites, farms, and software development factories. One example of the use of AI is to analyze time-series or non-time-series data to analyze productivity and quality. Introducing AI requires the extraction of the above data from databases of various systems, the generation of learning models using large amounts of diverse data, and analysis using the learning models, which is very costly.

[0003] The following Patent Document 1 describes a method, system, and device for reading equipment in a factory or plant environment, including capturing an image of an environment including a measuring device, detecting a target area included in the image, the target area including at least a portion of the measuring device, determining identification information associated with the measuring device based on detecting the target area, and extracting measurement information associated with the measuring device based on detecting the target area. In some embodiments, detecting the target area may include providing the image to a machine learning network, and receiving an output from the machine learning network in response to the machine learning network processing the image based on a detection model, the output including the target area.

[0004] The following Patent Document 2 includes identifying, for a current learning cycle, a target device for each of at least one learning task to be executed from among a plurality of candidate devices based on resource information of the plurality of candidate devices, transmitting a global model for each task to the target device for each task, so that the target device for each task trains the global model for each task, and in response to receiving the post-training model transmitted to all target devices for each task, updating the global model for each task based on the post-training model, thereby completing the current learning cycle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2023-29227 A [Patent Document 2] JP 2022-136234 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, data handled by production management systems and quality control systems is often composed of many types of information, not just one type. In Patent Document 1, only the range of the on-site meter that can be seen from the outside is photographed. Therefore, it cannot be used when it is necessary to scroll the display screen. For this reason, the data displayed on the dashboard by the production management system does not fit on one display screen, and in order to read all the data, it is necessary to scroll the display screen with a mouse. In other words, the above-mentioned Patent Document 1 does not take into consideration the acquisition of information on a dashboard where a large amount of information is displayed and the entire information cannot be displayed without scrolling.

[0007] Furthermore, since Patent Document 1 is based on one image and one piece of information, when multiple pieces of information are mixed on one screen, it does not take into consideration digitizing them and inputting them into the system of Patent Document 2. Furthermore, when digitizing multiple pieces of information, if the data formats are different, data shaping is required when inputting them into a learning model, but Patent Documents 1 and 2 do not take data shaping into consideration.

[0008] An object of the present invention is to improve the efficiency of learning. [Means for solving the problem]

[0009] An analysis device which is one aspect of the invention disclosed in the present application is an analysis device having a processor which executes a program and a storage device which stores the program, wherein the processor executes a first generation process which generates first image data of a graph relating to an analysis subject from a placement area on a display screen in which the graph is placed, as training data to be input into a learning model relating to the analysis subject; when the first image data is generated by the first generation process, an acquisition process which acquires target data relating to the analysis subject corresponding to the graph as correct answer data to be used for learning the learning model; and a learning process which learns the learning model based on a learning result output as a result of inputting the first image data into the learning model and the target data acquired by the acquisition process. Effect of the Invention

[0010] According to the representative embodiment of the present invention, it is possible to improve the efficiency of learning. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiment. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an analysis device. [Diagram 2] FIG. 2 is an explanatory diagram illustrating an example of a learning model DB. [Diagram 3] FIG. 3 is an explanatory diagram showing an example of acquiring a screenshot. [Figure 4] FIG. 4 is an explanatory diagram showing a learning example of the learning model M#. [Diagram 5] FIG. 5 is an explanatory diagram showing a first example of the dashboard. [Figure 6] FIG. 6 is an explanatory diagram showing a second example of the dashboard. [Figure 7] FIG. 7 is an explanatory diagram showing a third example of the dashboard. [Figure 8] FIG. 8 is an explanatory diagram showing a fourth example of the dashboard. [Figure 9] FIG. 9 is an explanatory diagram showing a fifth example of the dashboard. [Figure 10] FIG. 10 is a flowchart illustrating an example of an analysis process procedure performed by the analysis device. [Figure 11] FIG. 11 is a flowchart illustrating a detailed example of the setting process (step S1005) illustrated in FIG. [Figure 12] FIG. 12 is a flowchart illustrating a detailed example of the learning process (step S1006) illustrated in FIG. [Figure 13] FIG. 13 is a flowchart illustrating a detailed example of the processing procedure of the inference process (step S1007) illustrated in FIG. [Figure 14] FIG. 14 is an explanatory diagram showing an example of a system configuration of a federated learning system. [Figure 15] FIG. 15 is an explanatory diagram illustrating an example of the global learning DB. [Figure 16] FIG. 16 is an explanatory diagram illustrating an example of the local learning model DB. [Figure 17] FIG. 17 is a sequence diagram showing Example 1 of the setting process in the federated learning system. [Figure 18] FIG. 18 is a sequence diagram showing a second example of the setting process in the federated learning system. [Figure 19] FIG. 19 is a sequence diagram illustrating an example of a federated learning process performed by the federated learning system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0012] <Figure 1 Example of hardware configuration of analytical equipment> FIG. 1 is a block diagram showing an example of a hardware configuration of an analysis device. The analysis device 100 includes a processor 101, a storage device 102, an input device 103, an output device 104, and a communication interface (communication IF) 105. The processor 101, the storage device 102, the input device 103, the output device 104, and the communication IF 105 are connected by a bus 106. The processor 101 controls the analysis device 100. The storage device 102 serves as a working area for the processor 101. The storage device 102 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 102 include a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 103 inputs data. Examples of the input device 103 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 104 outputs data. The output device 104 may be, for example, a display, a printer, or a speaker. The communication IF 105 connects to a network and transmits and receives data.

[0013] <Figure 2 Learning model DB> 2 is an explanatory diagram showing an example of a learning model DB 200. The learning model DB 200 is stored in the storage device 102 of the analysis device 100 or another device capable of communicating with the analysis device 100.

[0014] The learning model DB 200 is a database that stores information about the learning model, and has the following fields: model ID 201, task type 202, objective 203, training data 204, correct answer data 205, and location information 206. The model ID 201 is identification information that uniquely identifies the learning model, and serves as a pointer to the learning model. Hereinafter, a learning model whose model ID 201 value is "M#" (# is a unique number) will be referred to as learning model M#.

[0015] Task type 202 indicates the type of task (e.g., discrimination, regression, etc.) executed by learning model M#. Objective 203 indicates the processing content (e.g., fault diagnosis, inventory forecast, etc.) for learning model M# to output a response variable, and indicates the attributes of the response variable that is the output result (e.g., fault diagnosis result, inventory forecast value, etc.).

[0016] The training data 204 is used for learning the learning model M# and is data paired with the correct answer data 205. The training data 204 is a screenshot to be input to the learning model M#.

[0017] A screenshot is image data capturing a graph arranged on a dashboard. A graph is a chart showing time-dependent changes or comparison results of various data (hereinafter referred to as analytical data), such as data input to an analysis target (for example, a production management system or quality management system that manages an assembly manufacturing line, a work management system that manages the behavior of heavy machinery in a heavy machinery work area, or a human resource management system that authenticates workers using a camera and manages the behavior of workers), data detected by the analysis target, data that detects the behavior of the analysis target, data generated by the internal processing of the analysis target, data output from the analysis target, or statistical values ​​such as the average value and standard deviation of these data. Examples of graphs include waveform diagrams, histograms, pie charts, scatter plots, radar charts, and box and whisker plots. Note that the analysis target is not limited to a production management system or a quality management system.

[0018] When there is no need to distinguish between the values ​​TR1, TR2, . . . of the training data 204, they are simply referred to as TR.

[0019] The supervised answer data 205 is used for learning the learning model M# and is objective data indicating the objective 203 for the analysis target paired with the training data 204. For example, if the task type 202 is identification and the objective 203 is fault diagnosis, the supervised answer data 205 (GT1) is a numerical value indicating a fault or normality of the analysis target at the time when the graph from which the paired training data 204 (TR1) is captured is displayed. Also, if the task type 202 is regression and the objective 203 is inventory forecasting, the supervised answer data 205 (GT2) is an actual measured value of the inventory of the analysis target at the time when the graph from which the paired training data 204 (TR2) is captured is displayed. The supervised answer data 205 is used when determining the value of the loss function for the learning model M#.

[0020] When there is no need to distinguish between the values ​​GT1, GT2, . . . of the correct answer data 205, they are simply written as GT.

[0021] The position information 206 is information indicating the position of the graph on the dashboard, which is the input source. The position information 206 may be the vertex coordinate values ​​of the screenshot or the center of gravity position. The position information 206 is specified by the user via the input device 103 and is stored in the learning model DB 200.

[0022] When the values ​​P1, 92, . . . of the position information 206 are not to be distinguished, they are simply referred to as coordinate value P.

[0023] <Figure 3 Example of screenshot> 3 is an explanatory diagram showing an example of a screenshot. Dashboard 300 is a layout area on a display screen displayed on a display, which is an example of output device 104 of analysis device 100. Dashboard 300 has at least display area 301 out of display area 301 and non-display area 302. Display area 301 is an area visible to a user of analysis device 100, and non-display area 302 is an area not visible to a user of analysis device 100. Display area 301 and non-display area 302 are changed by a user operating scroll bar 303.

[0024] 3, graphs G1 to G4 are arranged in display area 301, and graphs G5 and G6 are arranged in non-display area 302. When there is no need to distinguish between graphs G1 to G6, they will be referred to as graph G.

[0025] Analysis apparatus 100 captures a group of screenshots SC1s to SC4s of graphs G1 to G4 arranged in display area 301, and also captures a group of screenshots SC5s and SC6s of graphs G5 and G6 arranged in non-display area 302.

[0026] Screenshot group SC1s-SC6s is a time series collection of screenshots SC1-SC6 of graphs G1-G6. When there is no need to distinguish between screenshot groups SC1s-SC6s, they are referred to as screenshot group SCs. When there is no need to distinguish between screenshots SC1-SC6, they are referred to as screenshots SC.

[0027] <Figure 4 Learning example> FIG. 4 is an explanatory diagram showing a learning example of the learning model M#. In FIG. 4, learning models M1 to M4 are taken as examples for explanation. The learning model M1 is trained using a screenshot group SC1s of a graph G1 and its correct answer data GT1. The learning model M# is formed, for example, of a convolutional neural network.

[0028] During learning, when a group of screenshots SC1s, which is training data TR1, is input, the learning model M1 outputs a learning result RL1. The analysis device 100 inputs the learning result LR1 and the correct answer data GT1 into a loss function to calculate a loss value L1. The analysis device 100 updates the learning model M1 by adjusting learning parameters such as weights through error backpropagation so that the loss value L1 is reduced.

[0029] During inference, when a screenshot SC1 is input, the learning model M1 outputs an inference result IR1.

[0030] The learning model M2 is trained using a screenshot group SC1s of the graph G1 and a screenshot group SC2s of the graph G2, and their respective correct answer data GT1 and GT2.

[0031] During learning, when a group of screenshots SC1s and SC2s, which are training data TR2, are input, the learning model M2 outputs a learning result RL2. The analysis device 100 inputs the learning result LR2 and the correct answer data GT2 into a loss function to calculate a loss value L2. The analysis device 100 updates the learning model M2 by adjusting learning parameters such as weights through error backpropagation so that the loss value L2 becomes smaller.

[0032] During inference, when certain screenshots SC1 and SC2 are input, the learning model M2 outputs an inference result IR2.

[0033] The learning model M3 is trained using a screenshot group SC1s of the graph G1, a screenshot group SC2s of the graph G2, and a screenshot group SC3s of the graph G3, and their correct answer data GT1 to GT3.

[0034] During learning, when a group of screenshots SC1s to SC3s, which are training data TR3, is input, the learning model M3 outputs a learning result RL3. The analysis device 100 inputs the learning result LR3 and the correct answer data GT3 into a loss function to calculate a loss value L3. The analysis device 100 updates the learning model M3 by adjusting learning parameters such as weights through error backpropagation so that the loss value L3 becomes smaller.

[0035] During inference, when certain screenshots SC1 to SC3 are input, the learning model M3 outputs an inference result IR3.

[0036] The learning model M4 is trained using screenshot group SC1s of graph G1, screenshot group SC2s of graph G2, screenshot group SC3s of graph G3, and screenshot group SC4s of graph G4, and their correct answer data GT1 to GT4.

[0037] During learning, when a group of screenshots SC1s to SC4s, which are training data TR4, is input, the learning model M4 outputs a learning result RL4. The analysis device 100 inputs the learning result LR4 and the correct answer data GT4 into a loss function to calculate a loss value L4. The analysis device 100 updates the learning model M4 by adjusting learning parameters such as weights through error backpropagation so that the loss value L4 becomes smaller.

[0038] During inference, when certain screenshots SC1 to SC4 are input, the learning model M4 outputs an inference result IR4.

[0039] When the learning results LR1 to LR4 are not differentiated, they are simply referred to as the learning result LR. When the inference results IR1 to IR4 are not differentiated, they are simply referred to as the inference result IR. When the loss values ​​L1 to L4 are not differentiated, they are simply referred to as the loss value L.

[0040] <Figures 5 to 9 Dashboard 300> 5 is an explanatory diagram showing example 1 of the dashboard 300. The dashboard 300 has a first area 501, a second area 502, and a third area 503. The first area 501 is an area in which a graph G is arranged, and includes the display area 301 and the non-display area 302 described above.

[0041] The second area 502 is an area for displaying a group of buttons. The group of buttons includes a setting button 521, a learning button 522, and an inference button 523. The setting button 521 is a user interface for executing a setting process, which will be described later, when pressed. The learning button 522 is a user interface for executing a learning process, which will be described later, when pressed. The inference button 523 is a user interface for executing an inference process, which will be described later, when pressed.

[0042] The third area 503 is an area that displays details when any of the buttons is pressed. The third area 503 will be described with reference to FIG.

[0043] Fig. 6 is an explanatory diagram showing example 2 of dashboard 300. Fig. 6 is a display example when setting button 521 is pressed in the display state of Fig. 5. Pressing setting button 521 changes it to end setting button 621. End setting button 621 is a user interface for ending the setting process when pressed.

[0044] When the setting button 521 is pressed, a group of check boxes B1, B2, B3, ... (when there is no need to distinguish between them, they are referred to as a group of check boxes B) for selecting a graph G for each learning model M# are displayed as input destination learning model selection information in the third area 503. The graph G selected in the group of check boxes B becomes the training data TR for that learning model M#.

[0045] Fig. 7 is an explanatory diagram showing example 3 of the dashboard 300. Fig. 7 is a display example when the setting button 521 is pressed in the display state of Fig. 6. A state is shown in which graph G1 is selected in check box group B1, graphs G1 and G2 are selected in check box group B2, and graphs G1 to G3 are selected in check box group B3. Therefore, the screenshot group SC1s of graph G1 becomes the training data TR1 of the learning model M1, the screenshot groups SC1s and SC2s of graphs G1 and G2 become the training data TR2 of the learning model M2, and the screenshot group SC1s to SC3s of graphs G1 to G3 become the training data TR3 of the learning model M3.

[0046] Fig. 8 is an explanatory diagram showing Example 4 of the dashboard 300. Fig. 8 is a display example when the learning button 522 is pressed in the display state of Fig. 5. Pressing the learning button 522 changes it to an all learning button 822. The all learning button 822 is a user interface for executing learning processing for all learning models M# when pressed.

[0047] When the learning button 522 is pressed, individual learning buttons 831-1, 831-2, 831-3, ... (when there is no need to distinguish between these, they are referred to as individual learning buttons 831) are displayed for each learning model M# in the third area 503. The analysis device 100 learns the learning model M# for which the individual learning button 831 was pressed, using its training data TR and supervised data GT.

[0048] Fig. 9 is an explanatory diagram showing Example 5 of the dashboard 300. Fig. 9 is a display example when the inference button 523 is pressed in the display state of Fig. 5. Pressing the inference button 523 changes it to an all inference button 923. The all inference button 923 is a user interface for executing inference processing for all learning models M# when pressed.

[0049] When the inference button 523 is pressed, individual inference buttons 931-1, 931-2, 931-3, ... (when there is no need to distinguish between them, they are referred to as individual inference buttons 931) are displayed for each learning model M# in the third area 503. The analysis device 100 outputs an inference result IR for the learning model M# for which the individual inference button 931 was pressed, by capturing and inputting a screenshot SC of the graph G located in the position information 206 of the learning model M#.

[0050] <Figure 10 Analysis processing> 10 is a flowchart showing an example of an analysis process procedure by the analysis device 100. The analysis device 100 acquires analysis data and displays a graph G on the dashboard 300 (step S1001). The displayed graphs G include graphs G showing time-series changes in analysis data and graphs G comparing the same type of analysis data. The graphs G showing time-series changes in analysis data may show time-series changes within a certain period up to the present, or may show time-series changes within a period specified by the user. The graphs G comparing the same type of analysis data may show comparison results within a certain period up to the present, or may show comparison results within a period specified by the user.

[0051] The analytical device 100 determines whether or not there is a setting input by pressing the setting button 521 (step S1002). If there is a setting input (the setting button 521 is pressed) (step S1002: Yes), the analytical device 100 executes a setting process (step S1005).

[0052] On the other hand, if there is no setting input (set button 521 is not pressed) (step S1002: No), analysis device 100 determines whether there is a learning start input by pressing learn button 522 (step S1003). If there is a learning start input by pressing learn button 522 (step S1003: Yes), analysis device 100 executes a learning process (step S1006).

[0053] On the other hand, if there is no learning start input (learning button 522 is not pressed) (step S1003: No), analysis device 100 determines whether there is an inference start input by pressing inference button 523 (step S1004). If there is an inference start input by pressing inference button 523 (step S1004: Yes), analysis device 100 executes inference processing (step S1007). On the other hand, if there is no inference start input (inference button 523 is not pressed) (step S1004: No), the process returns to step S1002.

[0054] <Figure 11 Setting process (step S1005)> Fig. 11 is a flowchart showing a detailed example of the process procedure of the setting process (step S1005) shown in Fig. 10. The analysis device 100 displays input destination learning model selection information in the third area 503 (step S1101).

[0055] The analysis device 100 determines whether or not a graph has been selected (step S1102). Specifically, for example, the analysis device 100 determines whether or not the all learning button 822 or the individual learning button 831 has been pressed. When the all learning button 822 is pressed, all learning models M# become targets for association in step S1105, and when the individual learning button 831 is pressed, the learning model M# corresponding to the individual learning button 831 becomes targets for association in step S1105.

[0056] If no graph is selected (step S1102: No), the process proceeds to step S1106. If a graph is selected (step S1102: Yes), the analysis device 100 acquires, from the dashboard 300, a coordinate value P indicating the position of the graph G selected in the graph selection (step S1102) (hereinafter, the selected graph G) (step S1103).

[0057] The analysis device 100 determines whether the selected graph G is associated with the learning model M# (step S1104). If the coordinate value P of the graph G is registered as the position information 206 in the entry of the learning model M# in the learning model DB 200, the graph G and the learning model M# are associated with each other.

[0058] If the selected graph G is already associated with the learning model M# (step S1104: Yes), the process proceeds to step S1106. On the other hand, if the selected graph G is not associated with the learning model M# (step S1104: No), the analysis device 100 associates the selected graph G with the learning model M# (step S1105) and proceeds to step S1106. Specifically, for example, the analysis device 100 registers the coordinate value P of the selected graph G in an entry of the position information 206 of the graph-selected learning model M#.

[0059] The analysis device 100 determines whether or not the setting is finished (step S1106). Specifically, for example, the analysis device 100 determines whether or not the setting finish button 621 has been pressed. If the setting finish button 621 has not been pressed (step S1106: No), the process returns to step S1102. On the other hand, if the setting finish button 621 has been pressed (step S1106: Yes), the analysis device 100 ends the setting process (step S1005).

[0060] <FIG. 12 Learning process (step S1006)> Fig. 12 is a flowchart showing a detailed example of the process procedure of the learning process (step S1006) shown in Fig. 10. The learning process (step S1006) is executable when the setting process (step S1005) has been executed.

[0061] The analysis device 100 selects a learning model M# to be learned (step S1201). The learning model M# to be learned (hereinafter, the selected learning model M# in FIG. 12) is all learning models M# in the learning model DB200 when the All Learning button 822 is pressed, and is the learning model M# when the Individual Learning button 831 is pressed.

[0062] The analysis device 100 determines whether or not it is necessary to collect the training data TR and the supervised data GT based on the user's operation input on the input device 103 (step S1202). If the training data TR and the supervised data GT for the selected learning model M# have already been registered in the learning model DB 200, the analysis device 100 determines that it is not necessary to collect the training data TR and the supervised data GT (step S1202: No).

[0063] However, if a collection request is accepted by a user operating the input device 103, the analysis device 100 determines that it is necessary to collect the training data TR and the supervised data GT (step S1202: Yes). Also, if the training data TR and the supervised data GT are not registered in the entry of the selected learning model M# in the learning model DB 200, the analysis device 100 determines that it is necessary to collect the training data TR and the supervised data GT (step S1202: Yes).

[0064] If it is determined that the training data TR and the supervised data GT need to be collected (step S1202: Yes), the process proceeds to step S1203. If it is determined that the training data TR and the supervised data GT do not need to be collected (step S1202: No), the process proceeds to step S1206.

[0065] Analysis device 100 acquires coordinate value P of graph G corresponding to selected learning model M# from position information 206 of learning model DB 200 (step S1203).

[0066] Analysis apparatus 100 generates a screenshot SC of graph G at coordinate value P acquired in step S1203 (hereinafter, acquired coordinate value P in FIG. 12) (step S1204).

[0067] The analysis device 100 acquires the correct answer data GT for the screenshot SC of step S1204, and proceeds to step S1206 (step S1205). Specifically, for example, the analysis device 100 acquires the correct answer data GT that matches the task type 202 and the purpose 203 of the selected learning model M# by the user's operation input of the input device 103, and registers the correct answer data 205 of the learning model DB 200.

[0068] Furthermore, the analysis device 100 may automatically acquire the supervised data GT. For example, the analysis device 100 sets in advance certain analysis data (hereinafter, target analysis data) as supervised data 205 that matches the task type 202 and the purpose 203 of the selected learning model M#, in response to an operation input from a user via the input device 103. The analysis device 100 acquires the target analysis data corresponding to the screenshot SC generated in step S1204 from the analysis target, and registers it in the supervised data 205 of the learning model DB 200.

[0069] If graph G shows a time series change within a certain period up to the present or a comparison result within a certain period up to the present, analysis device 100 acquires from the analysis target the target analysis data at the time when screenshot SC was generated in step S1204, and registers the target analysis data in the learning model DB 200 in the correct answer data 205. If graph G shows a time series change within a period specified by the user, analysis device 100 acquires from the analysis target the target analysis data for the specified period at the time when screenshot SC was generated in step S1204, and registers the target analysis data in the learning model DB 200 in the correct answer data 205.

[0070] Analysis device 100 may repeatedly execute steps S1204 and S1205 for a certain period of time, thereby obtaining a plurality of combinations of screenshots SC (training data TR) and the target analysis data (correct answer data GT) at that time.

[0071] Analysis device 100 inputs screenshot SC to selective learning model M# and outputs learning result LR from selective learning model M# (step S1206).

[0072] The analysis device 100 inputs the learning result LR output in step S1206 and the supervised data GT paired with the screenshot SC input in step S1206 into a loss function to calculate a loss value L (step S1207).

[0073] Analysis device 100 updates selected learning model M# by adjusting learning parameters such as weights so that loss value L is reduced by error backpropagation (step S1208). Steps S1206 to S1208 are repeated for the number of screenshots SC. This ends the learning process (step S1006).

[0074] <Figure 13 Inference process (step S1007)> Fig. 13 is a flowchart showing a detailed example of the processing procedure of the inference process (step S1007) shown in Fig. 10. The inference process (step S1007) is executable when the learning process (step S1006) has already been executed.

[0075] The analysis device 100 selects a learning model M# to be inferred (step S1301). The learning model M# to be inferred (hereinafter, the selected learning model M# in FIG. 13) is all learning models M# in the learning model DB200 when the All Inference button 923 is pressed, and is the learning model M# when the Individual Inference button 931 is pressed.

[0076] Analysis device 100 acquires coordinate value P in selected learning model M# from position information 206 (step S1302).

[0077] Analysis apparatus 100 generates a screenshot SC of graph G at coordinate value P acquired in step S1302 (hereinafter, acquired coordinate value P in FIG. 13) (step S1303).

[0078] The analysis device 100 inputs the screenshot SC generated in step S1303 into the selected learning model M#, and outputs the inference result IR from the selected learning model M# (step S1304). Specifically, for example, the analysis device 100 displays the inference result IR on a display or transmits it to another computer that can communicate with the analysis device 100. This ends the inference process (step S1007).

[0079] Thus, according to the first embodiment, even if the user does not know which analytical data is appropriate as the training data TR for learning the learning model M#, the user can generate a desired learning model M# by using the screenshot SC of the graph G arranged on the dashboard 300 as the training data TR. Furthermore, when converting analytical data into training data, if the analytical data is expressed as the graph G on the dashboard 300, the training data TR is aggregated into image data called the screenshot SC of the graph G, so that the analytical data can be converted into training data without being aware of the data format of the analytical data. EXAMPLES

[0080] A description will be given of Example 2. Example 2 is an example in which the technology of Example 1 is applied to federated learning. In Example 2, differences from Example 1 will be mainly described, and therefore descriptions of overlapping parts with Example 1 will be omitted.

[0081] <Example of federated learning system configuration> 14 is an explanatory diagram showing an example of the system configuration of a federated learning system. Federated learning system 1400 has server 1401 and clients 1402-1 to 1402-f (f is an integer equal to or greater than 1). When clients 1402-1 to 1402-f do not need to be distinguished from one another, they are referred to as clients 1402. Server 1401 and clients 1402 are communicatively connected via a first network 1410 such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0082] Clients 1402-1 to 1402-f are local learning devices installed at sites ST1 to STf, respectively. When clients 1402-1 to 1402-f are not distinguished from one another, they are referred to as clients 1402. Client 1402 has one or more learning models M#. The learning model M# in client 1402 is referred to as a local learning model LM#.

[0083] When there is no need to distinguish between the sites ST1 to STf (including the site STx described later), they are referred to as site ST. The sites ST1 to STf are communicatively connected to sensors 1431-1 to 1431-f and terminals 1432-1 to 1432-f installed in the work area via second networks 1420-1 to 1420-f such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0084] When the sensors 1431-1 to 1431-f are not distinguished from one another, they are referred to as sensors 1431. When the terminals 1432-1 to 1432-f are not distinguished from one another, they are referred to as terminals 132. When the second networks 1420-1 to 1420-f are not distinguished from one another, they are referred to as second networks 1420.

[0085] The work area is an area in the site ST where a specific work is performed, and various sensors 1431 such as a surveillance camera, an infrared sensor, a vibration sensor, and an acceleration sensor are installed according to the specific work. Data detected by these sensors 1431 is transmitted to the client 1402 via the second network 1420. Data from the terminal 1432 is also transmitted to the client 1402 via the second network 1420.

[0086] For example, the sites ST1 and ST2 are construction sites, and various sensors 1431 are installed in a heavy machinery work area and a human work area. The site STf is a manufacturing site, and various sensors 1431 are installed in an assembly manufacturing area and a human work area.

[0087] Server 1401 is a federated learning device that integrates learning parameters of local learning models LM# of clients 1402-1 to 1402-f to generate global learning parameters. Server 1401 distributes the global learning parameters to clients 1402-1 to 1402-f, whereby clients 1402-1 to 1402-f update their local learning parameters.

[0088] In the second embodiment, the sensor 1431, the terminal 1432, and the management systems (not shown) in the heavy machinery work area, the human work area, and the assembly manufacturing area in the site ST are analyzed. The hardware configurations of the server 1401, the client 1402, and the terminal 1432 are the same as those shown in FIG.

[0089] In the case of the federated learning system 1400 , the dashboard 300 is displayed on a server 1401 and a client 1402 .

[0090] <Figure 15 Global Learning DB> 15 is an explanatory diagram showing an example of a global learning DB 1500. The global learning DB 1500 is stored in the storage device 102 of the server 1401 or another device capable of communicating with the server 1401.

[0091] Global learning DB 1500 is a database that stores information related to global learning parameters, and has the following fields: global ID 1501, task type 202, objective 203, location information 206, local learning parameter group 1507, and global learning parameters 1508. Global ID 1501 is identification information that uniquely identifies global learning parameters 1508.

[0092] The local learning parameter set 1507 is a collection of local learning parameters θ# set in the local learning model LM# of each client 1402. The global learning parameters 1508 are learning parameters obtained by averaging the local learning parameter set 1507. The global learning parameters 1508 are transmitted to each client 1402.

[0093] <Fig. 16 Local learning model DB> 16 is an explanatory diagram showing an example of a local learning model DB 1600. The local learning model DB 1600 is stored in the storage device 102 of the client 1402 or the terminal 1432 or in another device capable of communicating with the client 1402 or the terminal 1432.

[0094] The local learning model DB1600 is a database that stores information about a local learning model LM# specific to the site ST, and has as fields a local ID 1601, a task type 202, a purpose 203, local training data 1604, local answer data 1605, and location information 206. The local ID 1601 is identification information that uniquely identifies the local learning model LM#, and serves as a pointer to the local learning model LM#.

[0095] The local training data 1604 is used for learning the local learning model LM#, and is data that pairs with the local supervised answer data 1605. The local training data 1604 is a screenshot SC that is input to the local learning model LM#.

[0096] When there is no need to distinguish between the values ​​LTR1, LTR2, . . . of the local training data 1604, they are simply referred to as LTR.

[0097] The local superscript data 1605 is used for learning the local learning model LM#, and is paired with the local training data 1604. For example, if the task type 202 is identification and the objective 203 is fault diagnosis, the local superscript data 1605 (LGT1) is a numerical value indicating a fault or normality of the analysis target at the time of displaying the graph from which the paired local training data 1604 (LTR1) is captured.

[0098] If the task type 202 is regression and the objective 203 is inventory forecasting, the local ground truth data 1605 (LGT2) is the actual inventory value of the analysis target at the time of display of the graph from which the corresponding local training data 1604 (LTR2) is captured. The local ground truth data 1605 is used to determine the value of the loss function for the local learning model LM#.

[0099] When there is no need to distinguish between the values ​​LGT1, LGT2, . . . of the local answer data 1605, they are simply written as LGT.

[0100] <Figure 17 Setting process> 17 is a sequence diagram showing a first example of a setting process in the federated learning system 1400. In FIG. 17, a process in which the server 1401 executes a setting process and distributes setting information to the client 1402 is shown.

[0101] The server 1401 displays the graph G on the dashboard 300 (step S1701). Note that since the server 1401 is not connected to the sensors 1431 and terminals 1432 of each site ST to be analyzed, graph G does not display graph data such as waveforms, but instead displays coordinate axes and explanatory text.

[0102] The server 1401 executes the setting process (step S1702). The setting process (step S1702) is the same as the setting process (step S1005). That is, the server 1401 executes the setting process (step S1005), and the global learning parameters 1508, not the learning model M#, are associated with the coordinate value P of the selected graph G in the position information 206 in the global learning DB 1500.

[0103] The server 1401 transmits each entry of the global learning DB 1500 set by the setting process (step S1702) as setting information to each client 1402 (step S1703).

[0104] The client 1402 receives the setting information from the server 1401 (step S1704). The client 1402 registers the received setting information in the local learning model DB 1600 (step S1705). Specifically, for example, the client 1402 registers the global ID 1501 in the setting information as the local ID 1601 in the local learning model DB 1600, and registers the task type 202 and the purpose 203 in the same entry as the local ID 1601.

[0105] After that, each client 1402 executes a local learning process (step S1706). Specifically, for example, the dashboard 300 shown in Fig. 8 is displayed on the client 1402, and the client 1402 executes steps S1001 and S1003 shown in Fig. 10 and the learning process (step S1006) shown in Fig. 12. In this case, the learning model M# to be learned in Fig. 12 becomes the local learning model LM#, and the training data 204 and the supervised data 205 become the local training data 1604 and the local supervised data 1605.

[0106] Fig. 18 is a sequence diagram showing a second example of a setting process in the federated learning system 1400. Fig. 18 shows a process in which, in response to a request from a client 1402, a server 1401 searches for a setting process and delivers setting information to the requesting client 1402.

[0107] The client 1402 transmits setting request information to the server 1401 (step S1801). The setting request information includes the task type 202, the purpose 203, and the location information 206.

[0108] The server 1401 receives the setting request information from the client 1402 (step S1802). The server 1401 searches the global learning DB 1500 for the global ID 1501 and the location information 206 of an entry that matches the task type 202 and the purpose 203 of the received setting request information (step S1803).

[0109] Server 1401 transmits searched global ID 1501 and location information 206 as setting information to client 1402, which is the sender of the setting request information (step S1804). If server 1401 has global learning parameter Θ# that has been subjected to associative learning, server 1401 may transmit the learning parameter by including it in the setting information.

[0110] The client 1402 receives the setting information (step S1805). The client 1402 sets the global ID 1501 of the received setting information (global ID 1501 and location information 206) as the local ID 1601, and registers the location information 206 and the setting request information (objective 203 and location information 206) in the same entry as the local ID 1601 in the local learning model DB 1600 (step S1806).

[0111] If the setting information includes learning parameters, the client 1402 generates a local learning model LM# using the learning parameters.

[0112] Thereafter, the client 1402 executes local learning processing (step S1807), similar to step S1706.

[0113] <Figure 19 Associative learning process> 19 is a sequence diagram showing an example of a federated learning process by the federated learning system 1400. The server 1401 transmits the global ID 1501 to each client 1402 (step S1901).

[0114] Each client 1402 receives the global ID 1501 from the server 1401 (step S1902). Each client 1402 transmits the local learning parameter θ# of the local learning model LM# identified by the local ID 1601 that matches the global ID 1501 to the server 1401 (step S1903).

[0115] The server 1401 receives a set of local learning parameters θ#s, which are local learning parameters θ#, from each client 1402 (step S1904).

[0116] The server 1401 updates the local learning parameter set 1507 in the global learning DB 1500 with the local learning parameter set θ#s (step S1905).

[0117] The server 1401 integrates the local learning parameters θ#s by averaging them to generate global learning parameters Θ#, and registers them in the global learning parameters 1508 (step S1906).

[0118] The server 1401 transmits the global learning parameter Θ# to each client 1402 (step S1907).

[0119] Each client 1402 receives the global learning parameters Θ# (step S1908). Each client 1402 updates the local learning model LM# with the global learning parameters Θ# (step S1909).

[0120] Thereafter, each client 1402 executes the inference process (step S1007) shown in Fig. 13 using the local learning model LM# to output the inference result IR. Also, each client 1402 updates the local learning parameter θ# by executing the local learning process (steps S1706, S1807). The latest local learning parameter θ# is transmitted to the server 1401 in step S1903.

[0121] In this way, according to the second embodiment, the setting process (step S1005), the learning process (step S1006), and the inference process (step S1007) shown in the first embodiment can be applied to the federated learning. Therefore, leakage of the local training data 1604 and the local supervised answer data 1605 from the client 1402 can be suppressed.

[0122] As described above, according to the above-mentioned first and second examples, the analytical data of the analysis target is broken down for each visualized graph, so that even if the analysis targets provide the same functions, and the analytical data is expressed differently on the graph due to differences in users or development vendors, the differences can be absorbed.

[0123] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0124] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0125] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0126] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0127] 100 Analyzer 101 Processor 102 Storage Devices 200 Learning Model DB 203 Purpose 204 Training Data 205 Correct Data 206 Location information 300 Dashboard 301 Display area 302 Hidden area 303 Scrollbar 1400 Federated Learning System 1401 Server 1402 Client 1431 Sensors 1432 Terminal 1507 Local Learning Parameter Set 1508 Global Learning Parameters 1604 Local Training Data 1605 Local Answer Data 1500 Global Learning DB 1600 Local Learning Model DB G Graph GT correct answer data IR inference results LM# Local Learning Model LR learning results P coordinate value SC Screenshots TR training data Θ# Global learning parameters θ# Local learning parameters

Claims

1. An analysis apparatus having a processor that executes a program and a storage device that stores the program, The processor, a first generation process for generating first image data of a graph related to an analysis target from a layout area on a display screen in which the graph is laid out, as training data to be input to a learning model related to the analysis target; an acquisition process for acquiring target data related to the analysis target corresponding to the graph as correct answer data to be used in learning of the learning model when the first image data is generated by the first generation process; a learning process for learning the learning model based on a learning result output as a result of inputting the first image data into the learning model and target data acquired by the acquisition process; An analytical device characterized by performing the above.

2. The analysis device according to claim 1 , a plurality of the graphs are arranged in the arrangement area, the arrangement area including a display area in which a first graph of the plurality of graphs is displayed, and a non-display area in which a second graph of the plurality of graphs, which is different from the first graph, is not displayed; 1. An analytical device comprising:

3. The analysis device according to claim 1 , The graph is created based on at least one type of analysis data among data input to the analysis target, data detected by the analysis target, data obtained by detecting a behavior of the analysis target, data generated by an internal process of the analysis target, and data output from the analysis target.

1. An analytical device comprising:

4. The analysis device according to claim 1 , In the first generation process, when a plurality of the graphs are arranged in the arrangement area, the processor generates the first image data for each of the graphs; In the acquisition process, the processor acquires the target data for each combination of the first image data of two or more of the graphs, In the learning process, the processor learns the learning model based on a learning result output as a result of inputting the first image data of each of the two or more graphs into the learning model and the target data.

1. An analytical device comprising:

5. The analysis device according to claim 1 , The processor, a second generation process for generating second image data of the graph; an inference process for inputting the second image data generated by the second generation process into the learning model and outputting an inference result; An analytical device characterized by performing the above.

6. The analysis device according to claim 4, The processor, a second generation process for generating second image data for each of the two or more graphs; an inference process for inputting a combination of two or more second image data generated by the second generation process into the learning model, and outputting an inference result; An analytical device characterized by performing the above.

7. An analysis method using an analysis apparatus having a processor that executes a program and a storage device that stores the program, comprising: The processor, a first generation process for generating first image data of a graph related to an analysis target from a layout area on a display screen in which the graph is laid out, as training data to be input to a learning model related to the analysis target; an acquisition process for acquiring target data related to the analysis target corresponding to the graph as correct answer data to be used in learning of the learning model when the first image data is generated by the first generation process; a learning process for learning the learning model based on a learning result output as a result of inputting the first image data into the learning model and target data acquired by the acquisition process; An analysis method comprising the steps of:

8. The processor: a first generation process for generating first image data of a graph related to an analysis target from a layout area on a display screen in which the graph is laid out, as training data to be input to a learning model related to the analysis target; an acquisition process for acquiring target data related to the analysis target corresponding to the graph as correct answer data to be used in learning of the learning model when the first image data is generated by the first generation process; a learning process for learning the learning model based on a learning result output as a result of inputting the first image data into the learning model and target data acquired by the acquisition process; An analysis program characterized by executing the above.

Images