Analysis device, analysis method, and analysis program

Abstract

This analysis device having a processor for executing a program and a storage device for storing the program executes: first generation processing for generating first image data of a graph pertaining to an analysis target from an arrangement region on a display screen on which the graph is arranged, as training data input to a learning model pertaining to the analysis target; acquisition processing for, when the first image data is generated by the first generation processing, acquiring target data pertaining to the analysis target corresponding to the graph as correct answer data used for training the learning model; and training processing for training the learning model on the basis of a training result output as a result of inputting the first image data to the learning model, and target data acquired by the acquisition processing.

Description

Analytical device, analytical method, and analytical program Incorporation by Reference

[0001] This application claims priority from Japanese Patent Application No. 2023-118610, filed on July 20, 2023, the contents of which are incorporated herein by reference.

[0002] The present invention relates to an analysis device, an analysis method, and an analysis program for generating a learning model and performing inference using the learning model.

[0003] The use of AI (Artificial Intelligence) for the purpose of improving productivity is beginning to increase in various workplaces, such as manufacturing sites, farms, and software development factories. One example of the use of AI is to analyze productivity, quality, etc. by analyzing time-series data or non-time-series data. Introducing AI requires the extraction of the data from databases of various systems, the generation of learning models using large amounts of diverse data, and analysis using the learning models, which incurs significant costs.

[0004]

[0005] Patent Literature 1 below describes a method, system, and device for reading equipment in a factory or plant environment, including capturing an image of an environment including a measuring device, detecting a target area included in the image, the target area including at least a portion of the measuring device, determining identification information associated with the measuring device based on detecting the target area, and extracting measurement information associated with the measuring device based on detecting the target area. In some aspects, detecting the target area may include providing the image to a machine learning network, and receiving output from the machine learning network in response to the machine learning network processing the image based on a detection model, the output including the target area.

[0005] The following Patent Document 2 includes, for the current learning cycle, identifying a target device for each of at least one learning task to be executed from among a plurality of candidate devices based on resource information of the plurality of candidate devices; transmitting a global model for each task to the target device for each task, thereby causing the target device for each task to train the global model for each task; and, in response to receiving the trained models transmitted to all target devices for each task, updating the global model for each task based on the trained model, thereby completing the current learning cycle.

[0006] JP 2023-29227 A JP 2022-136234 A

[0007] However, production management systems and quality control systems often handle not just one type of data, but many different types of information. In Patent Document 1, only the range of on-site meters that can be seen from the outside is captured. Therefore, it cannot handle situations where scrolling of the display screen is required. As a result, the data displayed on the dashboard by the production management system does not fit on one display screen, and in order to read all the data, the display screen must be scrolled with a mouse. In other words, Patent Document 1 does not take into consideration the acquisition of information on a dashboard, which displays a large amount of information and requires scrolling to display the entire data.

[0008] Furthermore, since Patent Document 1 is based on one image per piece of information, when multiple pieces of information are mixed on one screen, it does not take into consideration converting them into data and inputting them into the system of Patent Document 2. Furthermore, when converting multiple pieces of information into data, if the data formats are different, data shaping is required when inputting them into a learning model, but Patent Documents 1 and 2 do not take data shaping into consideration.

[0009] An object of the present invention is to improve the efficiency of learning.

[0010] An analysis device that is one aspect of the invention disclosed in the present application is an analysis device having a processor that executes a program and a storage device that stores the program, wherein the processor executes a first generation process that generates first image data of a graph related to an analysis target from a placement area on a display screen in which the graph is placed, as training data to be input into a learning model related to the analysis target; an acquisition process that, when the first image data is generated by the first generation process, acquires target data related to the analysis target corresponding to the graph as correct answer data to be used in learning the learning model; and a learning process that learns the learning model based on the learning result output as a result of inputting the first image data into the learning model and the target data acquired by the acquisition process.

[0011] According to the representative embodiment of the present invention, it is possible to improve the efficiency of learning. Problems, configurations and effects other than those described above will become clear from the following description of the embodiment.

[0012] FIG. 1 is a block diagram showing an example of the hardware configuration of an analysis device. FIG. 2 is an explanatory diagram showing an example of a learning model DB. FIG. 3 is an explanatory diagram showing an example of screenshot acquisition. FIG. 4 is an explanatory diagram showing an example of learning of learning model M#. FIG. 5 is an explanatory diagram showing dashboard example 1. FIG. 6 is an explanatory diagram showing dashboard example 2. FIG. 7 is an explanatory diagram showing dashboard example 3. FIG. 8 is an explanatory diagram showing dashboard example 4. FIG. 9 is an explanatory diagram showing dashboard example 5. FIG. 10 is a flowchart showing an example of an analysis processing procedure by an analysis device. FIG. 11 is a flowchart showing an example of detailed processing procedures of the setting process (step S1005) shown in FIG. 10. FIG. 12 is a flowchart showing an example of detailed processing procedures of the learning process (step S1006) shown in FIG. 10. FIG. 13 is a flowchart showing an example of detailed processing procedures of the inference process (step S1007) shown in FIG. 10. FIG. 14 is an explanatory diagram showing an example of the system configuration of a federated learning system. FIG. 15 is an explanatory diagram showing an example of a global learning DB. FIG. 16 is an explanatory diagram showing an example of a local learning model DB. Fig. 17 is a sequence diagram showing example 1 of a setting process in a federated learning system. Fig. 18 is a sequence diagram showing example 2 of a setting process in a federated learning system. Fig. 19 is a sequence diagram showing an example of federated learning processing by the federated learning system.

[0013] <FIG. 1: Example of Hardware Configuration of Analytical Apparatus> FIG. 1 is a block diagram showing an example of the hardware configuration of an analytical apparatus. The analytical apparatus 100 includes a processor 101, a storage device 102, an input device 103, an output device 104, and a communication interface (communication IF) 105. The processor 101, the storage device 102, the input device 103, the output device 104, and the communication IF 105 are connected via a bus 106. The processor 101 controls the analytical apparatus 100. The storage device 102 serves as a working area for the processor 101. The storage device 102 is a non-transitory or temporary recording medium that stores various programs and data. Examples of the storage device 102 include a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 103 inputs data. Examples of the input device 103 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 104 outputs data. Examples of the output device 104 include a display, a printer, and a speaker. The communication IF 105 connects to a network and transmits and receives data.

[0014] 2 is an explanatory diagram showing an example of a learning model DB. The learning model DB 200 is stored in the storage device 102 of the analysis device 100 or another device that can communicate with the analysis device 100.

[0015] The learning model DB 200 is a database that stores information about learning models, and has the following fields: model ID 201, task type 202, objective 203, training data 204, correct answer data 205, and location information 206. The model ID 201 is identification information that uniquely identifies a learning model, and serves as a pointer to the learning model. Hereinafter, a learning model whose model ID 201 value is "M#" (# is a unique number) will be referred to as learning model M#.

[0016] The task type 202 indicates the type of task (e.g., identification, regression, etc.) executed by the learning model M#. The objective 203 indicates the processing content (e.g., fault diagnosis, inventory prediction, etc.) for the learning model M# to output a target variable, and indicates the attributes of the target variable that is the output result (e.g., fault diagnosis result, inventory prediction value, etc.).

[0017] The training data 204 is used for learning the learning model M# and is data paired with the correct answer data 205. The training data 204 is a screenshot input to the learning model M#.

[0018] A screenshot is image data captured from a dashboard. A graph is a chart that shows time-dependent changes or comparison results of various data (hereinafter referred to as analytical data), such as data input to an analytical target (e.g., a production management system or quality control system that manages an assembly manufacturing line, a work management system that manages the behavior of heavy equipment in a heavy equipment work area, or a human resource management system that uses cameras to authenticate workers and manage their behavior), data detected by the analytical target, data detected from the behavior of the analytical target, data generated by the internal processing of the analytical target, data output from the analytical target, or statistical values ​​such as the average value or standard deviation of these data. Examples of graphs include waveform diagrams, histograms, pie charts, scatter plots, radar charts, and box-and-whisker plots. Note that analytical targets are not limited to production management systems or quality control systems.

[0019] When there is no need to distinguish between the values ​​TR1, TR2, . . . of the training data 204, they are simply referred to as TR.

[0020] The supervised answer data 205 is used in learning the learning model M# and is objective data that indicates the objective 203 related to the analysis object that is paired with the training data 204. For example, if the task type 202 is identification and the objective 203 is fault diagnosis, the supervised answer data 205 (GT1) is a numerical value that indicates a fault or normal state of the analysis object at the time when the graph from which the paired training data 204 (TR1) is captured is displayed. Also, if the task type 202 is regression and the objective 203 is inventory forecasting, the supervised answer data 205 (GT2) is the actual measured value of the inventory of the analysis object at the time when the graph from which the paired training data 204 (TR2) is captured is displayed. The supervised answer data 205 is used to calculate the value of the loss function related to the learning model M#.

[0021] When there is no need to distinguish between the values ​​GT1, GT2, . . . of the correct answer data 205, they are simply written as GT.

[0022] The position information 206 is information indicating the position of the graph on the dashboard, which is the input source. The position information 206 may be the vertex coordinate values ​​of the screenshot or the center of gravity position. The position information 206 is specified by the user via the input device 103 and stored in the learning model DB 200.

[0023] When the values ​​P1, 92, . . . of the position information 206 are not distinguished, they are simply referred to as coordinate values ​​P.

[0024] <Fig. 3: Example of Screenshot Acquisition> Fig. 3 is an explanatory diagram showing an example of screenshot acquisition. Dashboard 300 is an arrangement area on a display screen displayed on a display, which is an example of output device 104 of analysis apparatus 100. Dashboard 300 has at least display area 301, including display area 301 and non-display area 302. Display area 301 is an area visible to the user of analysis apparatus 100, and non-display area 302 is an area not visible to the user of analysis apparatus 100. The user operates scroll bar 303 to change display area 301 and non-display area 302.

[0025] 3, graphs G1 to G4 are arranged in the display area 301, and graphs G5 and G6 are arranged in the non-display area 302. When there is no need to distinguish between the graphs G1 to G6, they will be referred to as graph G.

[0026] The analysis device 100 captures screenshot groups SC1s to SC4s of graphs G1 to G4 arranged in the display area 301, and also captures screenshot groups SC5s and SC6s of graphs G5 and G6 arranged in the non-display area 302.

[0027] Screenshot groups SC1s to SC6s are a chronological collection of screenshots SC1 to SC6 of graphs G1 to G6. When there is no need to distinguish between screenshot groups SC1s to SC6s, they are referred to as screenshot group SCs. When there is no need to distinguish between screenshots SC1 to SC6, they are referred to as screenshots SC.

[0028] <Fig. 4 Learning Example> Fig. 4 is an explanatory diagram showing a learning example of learning model M#. Fig. 4 explains learning models M1 to M4 as an example. Learning model M1 is trained using a group of screenshots SC1s of graph G1 and its correct answer data GT1. Learning model M# is configured, for example, by a convolutional neural network.

[0029] During learning, when a group of screenshots SC1s, which is training data TR1, is input, the learning model M1 outputs a learning result RL1. The analysis device 100 inputs the learning result LR1 and the correct answer data GT1 into a loss function to calculate a loss value L1. The analysis device 100 updates the learning model M1 by adjusting learning parameters such as weights through error backpropagation so that the loss value L1 is reduced.

[0030] During inference, when a screenshot SC1 is input, the learning model M1 outputs an inference result IR1.

[0031] The learning model M2 is trained using the screenshot group SC1s of the graph G1 and the screenshot group SC2s of the graph G2 and their correct answer data GT1 and GT2.

[0032] During learning, when the training data TR2, which is a set of screenshots SC1s and SC2s, is input, the learning model M2 outputs a learning result RL2. The analysis device 100 inputs the learning result LR2 and the correct answer data GT2 into a loss function to calculate a loss value L2. The analysis device 100 updates the learning model M2 by adjusting learning parameters such as weights through error backpropagation so that the loss value L2 decreases.

[0033] During inference, when certain screenshots SC1 and SC2 are input, the learning model M2 outputs an inference result IR2.

[0034] The learning model M3 is trained using the screenshot group SC1s of the graph G1, the screenshot group SC2s of the graph G2, and the screenshot group SC3s of the graph G3, and their correct answer data GT1 to GT3.

[0035] During learning, when the group of screenshots SC1s to SC3s, which are training data TR3, is input, the learning model M3 outputs a learning result RL3. The analysis device 100 inputs the learning result LR3 and the correct answer data GT3 into a loss function to calculate a loss value L3. The analysis device 100 updates the learning model M3 by adjusting learning parameters such as weights through error backpropagation so that the loss value L3 decreases.

[0036] During inference, when certain screenshots SC1 to SC3 are input, the learning model M3 outputs an inference result IR3.

[0037] The learning model M4 is trained using screenshot group SC1s of graph G1, screenshot group SC2s of graph G2, screenshot group SC3s of graph G3, and screenshot group SC4s of graph G4, and their correct answer data GT1 to GT4.

[0038] During learning, when a group of screenshots SC1s to SC4s, which are training data TR4, is input, learning model M4 outputs learning result RL4. Analysis device 100 inputs learning result LR4 and correct answer data GT4 into a loss function to calculate loss value L4. Analysis device 100 updates learning model M4 by adjusting learning parameters such as weights through error backpropagation so that loss value L4 is reduced.

[0039] During inference, when certain screenshots SC1 to SC4 are input, the learning model M4 outputs an inference result IR4.

[0040] When there is no distinction between the learning results LR1 to LR4, they are simply referred to as the learning result LR. When there is no distinction between the inference results IR1 to IR4, they are simply referred to as the inference result IR. When there is no distinction between the loss values ​​L1 to L4, they are simply referred to as the loss value L.

[0041] <Dashboard 300: Figures 5 to 9> Figure 5 is an explanatory diagram showing example 1 of the dashboard 300. The dashboard 300 has a first area 501, a second area 502, and a third area 503. The first area 501 is an area where the graph G is placed, and includes the display area 301 and non-display area 302 described above.

[0042] The second area 502 is an area for displaying a group of buttons. The group of buttons includes a setting button 521, a learning button 522, and an inference button 523. The setting button 521 is a user interface for executing a setting process, which will be described later, when pressed. The learning button 522 is a user interface for executing a learning process, which will be described later, when pressed. The inference button 523 is a user interface for executing an inference process, which will be described later, when pressed.

[0043] The third area 503 is an area that displays details when one of the buttons is pressed. The third area 503 will be described with reference to FIG. 6 and subsequent figures.

[0044] Fig. 6 is an explanatory diagram showing example 2 of the dashboard 300. Fig. 6 shows a display example when the setting button 521 is pressed in the display state of Fig. 5. Pressing the setting button 521 changes it to an end setting button 621. The end setting button 621 is a user interface for ending the setting process when pressed.

[0045] When the setting button 521 is pressed, a group of check boxes B1, B2, B3, ... (when not distinguishing between them, they are referred to as check box group B) for selecting a graph G for each learning model M# is displayed as input destination learning model selection information in the third area 503. The graph G selected in check box group B becomes the training data TR for that learning model M#.

[0046] FIG. 7 is an explanatory diagram showing Example 3 of the dashboard 300. FIG. 7 shows a display example when the setting button 521 is pressed in the display state of FIG. 6. Graph G1 is selected in checkbox group B1, graphs G1 and G2 are selected in checkbox group B2, and graphs G1 to G3 are selected in checkbox group B3. Therefore, the screenshot group SC1s of graph G1 becomes training data TR1 for the learning model M1, the screenshot groups SC1s and SC2s of graphs G1 and G2 become training data TR2 for the learning model M2, and the screenshot groups SC1s to SC3s of graphs G1 to G3 become training data TR3 for the learning model M3.

[0047] Fig. 8 is an explanatory diagram showing Example 4 of the dashboard 300. Fig. 8 shows a display example when the learning button 522 is pressed in the display state of Fig. 5. Pressing the learning button 522 changes it to an all learning button 822. The all learning button 822 is a user interface for executing the learning process for all learning models M# when pressed.

[0048] When the learning button 522 is pressed, individual learning buttons 831-1, 831-2, 831-3, ... (when not distinguishing between them, referred to as individual learning button 831) are displayed for each learning model M# in the third area 503. The analysis device 100 learns the learning model M# for which the individual learning button 831 was pressed, using the training data TR and the supervised answer data GT.

[0049] 9 is an explanatory diagram showing Example 5 of the dashboard 300. Fig. 9 is a display example when the inference button 523 is pressed in the display state of Fig. 5. Pressing the inference button 523 changes it to an all inference button 923. The all inference button 923 is a user interface for executing inference processing for all learning models M# when pressed.

[0050] When the inference button 523 is pressed, individual inference buttons 931-1, 931-2, 931-3, ... (when not distinguishing between them, they are referred to as individual inference buttons 931) are displayed for each learning model M# in the third area 503. The analysis device 100 outputs an inference result IR for the learning model M# for which the individual inference button 931 was pressed by capturing and inputting a screenshot SC of the graph G located in its position information 206.

[0051] <Fig. 10 Analysis Processing> Fig. 10 is a flowchart showing an example of the analysis processing procedure by the analysis apparatus 100. The analysis apparatus 100 acquires analysis data and displays a graph G on the dashboard 300 (step S1001). The displayed graph G includes a graph G showing time-series changes in analysis data and a graph G comparing the same type of analysis data. The graph G showing time-series changes in analysis data may show time-series changes within a certain period up to the present, or within a period specified by the user. The graph G comparing the same type of analysis data may also show the comparison results within a certain period up to the present, or within a period specified by the user.

[0052] The analysis device 100 determines whether or not a setting has been input by pressing the setting button 521 (step S1002). If a setting has been input (the setting button 521 has been pressed) (step S1002: Yes), the analysis device 100 executes a setting process (step S1005).

[0053] On the other hand, if there is no setting input (the setting button 521 is not pressed) (step S1002: No), the analysis device 100 determines whether there is a learning start input by pressing the learning button 522 (step S1003). If there is a learning start input by pressing the learning button 522 (step S1003: Yes), the analysis device 100 executes the learning process (step S1006).

[0054] On the other hand, if there is no learning start input (the learning button 522 is not pressed) (step S1003: No), the analysis device 100 determines whether there is an inference start input by pressing the inference button 523 (step S1004). If there is an inference start input by pressing the inference button 523 (step S1004: Yes), the analysis device 100 executes the inference process (step S1007). On the other hand, if there is no inference start input (the inference button 523 is not pressed) (step S1004: No), the process returns to step S1002.

[0055] 11 is a flowchart showing a detailed example of the procedure of the setting process (step S1005) shown in Fig. 10. The analysis apparatus 100 displays input destination learning model selection information in the third area 503 (step S1101).

[0056] The analysis apparatus 100 determines whether a graph has been selected (step S1102). Specifically, for example, the analysis apparatus 100 determines whether the All Learning button 822 or the Individual Learning button 831 has been pressed. If the All Learning button 822 has been pressed, all learning models M# become targets for association in step S1105, and if the Individual Learning button 831 has been pressed, the learning model M# corresponding to the Individual Learning button 831 becomes targets for association in step S1105.

[0057] If no graph is selected (step S1102: No), the process proceeds to step S1106. If a graph is selected (step S1102: Yes), the analysis apparatus 100 acquires, from the dashboard 300, coordinate values ​​P indicating the position of the graph G selected in the graph selection (step S1102) (hereinafter, the selected graph G) (step S1103).

[0058] The analysis apparatus 100 determines whether the selected graph G is associated with the learning model M# (step S1104). If the coordinate value P of the graph G is registered as the position information 206 in the entry of the learning model M# in the learning model DB 200, the graph G and the learning model M# are associated with each other.

[0059] If the selected graph G has already been associated with the learning model M# (step S1104: Yes), the process proceeds to step S1106. On the other hand, if the selected graph G has not been associated with the learning model M# (step S1104: No), the analysis device 100 associates the selected graph G with the learning model M# (step S1105) and proceeds to step S1106. Specifically, for example, the analysis device 100 registers the coordinate value P of the selected graph G in an entry of the position information 206 of the graph-selected learning model M#.

[0060] The analysis apparatus 100 determines whether the setting is complete (step S1106). Specifically, for example, the analysis apparatus 100 determines whether the setting completion button 621 has been pressed. If the setting completion button 621 has not been pressed (step S1106: No), the process returns to step S1102. On the other hand, if the setting completion button 621 has been pressed (step S1106: Yes), the analysis apparatus 100 ends the setting process (step S1005).

[0061] 12 is a flowchart showing a detailed example of the learning process (step S1006) shown in Fig. 10. The learning process (step S1006) can be executed when the setting process (step S1005) has already been executed.

[0062] The analysis device 100 selects a learning model M# to be learned (step S1201). The learning model M# to be learned (hereinafter referred to as the selected learning model M# in FIG. 12 ) is all the learning models M# in the learning model DB 200 when the All Learning button 822 is pressed, or is the learning model M# in question when the Individual Learning button 831 is pressed.

[0063] The analysis device 100 determines whether or not it is necessary to collect the training data TR and the supervised data GT based on the user's operation input via the input device 103 (step S1202). If the training data TR and the supervised data GT for the selected learning model M# have already been registered in the learning model DB 200, the analysis device 100 determines that it is not necessary to collect the training data TR and the supervised data GT (step S1202: No).

[0064] However, if a collection request is received through user input via the input device 103, the analysis device 100 determines that it is necessary to collect the training data TR and the supervised data GT (step S1202: Yes).Furthermore, if the training data TR and the supervised data GT are not registered in the entry for the selected learning model M# in the learning model DB 200, the analysis device 100 determines that it is necessary to collect the training data TR and the supervised data GT (step S1202: Yes).

[0065] If it is determined that the collection of training data TR and correct answer data GT is necessary (step S1202: Yes), the process proceeds to step S1203; if it is determined that the collection of training data TR and correct answer data GT is not necessary (step S1202: No), the process proceeds to step S1206.

[0066] The analysis device 100 acquires the coordinate value P of the graph G corresponding to the selected learning model M# from the position information 206 in the learning model DB 200 (step S1203).

[0067] The analysis apparatus 100 generates a screenshot SC of the graph G at the coordinate value P acquired in step S1203 (hereinafter, the acquired coordinate value P in FIG. 12) (step S1204).

[0068] The analysis apparatus 100 acquires the correct answer data GT for the screenshot SC of step S1204, and proceeds to step S1206 (step S1205). Specifically, for example, the analysis apparatus 100 acquires the correct answer data GT that matches the task type 202 and the purpose 203 of the selected learning model M# in response to a user's operation input on the input device 103, and registers the data as the correct answer data 205 in the learning model DB 200.

[0069] Furthermore, the analysis apparatus 100 may automatically acquire the supervised data GT. For example, the analysis apparatus 100 may set in advance certain analytical data (hereinafter, "target analytical data") as supervised data 205 that matches the task type 202 and purpose 203 of the selected learning model M#, in response to a user's operation input via the input device 103. The analysis apparatus 100 acquires the target analytical data corresponding to the screenshot SC generated in step S1204 from the analysis target, and registers it as supervised data 205 in the learning model DB 200.

[0070] If graph G shows time-series changes within a certain period up to the present or comparison results within a certain period up to the present, analysis device 100 acquires from the analysis target the target analysis data at the time when screenshot SC was generated in step S1204, and registers the data in supervised answer data 205 of learning model DB 200. If graph G shows time-series changes within a period specified by the user, analysis device 100 acquires from the analysis target the target analysis data for the specified period at the time when screenshot SC was generated in step S1204, and registers the data in supervised answer data 205 of learning model DB 200.

[0071] Analysis apparatus 100 may repeatedly execute steps S1204 and S1205 for a certain period of time, thereby obtaining multiple combinations of screenshots SC (training data TR) and the target analysis data (supervised answer data GT) at that time.

[0072] The analysis device 100 inputs the screenshot SC into the selected learning model M# and outputs the learning result LR from the selected learning model M# (step S1206).

[0073] The analysis device 100 inputs the learning result LR output in step S1206 and the correct answer data GT that is paired with the screenshot SC input in step S1206 into a loss function to calculate a loss value L (step S1207).

[0074] The analysis device 100 updates the selected learning model M# by adjusting learning parameters such as weights through backpropagation so that the loss value L is reduced (step S1208). Steps S1206 to S1208 are repeated the number of times equal to the number of screenshots SC. This completes the learning process (step S1006).

[0075] <Fig. 13 Inference process (step S1007)> Fig. 13 is a flowchart showing a detailed example of the processing procedure of the inference process (step S1007) shown in Fig. 10. The inference process (step S1007) can be executed when the learning process (step S1006) has already been executed.

[0076] The analysis device 100 selects a learning model M# to be inferred (step S1301). The learning model M# to be inferred (hereinafter, referred to as the selected learning model M# in FIG. 13 ) is all the learning models M# in the learning model DB 200 when the All Inference button 923 is pressed, or the individual learning model M# when the Individual Inference button 931 is pressed.

[0077] The analysis device 100 acquires the coordinate value P in the selected learning model M# from the position information 206 (step S1302).

[0078] The analysis apparatus 100 generates a screenshot SC of the graph G at the coordinate value P acquired in step S1302 (hereinafter referred to as the acquired coordinate value P in FIG. 13) (step S1303).

[0079] The analysis device 100 inputs the screenshot SC generated in step S1303 into the selected learning model M# and outputs the inference result IR from the selected learning model M# (step S1304). Specifically, for example, the analysis device 100 displays the inference result IR on a display or transmits it to another computer that can communicate with the analysis device 100. This completes the inference process (step S1007).

[0080] Thus, according to Example 1, even if a user does not know which analytical data is appropriate as training data TR for learning a learning model M#, the user can generate a desired learning model M# by using a screenshot SC of a graph G placed on the dashboard 300 as training data TR. Furthermore, when converting analytical data into training data, if the analytical data is represented as a graph G on the dashboard 300, the training data TR is aggregated into image data called a screenshot SC of the graph G, and therefore the analytical data can be converted into training data without being aware of the data format of the analytical data.

[0081] Example 2 will be described. Example 2 is an example in which the technology of Example 1 is applied to federated learning. In Example 2, differences from Example 1 will be mainly described, and therefore descriptions of overlapping parts with Example 1 will be omitted.

[0082] <Configuration Example of a Federated Learning System> Figure 14 is an explanatory diagram showing an example of the system configuration of a federated learning system. Federated learning system 1400 has server 1401 and clients 1402-1 to 1402-f (f is an integer greater than or equal to 1). When there is no need to distinguish between clients 1402-1 to 1402-f, they are referred to as clients 1402. Server 1401 and clients 1402 are communicatively connected via a first network 1410 such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0083] Clients 1402-1 to 1402-f are local learning devices installed at sites ST1 to STf, respectively. When there is no need to distinguish between clients 1402-1 to 1402-f, they are referred to as clients 1402. Client 1402 has one or more learning models M#. The learning model M# in client 1402 is referred to as a local learning model LM#.

[0084] When there is no need to distinguish between sites ST1 to STf (including site STx described later), they will be referred to as site ST. Sites ST1 to STf are communicably connected to sensors 1431-1 to 1431-f and terminals 1432-1 to 1432-f installed in the work area via second networks 1420-1 to 1420-f such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0085] When sensors 1431-1 to 1431-f are not distinguished from one another, they are referred to as sensors 1431. When terminals 1432-1 to 1432-f are not distinguished from one another, they are referred to as terminals 1432. When second networks 1420-1 to 1420-f are not distinguished from one another, they are referred to as second networks 1420.

[0086] The work area is an area in the site ST where specific work is performed, and various sensors 1431 such as surveillance cameras, infrared sensors, vibration sensors, and acceleration sensors are installed depending on the specific work. Data detected by these sensors 1431 is transmitted to the client 1402 via the second network 1420. Data from the terminal 1432 is also transmitted to the client 1402 via the second network 1420.

[0087] For example, sites ST1 and ST2 are construction sites, and various sensors 1431 are installed in heavy machinery work areas and human work areas. Site STf is a manufacturing site, and various sensors 1431 are installed in assembly manufacturing areas and human work areas.

[0088] Server 1401 is a federated learning device that integrates learning parameters of local learning models LM# of clients 1402-1 to 1402-f to generate global learning parameters. Server 1401 distributes the global learning parameters to clients 1402-1 to 1402-f, which then update their local learning parameters.

[0089] In Example 2, the analysis targets are a sensor 1431, a terminal 1432, and management systems (not shown) in the heavy machinery work area, the human work area, and the assembly manufacturing area within the site ST. The hardware configurations of the server 1401, the client 1402, and the terminal 1432 are the same as those shown in FIG. 1.

[0090] In the case of the federated learning system 1400 , the dashboard 300 is displayed on the server 1401 and the client 1402 .

[0091] 15 is an explanatory diagram showing an example of a global learning DB. The global learning DB 1500 is stored in the storage device 102 of the server 1401 or another device that can communicate with the server 1401.

[0092] Global learning DB 1500 is a database that stores information related to global learning parameters, and has the following fields: global ID 1501, task type 202, purpose 203, location information 206, local learning parameter group 1507, and global learning parameter 1508. Global ID 1501 is identification information that uniquely identifies global learning parameter 1508.

[0093] The local learning parameter set 1507 is a set of local learning parameters θ# set in the local learning model LM# of each client 1402. The global learning parameters 1508 are learning parameters obtained by averaging the local learning parameter set 1507. The global learning parameters 1508 are transmitted to each client 1402.

[0094] 16 is an explanatory diagram showing an example of a local learning model DB. The local learning model DB 1600 is stored in the storage device 102 of the client 1402 or the terminal 1432 or in another device that can communicate with the client 1402 or the terminal 1432.

[0095] The local learning model DB 1600 is a database that stores information about a local learning model LM# that is unique to the site ST, and has the following fields: a local ID 1601, a task type 202, a purpose 203, local training data 1604, local correct answer data 1605, and location information 206. The local ID 1601 is identification information that uniquely identifies the local learning model LM# and serves as a pointer to the local learning model LM#.

[0096] The local training data 1604 is used for learning the local learning model LM# and is data paired with the local supervised answer data 1605. The local training data 1604 is a screenshot SC input to the local learning model LM#.

[0097] When there is no need to distinguish between the values ​​LTR1, LTR2, . . . of the local training data 1604, they are simply referred to as LTR.

[0098] The local supervised answer data 1605 is used for learning the local learning model LM# and is paired with the local training data 1604. For example, if the task type 202 is identification and the objective 203 is fault diagnosis, the local supervised answer data 1605 (LGT1) is a numerical value indicating whether the analysis target is faulty or normal at the time of displaying the graph from which the paired local training data 1604 (LTR1) is captured.

[0099] If the task type 202 is regression and the objective 203 is inventory forecasting, the local ground truth data 1605 (LGT2) is the actual inventory value of the analysis target at the time of displaying the graph from which the corresponding local training data 1604 (LTR2) is captured. The local ground truth data 1605 is used to calculate the value of the loss function for the local learning model LM#.

[0100] When there is no need to distinguish between the values ​​LGT1, LGT2, . . . of the local supervised data 1605, they are simply written as LGT.

[0101] 17 is a sequence diagram showing example 1 of the setting process in the federated learning system 1400. In FIG. 17, the server 1401 executes the setting process and distributes the setting information to the client 1402.

[0102] The server 1401 displays the graph G on the dashboard 300 (step S1701). Note that since the server 1401 is not connected to the sensors 1431 and terminals 1432 of each site ST that is the subject of analysis, the graph G does not display graph data such as waveforms, but instead displays coordinate axes and explanatory text.

[0103] The server 1401 executes the setting process (step S1702). The setting process (step S1702) is the same as the setting process (step S1005). That is, the server 1401 executes the setting process (step S1005), and the global learning parameters 1508, rather than the learning model M#, are associated with the coordinate value P of the selected graph G in the position information 206 in the global learning DB 1500.

[0104] The server 1401 transmits each entry of the global learning DB 1500 set by the setting process (step S1702) as setting information to each client 1402 (step S1703).

[0105] The client 1402 receives the setting information from the server 1401 (step S1704). The client 1402 registers the received setting information in the local learning model DB 1600 (step S1705). Specifically, for example, the client 1402 registers the global ID 1501 in the setting information as the local ID 1601 in the local learning model DB 1600, and registers the task type 202 and the purpose 203 in the same entry as the local ID 1601.

[0106] After this, each client 1402 executes a local learning process (step S1706). Specifically, for example, the dashboard 300 shown in Fig. 8 is displayed on the client 1402, and the client 1402 executes steps S1001 and S1003 shown in Fig. 10 and the learning process (step S1006) shown in Fig. 12. In this case, the learning model M# to be learned in Fig. 12 becomes the local learning model LM#, and the training data 204 and the supervised data 205 become the local training data 1604 and the local supervised data 1605.

[0107] Fig. 18 is a sequence diagram showing a second example of the setting process in the federated learning system 1400. Fig. 18 shows a process in which, in response to a request from a client 1402, the server 1401 searches for the setting process and delivers the setting information to the requesting client 1402.

[0108] The client 1402 transmits setting request information to the server 1401 (step S1801). The setting request information includes the task type 202, the purpose 203, and the location information 206.

[0109] The server 1401 receives the setting request information from the client 1402 (step S1802). The server 1401 searches the global learning DB 1500 for the global ID 1501 and the location information 206 of the entry that matches the task type 202 and the purpose 203 of the received setting request information (step S1803).

[0110] The server 1401 transmits the retrieved global ID 1501 and location information 206 as setting information to the client 1402 that sent the setting request information (step S1804). If the server 1401 has a global learning parameter Θ# that has undergone federated learning, the server 1401 may transmit the learning parameter together with the setting information.

[0111] The client 1402 receives the setting information (step S1805). The client 1402 sets the global ID 1501 of the received setting information (global ID 1501 and location information 206) as the local ID 1601, and registers the location information 206 and the setting request information (objective 203 and location information 206) in the same entry as the local ID 1601 in the local learning model DB 1600 (step S1806).

[0112] If the setting information includes learning parameters, the client 1402 generates a local learning model LM# using the learning parameters.

[0113] Thereafter, the client 1402 executes the local learning process in the same manner as in step S1706 (step S1807).

[0114] 19 is a sequence diagram showing an example of federated learning processing by the federated learning system 1400. The server 1401 transmits the global ID 1501 to each client 1402 (step S1901).

[0115] Each client 1402 receives the global ID 1501 from the server 1401 (step S1902). Each client 1402 transmits to the server 1401 the local learning parameters θ# of the local learning model LM# identified by the local ID 1601 that matches the global ID 1501 (step S1903).

[0116] The server 1401 receives a set of local learning parameters θ#s, which are local learning parameters θ#, from each client 1402 (step S1904).

[0117] The server 1401 updates the local learning parameter set 1507 in the global learning DB 1500 with the local learning parameter set θ#s (step S1905).

[0118] The server 1401 integrates the local learning parameter set θ#s by averaging them to generate a global learning parameter Θ#, and registers it in the global learning parameter 1508 (step S1906).

[0119] The server 1401 transmits the global learning parameter Θ# to each client 1402 (step S1907).

[0120] Each client 1402 receives the global learning parameters Θ# (step S1908), and updates the local learning model LM# with the global learning parameters Θ# (step S1909).

[0121] Thereafter, each client 1402 executes the inference process (step S1007) shown in FIG. 13 using the local learning model LM# to output the inference result IR. Each client 1402 also executes the local learning process (steps S1706 and S1807) to update the local learning parameters θ#. The latest local learning parameters θ# are transmitted to the server 1401 in step S1903.

[0122] In this way, according to the second embodiment, the setting process (step S1005), the learning process (step S1006), and the inference process (step S1007) shown in the first embodiment can also be applied to federated learning, thereby preventing leakage of the local training data 1604 and the local supervised answer data 1605 from the client 1402.

[0123] As explained above, according to the above-mentioned first and second examples, the analytical data of the analysis target is broken down for each visualized graph, so that even if the analysis target provides the same function, the analytical data is expressed differently on the graph due to differences in the user or development vendor, and this difference can be absorbed.

[0124] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0125] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0126] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, SD card, or DVD (Digital Versatile Disc).

[0127] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected.

Claims

1. An analytical device having a processor that executes a program and a storage device that stores the program, wherein the processor executes the following: a first generation process that generates first image data of a graph related to an analysis target from a placement area on a display screen in which the graph is placed, as training data to be input into a learning model related to the analysis target; an acquisition process that, when the first image data is generated by the first generation process, acquires target data related to the analysis target corresponding to the graph as correct answer data to be used for learning the learning model; and a learning process that learns the learning model based on a learning result output as a result of inputting the first image data into the learning model and the target data acquired by the acquisition process.

2. An analysis device as described in claim 1, wherein a plurality of the graphs are arranged in the placement area, and the placement area includes a display area in which a first graph of the plurality of graphs is displayed, and a non-display area in which a second graph of the plurality of graphs, different from the first graph, is not displayed.

3. An analysis device as described in claim 1, characterized in that the graph is created based on at least one type of analysis data among data input to the subject to analysis, data detected by the subject to analysis, data detecting the behavior of the subject to analysis, data generated by internal processing of the subject to analysis, and data output from the subject to analysis.

4. An analysis device as described in claim 1, characterized in that, in the first generation process, the processor generates the first image data for each graph when multiple graphs are placed in the placement area; in the acquisition process, the processor acquires the target data for a combination of the first image data of each of two or more graphs; and in the learning process, the processor learns the learning model based on a learning result output as a result of inputting the first image data of each of the two or more graphs into the learning model and the target data.

5. An analysis device as described in claim 1, wherein the processor executes: a second generation process that generates second image data of the graph; and an inference process that outputs an inference result by inputting the second image data generated by the second generation process into the learning model.

6. An analysis device as described in claim 4, wherein the processor executes: a second generation process that generates second image data for each of the two or more graphs; and an inference process that outputs an inference result by inputting a combination of two or more second image data generated by the second generation process into the learning model.

7. An analysis method using an analysis device having a processor that executes a program and a storage device that stores the program, wherein the processor executes the following: a first generation process that generates first image data of a graph related to an analysis target from a placement area on a display screen in which the graph is placed, as training data to be input into a learning model related to the analysis target; an acquisition process that, when the first image data is generated by the first generation process, acquires target data related to the analysis target corresponding to the graph as correct answer data to be used for learning the learning model; and a learning process that learns the learning model based on a learning result output as a result of inputting the first image data into the learning model and the target data acquired by the acquisition process.

8. An analysis program that causes a processor to execute the following: a first generation process that generates first image data of a graph related to an analysis target from a placement area on a display screen in which the graph is placed, as training data to be input into a learning model related to the analysis target; an acquisition process that, when the first image data is generated by the first generation process, acquires target data related to the analysis target corresponding to the graph as correct answer data to be used for learning the learning model; and a learning process that learns the learning model based on a learning result output as a result of inputting the first image data into the learning model and the target data acquired by the acquisition process.

Images