Information processing device, learning method, and learning program

Abstract

To reduce the cost of generating an inference model of equipment status through machine learning.SOLUTION: An information processing device (1) includes: a first learning unit (103) configured to generate a first inference model (112) through machine learning using first training data in which time-series numerical data, as correct answer data, is associated with first time-series data reflecting equipment status; and a second learning unit (104) configured to generate a second inference model (114) that estimates the equipment status by fine-tuning the first inference model (112) using second training data in which second data reflecting the equipment status is associated with the equipment status as correct answer data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device or the like that generates a trained model through machine learning. [Background technology]

[0002] In recent years, the practical application of trained models generated by machine learning has been progressing in a wide range of fields. For example, Patent Document 1 below discloses an information processing device that uses a trained model to evaluate the combustion state inside an incinerator. This machine-learned model was generated by machine learning training data to which classification labels serving as factors for determining the combustion state are assigned to infrared imaging data inside the incinerator. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-001337 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, when generating the training data, an operator determines which classification label to associate with the infrared imaging data based on experience. Therefore, in order to generate a trained model with sufficient accuracy for practical use, experienced operators are required to spend a long time associating classification labels. Thus, the technology of Patent Document 1 has a problem in that the work costs for generating training data for generating a classification model are high. Furthermore, this problem is not limited to models for classifying combustion states as in Patent Document 1, but is a common problem that arises when an inference model of the state of a part or the entirety of any equipment is generated by machine learning.

[0005] One aspect of the present invention aims to realize an information processing device or the like that can reduce costs when generating an inference model of the state of part or the entire facility using machine learning. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to one embodiment of the present invention comprises: a first learning unit that generates a first inference model that infers the value of numerical data by machine learning using first training data in which each piece of time-series first data reflecting the state of part or the entirety of a specified facility is associated with each piece of time-series numerical data related to the facility as correct answer data; and a second learning unit that generates a second inference model that estimates the state of part or the entirety of the facility by fine-tuning the first inference model using second training data in which each piece of time-series first data reflecting the state of part or the entirety of the facility is associated with the state as correct answer data.

[0007] In addition, in order to solve the above-mentioned problem, another information processing device according to one embodiment of the present invention includes a first learning unit that generates a first inference model by machine learning using first data that reflects the state of part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data that reconstructs the first data based on the extracted features; and a second learning unit that generates a second inference model that estimates the state of part or the entirety of the facility by fine-tuning the first inference model using second training data that corresponds to second data that reflects the state of part or the entirety of the facility as correct answer data.

[0008] In addition, in order to solve the above-mentioned problems, a learning method according to one embodiment of the present invention is a learning method executed by one or more information processing devices, and includes: a first learning step of generating a first inference model that infers the value of numerical data by machine learning using first training data in which each piece of time-series first data reflecting the state of part or the entirety of a specified facility is associated with each piece of time-series numerical data related to the state as correct answer data; and a second learning step of generating a second inference model that estimates the state of part or the entirety of the facility by fine-tuning the first inference model using second training data in which each piece of time-series first data reflecting the state of part or the entirety of the facility is associated with the state as correct answer data.

[0009] In addition, in order to solve the above-mentioned problem, another learning method according to one embodiment of the present invention is a learning method executed by one or more information processing devices, and includes: a first learning step of generating a first inference model by machine learning using first data reflecting the state of part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data by reconstructing the first data based on the extracted features; and a second learning step of fine-tuning the first inference model using second training data that corresponds to second data reflecting the state of part or the entirety of the facility as correct answer data, thereby generating a second inference model that estimates the state of part or the entirety of the facility. [Effects of the Invention]

[0010] According to one aspect of the present invention, it is possible to reduce the cost of generating an inference model of the state of part or the entire facility using machine learning. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a main configuration of an information processing device according to an embodiment of the present invention; [Figure 2]1 is a diagram showing an overview of a control system according to an embodiment of the present invention and a configuration of a waste treatment facility to be controlled. [Figure 3] A figure showing an example of a first training dataset for generating a first inference model. [Figure 4] A figure showing an example of a second training dataset for generating a second inference model. [Figure 5] A figure showing an example of an inference model generated by the above-mentioned information processing device. [Figure 6] 1 is a flowchart showing the flow of a learning method according to an embodiment of the present invention. [Figure 7] A flowchart showing the flow of a learning method when the first inference model is a generative model. DETAILED DESCRIPTION OF THE INVENTION

[0012] [Configuration of information processing device 1] An information processing device 1 according to one embodiment of the present invention is a device equipped with a function for generating an inference model for estimating the state of a portion or the entirety of a specified facility. The facility whose state is to be estimated and the state estimated by the information processing device 1 are both arbitrary and not particularly limited. For example, when targeting a facility equipped with one or more devices, the information processing device 1 may estimate the state of a portion or all of the devices equipped in the facility. Furthermore, the "state of a portion or the entirety of a specified facility" here includes not only the state of the facility itself, but also the state of the object being processed or manufactured by the facility and the state of a phenomenon (e.g., a chemical reaction) occurring in the facility. The size of the facility is not particularly limited and may be large-scale with multiple devices or small-scale with one or a small number of devices. The facility may also be referred to as a facility, machinery, plant, etc.

[0013] The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the main parts of the information processing device 1. As shown in the figure, the information processing device 1 includes a control unit 10 that controls all parts of the information processing device 1, and a storage unit 11 that stores various data used by the information processing device 1. The information processing device 1 also includes a communication unit 12 that enables the information processing device 1 to communicate with other devices, an input unit 13 that accepts input to the information processing device 1, and an output unit 14 that enables the information processing device 1 to output various data.

[0014] The control unit 10 also includes a data acquisition unit 101, a training data generation unit 102, a first learning unit 103, a second learning unit 104, an inference unit 105, and an equipment control unit 106. The memory unit 11 stores a first training data set 111, a first inference model 112, a second training data set 113, and a second inference model 114.

[0015] The data acquisition unit 101 acquires various data necessary to generate the first training data set 111. Specifically, the data acquisition unit 101 acquires first time-series data reflecting the state of a part or the whole of a predetermined facility, and time-series numerical data relating to the state of a part or the whole of the predetermined facility.

[0016] The first training data set 111 is a data set including a plurality of first training data. The first training data is data in which the above-mentioned numerical data is associated with the above-mentioned first data as correct answer data. The first training data set 111 is used to generate a first inference model 112.

[0017] The first inference model 112 is a machine-learned model that infers the value of the numerical data from the first data. In other words, the first inference model 112 is a trained model that uses the first data as an explanatory variable and the numerical data as a target variable.

[0018] The first data may be time-series data that reflects the state of a part or the whole of a predetermined facility, and is preferably data that requires low costs (either time cost or financial cost, or both) for generation or acquisition. The type of data to be used as the first data may be determined depending on the facility in question, the state to be estimated, etc.

[0019] For example, when the condition of a part or the whole of a facility is reflected in its appearance, an image obtained by photographing a part or the whole of the facility may be used as the first data. The image may be an RGB image photographed by a general camera, or an image photographed using light of a specific wavelength (for example, an infrared image photographed by an infrared camera, an ultraviolet image photographed by an ultraviolet camera, etc.). In addition, for example, an X-ray image photographed using X-rays, etc. may also be used as the first data.

[0020] Furthermore, for example, when the state of a part or the whole of a facility is expressed in its shape, point cloud data (a set of points indicating positions on the surface of the facility) showing the shape of the part or the whole of the facility may be used as the first data. Such point cloud data can be generated, for example, by LiDAR (Light Detection and Ranging).

[0021] The above-mentioned numerical data may be time-series data relating to the state of a part or the whole of the equipment. However, if numerical data with low relevance to the state is used, it is expected that the time required for fine tuning (described later) will increase, or the amount of second training data required for fine tuning will increase. For this reason, it is preferable to apply numerical data with high relevance to the state of a part or the whole of the equipment.

[0022] Here, the state of equipment is often reflected in measured values ​​measured by the equipment, set values ​​set for devices operating in the equipment, and manipulated variables for the devices. The state of equipment may also be reflected in calculated values ​​calculated using at least one of the measured values, set values, and manipulated variables. Therefore, the numerical data may be data indicating at least one of the measured values ​​measured by the equipment, set values ​​set for devices operating in the equipment, manipulated variables for the devices, and calculated values ​​calculated using at least one of the measured values, set values, and manipulated variables.

[0023] This makes it possible to generate a first inference model 112 that has accurately learned features related to the state of the equipment, and to generate a second inference model 114 with high inference accuracy by fine-tuning the first inference model 112. Furthermore, since the above-mentioned numerical data can be collected automatically at the equipment, it is also possible to reduce the cost required to procure the first training data.

[0024] Furthermore, the data acquisition unit 101 acquires a plurality of pieces of second training data used to generate a second inference model 114. The plurality of pieces of second training data acquired by the data acquisition unit 101 are stored in the storage unit 11 as a second training data set 113.

[0025] As described above, the second training data set 113 is a data set including a plurality of second training data. The second training data is data in which the second data reflects the state of a part or the whole of a predetermined facility and the state is associated with the second data as correct answer data.

[0026] Like the first data, the second data reflects the state of a part or the whole of a specified facility, and therefore may be the same type of data as the first data. For example, if the first data is an image of a part or the whole of a facility, the second data may also be an image of a part or the whole of the facility. Since the image of the facility reflects its state, the image is suitable as the first data and the second data. By using the image as the first data and the second data, a second inference model 114 is generated that estimates the state of the facility (which may be a part or the whole) shown in the image.

[0027] The association of the second data with the correct answer data may be performed based on a manual state determination. For example, if the second data is an image of a predetermined facility, a person may look at the image and determine the state of the facility (which may be a part or the whole) shown in the image, and correct answer data indicating the result of the determination may be associated with the image to be used as the second training data.

[0028] Although the above-described work requires human and time costs, the cost of the information processing device 1 generating the second inference model 114 can be kept lower than when the second inference model 114 is generated from scratch using only the second training data, as in the conventional method. This is because the second training data is used to fine-tune the first inference model 112 that has already learned about the features of the first data, and only a small amount of second training data is required to generate the second inference model 114.

[0029] The training data generation unit 102 generates first training data by associating the first data acquired by the data acquisition unit 101 with the numerical data. More specifically, the training data generation unit 102 generates first training data by associating each piece of first data constituting a time series with the numerical data at the corresponding time. The method for generating the first training data will be described again with reference to FIG. 3.

[0030] The first learning unit 103 generates a first inference model 112 that infers the value of the above-mentioned numerical data through machine learning using the first training data included in the first training data set 111. The learning by the first learning unit 103 is pre-learning for generating the second inference model 114.

[0031] The algorithm applied to the machine learning by the first learning unit 103 may be any algorithm capable of learning the features of the first data and generating the first inference model 112 that outputs a predicted value of the numerical data. Therefore, the algorithm may be appropriately determined depending on the type of data to be used as the first data. The type of the first inference model 112 is not particularly limited. For example, the first inference model 112 may be a convolutional neural network model such as ResNET (Residual Neural Networks), a Vision Transformer (ViT) described later, or another type of model.

[0032] The second learning unit 104 fine-tunes the first inference model 112 using second training data included in the second training dataset 113, thereby generating a second inference model 114 that estimates the state of part or the entire facility. Fine-tuning refers to changing the output of a target model to a desired content and performing machine learning on the changed model using training data, thereby generating a model that obtains a desired output while utilizing the learning results of the model before the change. In fine-tuning the first inference model 112, the second learning unit 104 changes the output layer of the first inference model 112 so that the output of the first inference model 112 is a value indicating the state of part or the entire facility, rather than the above-mentioned numerical data. The second learning unit 104 then fine-tunes the changed model using the second training data, thereby generating a second inference model 114 that outputs an estimation result of the state of part or the entire facility. In fine tuning, only the parameters of the parts that have been changed in the first inference model 112 may be updated, or parameters other than the parts that have been changed may also be updated.

[0033] The inference unit 105 infers the state of a part or the whole of a specified facility. For this inference, the second inference model 114 is used. In other words, the inference unit 105 infers the state of a part or the whole of a specified facility from the output value obtained by inputting the second data into the second inference model 114.

[0034] The device control unit 106 controls devices operating in a predetermined facility based on the inference result of the inference unit 105. It is sufficient to determine in advance what control to perform when what inference result is obtained. Furthermore, the device control unit 106 may directly control the target device, or may indirectly control it via the control device 2 described below.

[0035] As described above, the information processing device 1 includes a first learning unit 103 that generates a first inference model 112 that infers the value of the numerical data by machine learning using first training data in which each piece of time-series first data reflecting the state of part or the entirety of a specified facility is associated with each piece of time-series numerical data related to the facility as correct answer data, and a second learning unit 104 that generates a second inference model 114 that estimates the state of part or the entirety of the facility by fine-tuning the first inference model 112 using second training data in which each piece of time-series first data reflecting the state of part or the entirety of the facility is associated with the state as correct answer data.

[0036] According to the above configuration, machine learning is performed using first training data in which each piece of time-series first data is associated with each piece of time-series numerical data as a correct label, thereby generating a first inference model 112 that has learned the characteristics of the first data. Because the data associated as the correct label in the first training data is numerical data, the first training data can be generated without the costly task of manually determining a state. Furthermore, because both the first data and the numerical data are time-series data, the association between them can be easily automated, which also means that the cost of generating the first training data is low.

[0037] According to the above configuration, the second inference model 114 is generated by fine-tuning the first inference model 112 using second training data in which the state of a part or the entire specified equipment is associated with the second data as a correct label.

[0038] Here, since the data associated as the correct label in the second training data indicates the state of part or the entire equipment, generating the second training data may require manual state determination. However, since the second training data is used for fine-tuning the first inference model 112 that has already learned the features of the first data, the amount of second training data required to generate the second inference model 114 can be kept small compared to generating the second inference model 114 from scratch.

[0039] Therefore, according to the information processing device 1, it is possible to reduce the cost of generating the second inference model 114 that estimates the state of a part or the entirety of a specified facility by machine learning.

[0040] [Control system 3 including information processing device 1] The information processing device 1 is capable of estimating the state of equipment, and therefore can be used to control the equipment. Here, a control system 3 including the information processing device 1 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an overview of the control system 3. Fig. 2 also shows the configuration of a portion of a waste treatment facility A that is the control target of the control system 3. The waste treatment facility A is a facility that incinerates combustible waste, such as household garbage.

[0041] The following describes an example in which the information processing device 1 estimates the combustion state of waste in waste treatment facility A and controls the equipment installed in waste treatment facility A based on the estimation result. Note that the information processing device 1 is capable of estimating the state of part or the whole of any facility, and the target facility is not limited to waste treatment facility A, and the estimated state is not limited to the combustion state of waste. In other words, in the following description, "waste treatment facility A" can be read as any "specified facility," and "combustion state" can be read as any state of part or the whole of any facility.

[0042] As shown in Fig. 2, the control system 3 includes an information processing device 1 and a control device 2. The control device 2 is a device that controls the devices provided in the waste treatment facility A in accordance with instructions from the information processing device 1. There may be multiple devices to be controlled, in which case one control device 2 may control multiple devices, or one control device 2 may control one device. Furthermore, as described above, the information processing device 1 may directly control the devices.

[0043] Figure 2 shows a schematic cross section of the incinerator portion of waste treatment facility A. As shown in the figure, waste treatment facility A is equipped with a hopper A1 for feeding waste into the incinerator, a combustion chamber A2 which is a space for incinerating the waste, and a fire grate A3 for transporting the waste within the combustion chamber A2. The fire grate A3 is divided into a drying stage A31 for drying the waste, a combustion stage A32 for burning the waste, and a post-combustion stage A33 for completely burning the waste that remains unburned in the combustion stage A32.

[0044] When estimating the combustion state of waste in an incinerator, for example, images taken inside the combustion chamber A2 in the direction indicated by the white arrow in the figure (direction from downstream to upstream of the grate A3) may be used as the first and second data. Images taken in this way reflect the combustion state of waste throughout the grate A3, so using such images makes it possible to accurately estimate the combustion state.

[0045] Furthermore, when estimating the combustion state, the numerical data associated with the first data may be time-series data relating to the combustion state. For example, the numerical data may be measurement values ​​(PV: Process Variable values) measured by various sensors installed in the waste treatment facility A, setting values ​​(SV: Set Variable values) set in various devices operating in the waste treatment facility A, operation amounts (Manipulative Variable values) of the devices, and calculated values ​​calculated using at least any of the measurement values, setting values, and operation amounts. Note that measurement values ​​measured by sensors may also be called sensor data.

[0046] Here, as a result of experiments conducted by the inventors of the present application, it was found that it is preferable to use data indicating any one of the temperature of the post-combustion stage A33, the flow rate of air sent into the incinerator, and the thickness of the waste on the grate A3 as the numerical data. Therefore, it is preferable to use data indicating any one of the temperature of the post-combustion stage A33, the flow rate of air sent into the incinerator, and the thickness of the waste on the grate A3 as the numerical data. This makes it possible to generate a first inference model 112 that accurately learns characteristics related to the combustion state of the waste.

[0047] The temperature of the post-combustion stage A33 is, more precisely, the temperature in the vicinity of the post-combustion stage A33, and can be measured by installing a temperature sensor above the post-combustion stage A33. The flow rate of air sent into the incinerator can be determined from the flow rate setting of a combustion air supply device (not shown). The thickness of the waste (pile thickness) on the fire grate A3 can be determined from the measurement value of a pressure sensor installed on the fire grate A3 (for example, the drying stage A31 or the combustion stage A32).

[0048] The second inference model 114 that estimates the combustion state of waste may be, for example, a classification model that classifies the combustion state of waste. The combustion state may be classified into two classes, for example, normal and abnormal. Abnormalities may also be classified by type. Examples of abnormal combustion states include an abnormality where the amount of waste to be burned is insufficient, or an abnormality where clumps of unburned waste are found on the combustion stage A32 or post-combustion stage A33. In addition, unknown states where it is difficult to determine the combustion state due to smoke, noise, etc. may also be added to the classification.

[0049] If the second inference model 114 is a classification model that classifies the combustion state of waste, no control is required if the classification result is normal. If the classification result is abnormal, control to normalize the combustion state is sufficient. Furthermore, by classifying the type of abnormality, more accurate control can be achieved. For example, if the classification result by the inference unit 105 indicates that the amount of waste is insufficient, the equipment control unit 106 may cause the control device 2 to increase the speed at which waste is sent from the hopper A1 to the grate A3. Furthermore, if the classification result by the inference unit 105 indicates that clumps of unburned waste are present on the combustion stage A32 or the post-combustion stage A33, the equipment control unit 106 may cause the control device 2 to increase the amount of combustion air supplied or slow down the speed at which waste is transported through the grate A3.

[0050] [Example of generating the first training data set 111] An example of generating the first training data set 111 when the target of state estimation by the second inference model 114 is waste treatment facility A will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of generating the first training data set 111.

[0051] Graph G in Figure 3 shows the change over time in measured values ​​taken by a sensor installed in waste treatment facility A. The measured values ​​shown in graph G are time-series numerical data related to the combustion state, and can therefore be used to generate first training data. Also shown below graph G are images of the inside of the incinerator taken from time t = 1 to t = T. These images are the first time-series data used to generate the first training data.

[0052] The training data generation unit 102 generates first training data by associating each of the images (first data) constituting the time series with numerical data at the corresponding time. For example, as shown in the figure, the training data generation unit 102 may generate first training data by associating an image taken at time t=1 with a measurement value measured at time t=1. In this case, the training data generation unit 102 generates first training data in the same manner for times t=2 and onward. This generates a first training data set 111 consisting of first training data in which images taken or measured at the same time are associated with numerical data.

[0053] Note that there may be a time lag between the time-series first data and the time-series numerical data associated in the first training data. This is because there may be a time lag between the timing at which the combustion state is reflected in the first data and the timing at which the combustion state is reflected in the numerical data. With the above configuration, it is possible to cancel such a lag within the first training data and generate a first inference model 112 that accurately learns features related to the combustion state. Then, by fine-tuning this first inference model 112, it is possible to generate a second inference model 114 with high inference accuracy.

[0054] For example, suppose the time from when the combustion state is reflected in the first data until when it is reflected in the numerical data is Δt. In this case, the training data generation unit 102 associates the first data at time t with the numerical data at time (t+Δt) to generate the first training data. In this case, the training data generation unit 102 similarly associates the numerical data delayed by Δt with the first data for t=2 and onward. This generates a first training data set 111 consisting of first training data in which there is a Δt difference between the associated first data and numerical data.

[0055] Although it depends on the images and numerical data used, the accuracy required for inference, etc., it is preferable that the first training data set 111 includes at least several thousand pieces of first training data. Although this number is not small, time-series images can be automatically extracted from video images taken inside an incinerator, for example, and numerical data can also be extracted from continuously measured values. Furthermore, the process of generating the first training data from these data is automatically performed by the training data generation unit 102, so the cost of generating the first training data set 111 can be reduced.

[0056] [Example of the second training data set 113] An example of the second training data set 113 when the target of state estimation by the second inference model 114 is waste treatment facility A will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the second training data set 113.

[0057] The second training data set 113 shown in FIG. 4 is made up of second training data in which images taken inside an incinerator when the incinerator is in a predetermined combustion state are associated with ground truth data indicating the combustion state.

[0058] Specifically, the second training data set 113 shown in Figure 4 includes a plurality of second training data in which images taken inside the incinerator when the combustion state is normal are associated with correct answer data indicating that the combustion state is normal. The second training data set 113 shown in Figure 4 also includes a plurality of second training data in which images taken inside the incinerator when there is a shortage of waste on the grate A3 are associated with correct answer data indicating that the amount of waste is insufficient. The second training data set 113 shown in Figure 4 also includes a plurality of second training data in which images taken inside the incinerator when the combustion state is unknown are associated with correct answer data indicating that the combustion state is unknown.

[0059] A second inference model 114 that classifies combustion states is generated by fine-tuning the first inference model 112 using a second training data set 113 shown in FIG. 4. When an image of the inside of an incinerator is input to this second inference model 114, a classification result of the combustion state shown in the image is output. For example, when the three types of second training data shown in FIG. 4 are used, the second inference model 114 outputs a numerical value indicating the likelihood that each state corresponds to normal, insufficient waste amount, and unknown. The inference unit 105 may, for example, estimate the state corresponding to the largest numerical value as the combustion state corresponding to the input image.

[0060] The association of the second data with the correct data may be performed by a person viewing the image and determining the combustion state. This task requires human resources and time. However, experiments by the inventors of the present application have shown that sufficient fine-tuning can be performed with just a few pieces of second training data per classification class. It has also been confirmed that a second inference model 114 with sufficient classification accuracy can be generated using only two pieces of second training data per classification class. The cost of preparing such a small amount of second training data is significantly less than the cost of generating a classification model from scratch (which generally requires several thousand or more pieces of labeled training data).

[0061] [Example of the first inference model 112 and the second inference model 114] 5 is a diagram showing examples of a first inference model 112 and a second inference model 114. The first inference model 112 and the second inference model 114 shown in FIG. 5 are models of ViT. In ViT, as shown in the figure, an input image is divided into patch images, each patch image is converted into a vector and linearly projected, and information indicating the position of each patch image is embedded in the linearly projected data and input to an encoder. The output from the encoder is then input to an MLP (Multi-Layer Perceptron) head, and a predicted value (estimated value) of the numerical data is output from the MLP head.

[0062] The first learning unit 103 can generate such a first inference model 112 through machine learning using a first training data set 111. In this machine learning, each parameter in the first inference model 112 is learned so as to reduce the error between a predicted value obtained by inputting first data (images) included in the first training data into the first inference model 112 and the numerical data included in the first training data (i.e., ground truth data). More specifically, the first learning unit 103 updates each parameter so as to minimize a loss function including an error term and a regularization term. Note that the error may be a squared error (e.g., mean squared error), an absolute error (e.g., L1 norm or mean absolute error), or a combination thereof. It is not essential to include a regularization term in the loss function.

[0063] The second learning unit 104 can then fine-tune the first inference model 112 using the second training dataset 113 to generate a second inference model 114 as shown in the figure.

[0064] 5 differs from the first inference model 112 in that an MLP head outputs a combustion state classification result. That is, in fine tuning, the second learning unit 104 changes the MLP head in the first inference model 112 to an MLP head that outputs a combustion state classification result, and performs machine learning on the changed model using a second training data set 113. In this machine learning, each parameter in the second inference model 114 is learned so as to reduce the error between the combustion state classification result obtained by inputting second data (images) included in the second training data into the second inference model 114 and the combustion state indicated in the second training data (i.e., correct data).

[0065] 5, one image is input to the first inference model 112 and a predicted value of the numerical data is output, but a group of images consisting of multiple images forming a time series may also be input as a single piece of first data to the first inference model 112. In this case, the data input to the second inference model 114 will also be a group of images consisting of multiple images forming a time series.

[0066] As a result, features that change over time are learned when the first inference model 112 is generated (i.e., during pre-learning). Then, by using the second inference model 114 generated by fine-tuning the generated first inference model 112, inference can be performed that takes into account features that change over time.

[0067] If the first inference model 112 and the second inference model 114 are ViT, each of the images constituting the time series can be divided to generate patch images, and these patch images can be converted into vectors and linearly projected. If the first inference model 112 and the second inference model 114 are models other than ViT, the multiple images constituting the time series can be converted, if necessary, into a format acceptable to the model before being input.

[0068] Note that the first inference model 112 and the second inference model 114 shown in FIG. 5 are merely examples, and the configurations of the first inference model 112 and the second inference model 114 applicable to the present invention are not limited to these examples. Furthermore, the data format and data processing content described in this section are also merely examples, and can be modified as appropriate within the scope of generating an inference model that can obtain the desired inference results. For example, each patch image can be input to the encoder as a two-dimensional array (i.e., an image). It is also possible to input a group of images consisting of multiple images that make up a time series to the first inference model 112 as three-dimensional data.

[0069] [Processing flow] The flow of processing executed by the information processing device 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of processing executed by the information processing device 1. The flowchart in Fig. 6 includes each step of the learning processing according to this embodiment.

[0070] In S1, the data acquisition unit 101 acquires first time-series data reflecting the state of a part or the whole of a predetermined facility and time-series numerical data related to the state. As described above, the first data may be, for example, an image of a part or the whole of the predetermined facility. Furthermore, the numerical data may be, for example, measured values ​​or setting values ​​of the facility. The method of acquiring these data is not particularly limited. For example, the data acquisition unit 101 may acquire these data input via the input unit 13, or may acquire these data from another device by communication via the communication unit 12.

[0071] In S2, the training data generation unit 102 generates first training data by associating the first data acquired in S1 with the numerical data also acquired in S1. In S2, a plurality of pieces of first training data are generated, and the generated plurality of pieces of first training data are stored in the storage unit 11 as a first training data set 111.

[0072] In S3 (first learning step), the first learning unit 103 generates a first inference model 112 through machine learning using the first training data generated in S2. The generated first inference model 112 is stored in the memory unit 11. The processing of S3 can also be considered a step of performing pre-learning for generating a second inference model 114.

[0073] In S4 (second learning step), the second learning unit 104 fine-tunes the first inference model 112 generated in S3 using the second training data included in the second training data set 113. This generates a second inference model 114 that estimates the state of part or the entire facility. The generated second inference model 114 is stored in the memory unit 11, and the processing of FIG. 6 ends.

[0074] As described above, the learning method according to this embodiment includes a first learning step (S3) of generating a first inference model 112 that infers the value of a certain piece of equipment's state by machine learning using first training data in which time-series first data reflecting the state of the entire equipment is associated with time-series numerical data relating to the state as ground truth data, and a second learning step (S4) of generating a second inference model 114 that estimates the state of the entire equipment by fine-tuning the first inference model 112 using second training data in which second data reflecting the state of the entire equipment is associated with the state as ground truth data. This makes it possible to reduce the cost of generating the second inference model 114 that estimates the state of the equipment by machine learning.

[0075] [Other examples of pre-learning] In the above-described embodiment, an example has been described in which a first inference model 112 that infers the value of numerical data from first data is generated in pre-learning by the first learning unit 103. Below, an example will be described in which, instead of the first inference model 112, a first inference model (hereinafter referred to as a generative model) is generated in pre-learning, which extracts features from the first data and generates reconstructed data by reconstructing the first data based on the extracted features.

[0076] A generative model can be generated by unsupervised learning. Therefore, when generating a generative model in pre-learning, the training data generation unit 102 is not necessary, and the data acquisition unit 101 simply acquires first data that is not associated with ground truth data. The first learning unit 103 then extracts features from the first data acquired by the data acquisition unit 101, generates reconstructed data by reconstructing the first data based on the extracted features, and updates the parameters of the generative model so as to reduce the error between the first data and the reconstructed data. This process is repeated while changing the target first data, thereby generating a generative model. The generative model may be, for example, an autoencoder (a model that combines an encoder that extracts features of input data and a decoder that generates reconstructed data from the features extracted by the encoder).

[0077] The processing after the generative model is generated is the same as in the above embodiment. That is, the generated generative model is fine-tuned to generate the second inference model 114.

[0078] In this way, the information processing device 1 may be configured to include a first learning unit 103 that generates a first inference model (generative model) by machine learning using first data that reflects the state of part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data that reconstructs the first data based on the extracted features, and a second learning unit 104 that generates a second inference model 114 that estimates the state of part or the entirety of the facility by fine-tuning the first inference model (generative model) using second training data that corresponds to second data that reflects the state of part or the entirety of the facility as correct answer data.

[0079] Even with this configuration, it is possible to reduce the cost of generating the second inference model 114 that estimates the state of the equipment through machine learning. In particular, since it is not necessary to use training data associated with ground truth data to generate a generative model, costs can be further reduced compared to generating the first inference model 112 in pre-learning. However, experiments conducted by the inventors of the present application have shown that generating the first inference model 112 that predicts numerical data such as sensor data in pre-learning tends to result in higher learning stability than generating a generative model. Therefore, if stability is important, it is preferable to generate the first inference model 112 in pre-learning.

[0080] In the pre-learning, it is sufficient to learn the characteristics of the first data. Therefore, in the pre-learning, a first inference model that both generates reconstructed data and predicts numerical data may be generated. In addition, in the pre-learning, a first inference model that predicts each of multiple types of numerical data may be generated.

[0081] [Processing flow] The flow of processing executed by the information processing device 1 when generating a generative model in pre-learning will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of processing executed by the information processing device 1 when generating a generative model in pre-learning.

[0082] In S11, the data acquisition unit 101 acquires a plurality of first data reflecting the state of a part or the entirety of a predetermined facility. These first data may or may not be configured in a time series. In addition, the method of acquiring these data is not particularly limited, as in S1 of FIG. 6.

[0083] In S12 (first learning step), the first learning unit 103 performs machine learning using the first data acquired in S11 to extract features from the first data and generate a first inference model (generative model) that generates reconstructed data by reconstructing the first data based on the extracted features. The generated first inference model (generative model) is stored in the memory unit 11. The processing of S12 can also be considered a step of performing pre-learning for generating the second inference model 114.

[0084] In S13 (second learning step), the second learning unit 104 fine-tunes the first inference model (generative model) generated in S12 using the second training data included in the second training data set 113. This generates a second inference model 114 that estimates the state of a part or the entire specified facility. The generated second inference model 114 is stored in the memory unit 11, and the processing of FIG. 7 ends.

[0085] As described above, another learning method according to this embodiment includes a first learning step (S12) of generating a first inference model (generative model) by machine learning using first data reflecting the state of a part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data by reconstructing the first data based on the extracted features, and a second learning step (S13) of fine-tuning the first inference model (generative model) using second training data that corresponds to second data reflecting the state of a part or the entirety of the facility as ground truth data, thereby generating a second inference model 114 that estimates the state of a part or the entirety of the facility. This makes it possible to reduce the cost of generating the second inference model 114 that estimates the state of the facility by machine learning.

[0086] [Examples of application to other facilities] The information processing device 1 can generate an inference model (second inference model 114) of the state of a part or the entirety of various pieces of equipment. For example, the information processing device 1 can generate an inference model that estimates the state (e.g., quality) of a product manufactured (or to be manufactured) in a manufacturing facility from an image of the manufacturing equipment operating in the facility. In this case, the numerical data included in the first training data used in pre-learning may be various measured values ​​related to the state of the product (e.g., temperature, pressure, etc. measured during manufacturing).

[0087] The information processing device 1 can also generate an inference model that estimates the current or future state of a wind power generation facility (e.g., the presence or absence of an abnormality) from images taken of the inside or outside of the wind power generation facility. In this case, the numerical data included in the first training data used for pre-learning may be various measurement values ​​related to the state of the facility (e.g., vibration data measured by a vibration sensor provided in the wind power generation facility).

[0088] [Modification] The execution entity of each process described in the above-described embodiment may be any entity and is not limited to the above-described example. In other words, functions similar to those of the information processing device 1 can be realized by a plurality of information processing devices (which may also be called processors) that can communicate with each other. For example, the execution of each process shown in FIG. 6 or FIG. 7 may be shared among a plurality of information processing devices. In other words, the execution entity of the learning method according to the above-described embodiment may be one information processing device (for example, the information processing device 1) or a plurality of information processing devices.

[0089] [Software implementation example] The functions of the information processing device 1 are programs for causing a computer to function as the device, and can be realized by a program (learning program) for causing a computer to function as each control block of the information processing device 1 (particularly each part included in the control unit 10).

[0090] In this case, the information processing device 1 includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing each function described in the above embodiment.

[0091] The program may be stored non-transitory on one or more computer-readable storage media. The storage media may or may not be included in the information processing device 1. In the latter case, the program may be supplied to the information processing device 1 via any wired or wireless transmission medium.

[0092] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0093] 〔summary〕 An information processing device according to aspect 1 of the present invention comprises a first learning unit that generates a first inference model that infers the value of numerical data by machine learning using first training data in which time-series numerical data relating to a state of a part or the entirety of a specified facility is associated with each piece of time-series first data reflecting the state of the part or the entirety of the facility, and a second learning unit that generates a second inference model that estimates the state of the part or the entirety of the facility by fine-tuning the first inference model using second training data in which the state is associated with second data reflecting the state of the part or the entirety of the facility as the correct answer data.

[0094] In the information processing device according to aspect 2 of the present invention, in accordance with aspect 1, the first data and the second data are images of a part or the whole of the facility.

[0095] In an information processing device according to aspect 3 of the present invention, in aspect 1 or 2, the numerical data is data indicating at least one of the following values: a measurement value measured in the facility, a setting value set in an apparatus operating in the facility, an operation variable for the apparatus, and a calculated value calculated using at least one of the measurement value, the setting value, and the operation variable.

[0096] An information processing device according to aspect 4 of the present invention is an information processing device according to aspect 3, wherein the equipment is a waste treatment equipment that incinerates waste on a grate, and the numerical data is data indicating any one of the temperature of the post-combustion stage, the flow rate of air sent into the incinerator, and the thickness of the waste on the grate.

[0097] The information processing device according to a fifth aspect of the present invention is in accordance with the second aspect, and uses an image group consisting of a plurality of images forming a time series as one piece of the first data and one piece of the second data.

[0098] An information processing device according to aspect 6 of the present invention is any of aspects 1 to 5, wherein there is a time lag between the first data in a time series and the numerical data in a time series that are associated in the first training data.

[0099] An information processing device according to aspect 7 of the present invention comprises a first learning unit that generates a first inference model by machine learning using first data that reflects the state of part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data that reconstructs the first data based on the extracted features; and a second learning unit that generates a second inference model that estimates the state of part or the entirety of the facility by fine-tuning the first inference model using second training data that corresponds to second data that reflects the state of part or the entirety of the facility as correct answer data.

[0100] A learning method according to aspect 8 of the present invention is a learning method executed by one or more information processing devices, and includes: a first learning step of generating a first inference model that infers the value of numerical data by machine learning using first training data in which each piece of time-series first data reflecting the state of part or the entirety of a specified facility is associated with each piece of time-series numerical data related to the state as correct answer data; and a second learning step of generating a second inference model that estimates the state of part or the entirety of the facility by fine-tuning the first inference model using second training data in which each piece of time-series first data reflecting the state of part or the entirety of the facility is associated with the state as correct answer data.

[0101] A learning method according to aspect 9 of the present invention is a learning method executed by one or more information processing devices, and includes a first learning step of generating a first inference model by machine learning using first data reflecting the state of part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data by reconstructing the first data based on the extracted features; and a second learning step of fine-tuning the first inference model using second training data that corresponds to second data reflecting the state of part or the entirety of the facility as correct data, thereby generating a second inference model that estimates the state of part or the entirety of the facility.

[0102] A learning program according to aspect 10 of the present invention is a learning program for causing a computer to function as an information processing device described in any of aspects 1 to 7, and causes the computer to function as the first learning unit and the second learning unit.

[0103] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0104] 1. Information processing equipment 103 First Study Section 104 Second Study Section 112 First Inference Model 114 Second Inference Model

Claims

1. a first learning unit that generates a first inference model by machine learning using first training data in which time-series first data reflecting the state of a part or the entirety of a specified facility is associated with time-series numerical data relating to the state as correct answer data, and that infers the value of the numerical data; An information processing device comprising: a second learning unit that generates a second inference model that estimates the state of part or the entire facility by fine-tuning the first inference model using second training data that corresponds to second data reflecting the state of part or the entire facility as correct answer data.

2. The information processing device according to claim 1 , wherein the first data and the second data are images of a part of or the entire facility.

3. 3. The information processing device according to claim 1, wherein the numerical data is data indicating at least one of a measured value measured in the facility, a set value set in an apparatus operating in the facility, an operation amount for the apparatus, and a calculated value calculated using at least one of the measured value, the set value, and the operation amount.

4. the facility is a waste treatment facility that incinerates waste on a grate; 4. The information processing device according to claim 3, wherein the numerical data is data indicating any one of a temperature in a post-combustion stage, a flow rate of air sent into the incinerator, and a thickness of the waste on the grate.

5. The information processing apparatus according to claim 2 , wherein an image group consisting of a plurality of images forming a time series is used as the first data and the second data.

6. The information processing apparatus according to claim 1 , wherein there is a time lag between the first data in the time series and the numerical data in the time series associated with each other in the first training data.

7. a first learning unit that generates a first inference model by machine learning using first data that reflects the state of a part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data that reconstructs the first data based on the extracted features; An information processing device comprising: a second learning unit that generates a second inference model that estimates the state of part or the entire facility by fine-tuning the first inference model using second training data that corresponds to second data reflecting the state of part or the entire facility as correct answer data.

8. A learning method executed by one or more information processing devices, comprising: a first learning step of generating a first inference model by machine learning using first training data in which time-series first data reflecting the state of a part or the entirety of a specified facility is associated with time-series numerical data relating to the state as correct answer data, to infer the value of the numerical data; A learning method including: a second learning step of generating a second inference model that estimates the state of part or the entire facility by fine-tuning the first inference model using second training data that corresponds to second data reflecting the state of part or the entire facility as correct answer data.

9. A learning method executed by one or more information processing devices, comprising: a first learning step of generating a first inference model by machine learning using first data that reflects the state of a part or the entirety of a specified facility, extracting features from the first data, and generating reconstructed data that reconstructs the first data based on the extracted features; A learning method including: a second learning step of generating a second inference model that estimates the state of part or the entire facility by fine-tuning the first inference model using second training data that corresponds to second data reflecting the state of part or the entire facility as correct answer data.

10. 10. A learning program for causing a computer to function as the information processing device according to claim 1, wherein the learning program causes the computer to function as the first learning unit and the second learning unit.

Images