Information processing device and information processing method

Abstract

To determine whether a thermal image showing a complex temperature distribution is normal or abnormal.SOLUTION: The information processing device includes a database 3 for storing thermal images, an abnormality detection data registration unit 5 that, if a thermal image of a measurement object is determined to be normal, registers it in the database 3 as a thermal image in a normal state, and, if the thermal image is determined to be abnormal, registers it in the database 3 as a thermal image in an abnormal state, and an abnormality detection learning unit 7 that performs machine learning using the data registered in the database 3 by the abnormality detection data registration unit 5, and generates an abnormality detection machine learning model 9 for determining whether the thermal image is normal or abnormal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method for generating a model for determining whether a thermal image is normal or abnormal, or for estimating an appropriate control command. [Background technology]

[0002] A method has been disclosed in which a thermal image of a determination area including multiple devices is acquired, features are extracted from the thermal image to identify the sections and types of the multiple devices in the thermal image, the temperature in each section of the thermal image corresponding to each of the identified sections of the multiple devices is calculated, and the calculated temperature is compared with a set reference temperature to determine whether the device is abnormal (see Patent Document 1).

[0003] For example, in the field of FA (Factory Automation), there are many objects that can be captured using thermal images. It is expected that this technology will be applied to a wider range of new and effective applications. However, the technology disclosed in Patent Document 1 has the drawback of being difficult to apply to thermal images that show complex temperature distributions, and it is not possible to divide thermal images into sections corresponding to the same model, making it difficult to expand the range of applications. For example, in the case of a thermal image of a mechanism that includes sliding parts, since it is a thermal image of a single mechanism, it is not possible to divide the thermal image into multiple sections, which may result in an erroneous determination of whether the thermal image is normal or abnormal.

[0004] Furthermore, although the technology disclosed in Patent Document 1 can determine whether a thermal image is normal or abnormal, there has not previously been any technology that can estimate what control should be performed to bring the thermal image back to a normal state. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2023-076913 Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made to solve the above-mentioned problems, and aims to provide an information processing device and an information processing method that are capable of determining whether a thermal image showing a complex temperature distribution is normal or abnormal.

[0007] Another object of the present invention is to provide an information processing device and an information processing method that are capable of estimating what control should be performed to bring a thermal image back to a normal state. [Means for solving the problem]

[0008] The information processing device of the present invention is characterized by comprising: a database configured to store thermal images; an anomaly detection data registration unit configured to register a thermal image of a measurement object taken as a normal state in the database if the thermal image is determined to be normal, and to register the thermal image of an abnormal state in the database if the thermal image is determined to be abnormal; and an anomaly detection learning unit configured to perform machine learning using the data registered in the database by the anomaly detection data registration unit, and to generate an anomaly detection machine learning model for determining whether the thermal image is normal or abnormal.

[0009] The information processing device of the present invention is also characterized by comprising: a control command identification unit configured to determine whether the timing at which a thermal image of a measurement object was captured is the timing immediately before the timing at which a control command from a controller controlling the measurement object changed; a mode setting unit configured to set the device to a control command learning mode if the timing at which the thermal image was captured is the timing immediately before the timing at which the control command changed; a first database configured to store thermal images and control command information; a control command data registration unit configured, in the control command learning mode, to register in the first database the thermal image captured the timing immediately before the timing at which the control command changed, the judgment result of whether the thermal image is good or bad after the control command changed, and control command information indicating the content of the control command; and a control command learning unit configured to perform machine learning using the data registered in the first database by the control command data registration unit, and generate a control command machine learning model for estimating a control command that will transition the thermal image to a normal state.

[0010] Furthermore, one configuration example of the information processing device of the present invention further includes a second database configured to store thermal images, an anomaly detection data registration unit configured to, in an anomaly detection learning mode, register a thermal image of the measurement object taken as a thermal image of a normal state in the second database if the thermal image is determined to be normal, and to register the thermal image of the measurement object as a thermal image of an abnormal state in the second database if the thermal image is determined to be abnormal, and an anomaly detection learning unit configured to perform machine learning using the data registered in the second database by the anomaly detection data registration unit to generate an anomaly detection machine learning model for determining whether the thermal image is normal or abnormal, and is characterized in that the mode setting unit sets the anomaly detection learning mode if the timing at which the thermal image was taken is not immediately prior to the timing at which the control command changed. In addition, in one configuration example of the information processing device of the present invention, the control command data registration unit is characterized in that, when a trained anomaly detection machine learning model has been generated, it receives from the anomaly detection machine learning model a judgment result as to whether the thermal image is good or bad after the control command has changed. In one configuration example of the information processing device of the present invention, the measurement object is a mechanism including a sliding portion.

[0011] Furthermore, the information processing method of the present invention is characterized by including a first step of registering a thermal image of a measurement object in a database as a thermal image of a normal state if the thermal image is determined to be normal, and registering the thermal image in the database as a thermal image of an abnormal state if the thermal image is determined to be abnormal, and a second step of performing machine learning using the data registered in the database by the first step to generate an anomaly detection machine learning model for determining whether the thermal image is normal or abnormal.

[0012] The information processing method of the present invention is characterized by including a first step of determining whether the timing at which a thermal image of the object to be measured was captured is the timing immediately before the time at which a control command from a controller controlling the object to be measured changed; a second step of setting a control command learning mode if the timing at which the thermal image was captured is the timing immediately before the time at which the control command changed; a third step of, in the control command learning mode, registering in a first database the thermal image captured at the timing immediately before the time at which the control command changed, a judgment result on the quality of the thermal image after the control command changed, and control command information indicating the content of the control command; and a fourth step of performing machine learning using the data registered in the first database by the third step to generate a control command machine learning model for estimating a control command that will transition the thermal image to a normal state.

[0013] Furthermore, one configuration example of the information processing method of the present invention further includes a fifth step of, in an anomaly detection learning mode, if the thermal image of the object to be measured is determined to be normal, registering it in a second database as a thermal image of a normal state, and if the thermal image is determined to be abnormal, registering it in the second database as a thermal image of an abnormal state; and a sixth step of performing machine learning using the data registered in the second database by the fifth step to generate an anomaly detection machine learning model for determining whether the thermal image is normal or abnormal, wherein the second step is characterized by including a step of setting the anomaly detection learning mode if the timing at which the thermal image was taken is not immediately prior to the timing at which the control command changed. Furthermore, in one configuration example of the information processing method of the present invention, the third step is characterized in that, when a trained anomaly detection machine learning model has been generated, a judgment result as to whether the thermal image is good or bad after the control command has changed is received from the anomaly detection machine learning model. In one configuration example of the information processing method of the present invention, the measurement object is a mechanism including a sliding portion. [Effects of the Invention]

[0014] According to the present invention, by providing a database, an anomaly detection data registration unit, and an anomaly detection learning unit, it is possible to generate an anomaly detection machine learning model. By using the anomaly detection machine learning model, it is possible to determine whether a thermal image is normal or abnormal, even if the thermal image shows a complex temperature distribution.

[0015] Furthermore, in the present invention, a control command machine learning model can be generated by providing a control command identification unit, a mode setting unit, a first database, a control command data registration unit, and a control command learning unit. In the present invention, by using the control command machine learning model, it is possible to estimate what control should be performed to bring the thermal image back to a normal state. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a block diagram showing the configuration of an information processing device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart illustrating the operation of the information processing device according to the embodiment of the present invention. [Figure 3] FIG. 3 is a side view and a cross-sectional view of a mechanism to be measured according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of a thermal image registered in a database in the anomaly detection learning mode in an embodiment of the present invention. [Figure 5] FIG. 5 is a diagram showing another example of a thermal image registered in the database in the anomaly detection learning mode in an embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of a thermal image after a control command is output in the embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing another example of a thermal image after a control command is output in the embodiment of the present invention. [Figure 8] FIG. 8 is a block diagram showing an example of the configuration of a computer that realizes an information processing device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] [Principle 1] For example, manufacturing equipment such as plastic molding machines, food processing machines, machine tools, and semiconductor manufacturing equipment (e.g., CMP (Chemical Mechanical Polisher)) often involves mechanical operations such as conveying, compressing, and polishing, and has some kind of sliding parts. The inventors have discovered that in these manufacturing equipment, mechanisms including sliding parts often repeat the same operations, and that if these operations are repeated under normal conditions, the heat distribution, including frictional heat from sliding, is reproduced in a specific, limited state. In particular, when multiple operations are combined (such as a combination of linear and rotational motion or a combination of rotations of different rotational axes), areas of high-speed operations and areas of low-speed operations are generated in a geometrically complex manner, which easily leads to a unique heat distribution state.

[0018] The inventors then came to the idea that machine learning (unsupervised learning) of thermal images in normal times can be used to detect deviations from the normal state, which can be applied to anomaly detection. Alternatively, machine learning (supervised learning) of thermal images in normal times and thermal images in abnormal times can be applied to detect pre-specified anomalies.

[0019] [Principle 2] In the above case, the heat distribution (temperature distribution) itself may be the target to be controlled. In such cases, depending on the required accuracy of the heat distribution, it may no longer be a simple configuration such as a single-loop PID control system, but may become a complex control system equipped with multiple heating and cooling functions, for example. Therefore, it is preferable to add machine learning using AI (artificial intelligence) or the like so that data on the appropriate control method can be collected through trial and error and the control command can be estimated.

[0020] It is important to note that this is a heat distribution that includes frictional heat from sliding, and is based on a surface thermal image. In other words, unless the control operation directly affects the surface of the object of heat distribution measurement, the effect of the control command will not necessarily appear immediately. Therefore, during trial and error, depending on the type of control operation, the control command tends to be output intermittently (for example, a type in which a cooling fluid is intermittently sprayed onto the heat distribution surface and diffused onto the surface by sliding). Therefore, the inventors came up with the idea of ​​configuring machine learning using AI or the like to limit learning to data (control command information and control result information) at the timing when the control command was output.

[0021] [Example] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of an information processing device according to an embodiment of the present invention. The information processing device includes a control command identifying unit 1 that determines whether the timing at which a thermal image of a measurement target is captured is the timing immediately before a change in a control command from a controller 12 that controls the temperature of the measurement target; a mode setting unit 2 that sets the device to a control command learning mode if the timing at which the thermal image is captured is the timing immediately before a change in the control command, and sets the device to an abnormality detection learning mode if the timing at which the thermal image is captured is not the timing immediately before a change in the control command; a database 3 that stores thermal images for the abnormality detection learning mode; a database 4 that stores thermal images and control command information for the control command learning mode; and a database 5 that stores a thermal image in the abnormality detection learning mode and a thermal image in the database 3 when the thermal image is determined to be normal and when the thermal image is determined to be abnormal. an anomaly detection data registration unit 5 that registers in database 3 a thermal image of an abnormal state when the control command is changed; a control command data registration unit 6 that registers in database 4 the thermal image taken at the timing immediately before the control command is changed in the control command learning mode, the result of determining whether the thermal image is normal or abnormal after the control command is changed, and control command information indicating the content of the control command; an anomaly detection learning unit 7 that performs machine learning using the data registered in database 3 by the anomaly detection data registration unit 5 to generate an anomaly detection machine learning model 9 for determining whether the thermal image is normal or abnormal; and a control command learning unit 8 that performs machine learning using the data registered in database 4 by the control command data registration unit 6 to generate a control command machine learning model 10 for estimating a control command that will cause the thermal image to transition to a normal state.

[0022] 2 is a flowchart illustrating the operation of the information processing device of this embodiment. In this embodiment, the measurement object is, for example, a mechanism including a sliding part in a manufacturing device. The thermal image sensor 11 is, for example, a thermograph, which detects infrared rays emitted from the measurement object to generate two-dimensional temperature distribution data of the measurement object, and converts the temperature indicated by each pixel of the temperature distribution data into a color to generate a thermal image. The thermal image sensor 11 periodically detects infrared rays and generates a thermal image.

[0023] The controller 12 controls the temperature of the measurement object. Here, "control" refers to direct temperature control of the measurement object or control that may affect the temperature of the measurement object. For example, if the measurement object is a mechanism including a sliding part in a manufacturing device, direct temperature control of the measurement object may include control such as blowing cold air to a specific position. Also, control that may affect the temperature of the measurement object may include control of the rotation or linear motion of the mechanism. If the sliding speed (speed of rotation or linear motion) changes, the temperature of the mechanism including the sliding part may also change.

[0024] The control command identification unit 1 determines whether the timing at which the thermal image sensor 11 captured the thermal image was immediately before the time at which the control command from the controller 12 changed (step S100 in FIG. 2). Here, the timing immediately before the time at which the control command changed refers to either the time immediately before or the time immediately after the time at which the control command changed, or the time at which the control command changed. "Immediately before the time at which the control command changed" refers to the range from a time specified time before the time at which the control command changed to the time at which the control command changed. "Immediately after the time at which the control command changed" refers to the range from the time at which the control command changed to the time specified time after the time at which the control command changed.

[0025] The specified time from the point when the control command changes must be set to a time longer than the time from the point when the control command changes until a change appears in the thermal image. In other words, the control command learning mode is specified as the time range for executing a series of controls and confirming their effects. Needless to say, a change in the control command does not only include a change in the content of the control command, but also includes a change from a state where no control command is being output to a state where a control command is being output.

[0026] The mode setting unit 2 sets the mode to the anomaly detection learning mode if the thermal image sensor 11 captures a thermal image not immediately before the control command changes (step S101 in FIG. 2). On the other hand, the mode setting unit 2 sets the mode to the control command learning mode if the thermal image sensor 11 captures a thermal image not immediately before the control command changes (step S102 in FIG. 2).

[0027] In the anomaly detection learning mode, the anomaly detection data registration unit 5 receives the judgment result of the thermal image captured by the thermal image sensor 11 from, for example, an operator of the information processing device, and if the operator judges it to be normal (YES in step S103 of Figure 2), it registers the thermal image in the database 3 as a thermal image of a normal state (step S104 of Figure 2).

[0028] Furthermore, if the operator determines that the thermal image is abnormal (YES in step S105 in FIG. 2), the abnormality detection data registration unit 5 registers the thermal image in the database 3 as a thermal image of an abnormal state (step S106 in FIG. 2). Furthermore, if the operator determines that the thermal image is not abnormal but not normal either, and is transitioning to an abnormal state (NO in step S105), the abnormality detection data registration unit 5 registers the thermal image in the database 3 as a thermal image of a transitioning to an abnormal state (step S107 in FIG. 2).

[0029] The operator checks the thermal image captured by the thermal image sensor 11, for example, on a PC (Personal Computer) 13, and inputs the determination result of whether the image is normal, abnormal, or in a state of transition to abnormality into the information processing device. In addition, the abnormality detection data registration unit 5 assigns one of the labels "normal state," "abnormal state," or "state of transition to abnormality" to the thermal image to be registered in the database 3, depending on the operator's determination result.

[0030] If the control command is intended to return the heat distribution of the measurement object to a normal state, the control command is unnecessary unless an abnormality is detected in the heat distribution. If the heat distribution is determined to be abnormal, the control command output from the controller 12 is likely to change in response to an operator's command.

[0031] In the control command learning mode, the control command data registration unit 6 receives from the operator a judgment result as to whether the change in the thermal image due to the change in the control command is good or bad. If the operator judges that the change in the thermal image is good (the control result is good) (YES in step S108 of FIG. 2), the control command data registration unit 6 registers in the database 4 the thermal image captured immediately before the change in the thermal image to a control command that resulted in a good result (the thermal image captured immediately before or after the change, or the thermal image captured at the time of the change), the judgment result as to whether the change in the thermal image is good or bad, and control command information indicating the content of the control command that resulted in the good change in the thermal image (step S109 of FIG. 2).

[0032] Furthermore, if the operator determines that the change in the thermal image is bad (the control result is bad) (NO in step S108), the control command data registration unit 6 registers in the database 4 the thermal image captured immediately before the change in the thermal image to a control command that resulted in a bad result, the determination result of whether the change in the thermal image is good or bad, and control command information indicating the content of the control command that resulted in the change in the thermal image being bad (step S110 in Figure 2).

[0033] If the control command is intended to return the thermal distribution of the measurement target to a normal state, the thermal distribution for various control commands is collected as thermal image data as a result of various trial operations using the control command. A good thermal image change includes a normal thermal image state, a state where the thermal image is not abnormal but not completely normal, and a state where the operator judges the thermal image to be in a transition to normal. If there are multiple thermal images taken immediately before the control command change, for example, the oldest image may be registered. The control command data registration unit 6 assigns a label to the thermal image registered in the database 4 indicating the judgment result of the good or bad thermal image change and the control command information.

[0034] It is preferable to set the abnormality detection learning mode in the initial state, and to return from the control command learning mode to the abnormality detection learning mode when data registration in the database 4 in the control command learning mode is completed.

[0035] In the above description, the operator is asked to provide a judgment result as to whether the change in the thermal image due to the change in the control command is acceptable or not. However, if a trained anomaly detection machine learning model 9 has already been generated, the control command data registration unit 6 may receive the output of the anomaly detection machine learning model 9 for a thermal image captured after a predetermined waiting time has elapsed since the control command changed as the judgment result as to whether the change in the thermal image due to the change in the control command is acceptable or not. Here, the waiting time must be set to a time longer than the time from the time the control command changed until the change appears in the thermal image. This is the same as the condition for the specified time from the time the control command changed, but the waiting time must be shorter than the specified time.

[0036] When supervised learning is performed on the anomaly detection machine learning model 9 as described below, the trained anomaly detection machine learning model 9 can output a judgment result for the input thermal image, such as "normal state," "abnormal state," or "state of transition to abnormality."

[0037] Furthermore, when unsupervised learning is performed on the anomaly detection machine learning model 9 using a group of normal thermal images, the trained anomaly detection machine learning model 9 can output a score indicating the degree of deviation of the input thermal image from the group of normal thermal images. By comparing the score with a threshold, it is possible to obtain a judgment result for the input thermal image, for example, as being in a "normal state" (close to the group of normal thermal images), an "abnormal state" (deviating from the group of normal thermal images), or a "state of transition to an abnormality" (slightly deviating from the group of normal thermal images).

[0038] Next, when it is time to generate an anomaly detection machine learning model (YES in step S111 of FIG. 2), the anomaly detection learning unit 7 performs machine learning of the anomaly detection machine learning model 9, using the thermal images registered in the database 3 by the anomaly detection data registration unit 5 as input variables and the judgment results (labels) of the thermal images registered in the database 3 as output variables, based on a dataset which is a collection of input variables and output variables, so as to obtain the desired output variables (step S112 of FIG. 2).

[0039] The thermal images registered in the database 3 are labeled as either "normal state," "abnormal state," or "state transitioning to abnormality," so an anomaly detection machine learning model 9 can be generated using a known supervised learning method. A convolutional neural network (CNN) is suitable for machine learning. The timing for generating the anomaly detection machine learning model can be, for example, when an operator gives a command, or when a predetermined number of thermal images have been registered in the database 3.

[0040] Depending on the conditions of the object being measured, there may be cases where only normal thermal images are obtained. In such cases, the anomaly detection learning unit 7 is pre-configured to perform unsupervised learning. In the case of unsupervised learning, the anomaly detection learning unit 7 generates an anomaly detection machine learning model 9 that models the relationship between an input thermal image and a score indicating the degree of deviation of the input thermal image from a group of normal thermal images.

[0041] On the other hand, when the time comes to generate a control command machine learning model (YES in step S113 of Figure 2), the control command learning unit 8 performs machine learning of the control command machine learning model 10 using the thermal image registered in the database 4 by the control command data registration unit 6 as input variables, and the judgment result of whether the change in the thermal image registered in the database 4 is good or bad and the control command information as output variables, based on a dataset which is a collection of input variables and output variables, so as to obtain the desired output variable (step S114 of Figure 2).

[0042] The thermal images registered in the database 4 are assigned labels indicating the result of determining whether the change in the thermal image is good or bad and the control command information, so that the control command machine learning model 10 can be generated by a known supervised learning method. The control command machine learning model can be generated, for example, when an instruction is received from an operator, or when the number of thermal images registered in the database 4 reaches a predetermined number.

[0043] Once the trained anomaly detection machine learning model 9 is generated in this manner, a thermal image can be input to the anomaly detection machine learning model 9, causing the model to determine whether the thermal image (temperature distribution of the measurement object) is normal or abnormal. For example, when supervised learning is performed on the anomaly detection machine learning model 9, the trained anomaly detection machine learning model 9 outputs a determination result for the input thermal image, either a "normal state," an "abnormal state," or a "state transitioning to an abnormal state."

[0044] Furthermore, when unsupervised learning is performed on the anomaly detection machine learning model 9, the trained anomaly detection machine learning model 9 outputs a score indicating the degree of deviation of the input thermal image from a group of normal thermal images. An operator can determine whether the input thermal image is normal or abnormal from the score. The anomaly detection machine learning model 9 may also have a function to compare the score with a threshold and output a determination result of either a "normal state" (the input thermal image is close to the group of normal thermal images), an "abnormal state" (the input thermal image deviates from the group of normal thermal images), or a "transition state to abnormality" (the input thermal image deviates slightly from the group of normal thermal images).

[0045] Furthermore, once a trained control command machine learning model 10 has been generated, a thermal image can be input to the control command machine learning model 10, which can then output control command information and a judgment result on whether the change in the thermal image is good or bad. If a judgment result is obtained that the change in the thermal image is good, the control command information output in combination with this result will be an appropriate control command. The control command machine learning model 10 may be provided with a function to output only control command information that indicates a good change in the thermal image.

[0046] In the following, to make the explanation easier to understand, a fictitious specific example will be used. Figures 3(A) and 3(C) are side views of a mechanism 100 composed of a shaft 101 and a sliding bearing (plain bearing) 102, and Figures 3(B) and 3(D) are cross-sectional views of the mechanism 100. Here, it is assumed that the shaft 101 repeats half-rotation and counter-rotation while simultaneously repeating linear reciprocating motion in the mechanism 100.

[0047] When the shaft 101 repeats movements in multiple directions like this, it is difficult to use a low-friction mechanism such as a ball bearing, and a mechanism including a sliding part is generally used. Note that there are also combinations of movements in multiple directions, such as rotating both the table and the wafer, as in the case of CMP in semiconductor manufacturing equipment. In other words, there are actually mechanisms including a variety of sliding parts in manufacturing equipment.

[0048] Examples of thermal images registered in the database 3 in the anomaly detection learning mode are shown in Figures 4 and 5. In Figures 4 and 5, the thermal image 200 is overlaid on the side view of the mechanism 100 shown in Figures 3(A) and 3(C) so that the imaged location can be seen. While an actual thermograph would display the temperature distribution in color, here a black and white thermal image is used to show an outline of the temperature distribution, with higher temperatures being darker and lower temperatures being lighter.

[0049] Figure 4 shows a thermal image under normal conditions. The notable feature of Figure 4 is that the temperature is maintained sufficiently low except for the upper left of the image area, which is hot (close to black) due to frictional heat. On the other hand, Figure 5 shows a thermal image of an abnormal state. In this example, not only the upper left but also the lower left of the imaged area has become hot due to frictional heat, and the temperature itself is higher than in the example of Figure 4.

[0050] Figures 4 and 5 are for reference purposes only, and it goes without saying that a sufficiently large number of thermal images is required to perform machine learning. By performing machine learning using thermal image data such as those in Figures 4 and 5, it is possible to generate an anomaly detection machine learning model9 that can automatically determine whether a thermal image is normal or abnormal.

[0051] Figures 6 and 7 show examples of thermal images after a control command has been output when the temperature distribution is as shown in Figure 5. Figure 6 shows a thermal image with a favorable change in temperature distribution (good control result). This shows an example of temperature control, specifically the result of control to blow cool air at the position marked with an X 201 in Figure 6. The control command is judged to be good because the temperature has dropped, particularly in the upper right and lower left of the imaging area. As a result, the thermal image shown in Figure 5, which is the thermal image taken just before the control command was output, the judgment that the change in the thermal image is good, and the control command (position information, temperature and volume of the blown cool air) are registered in database 4. The volume and duration of the cool air blown can be precisely adjusted by using a mass flow controller (MFC).

[0052] Figure 7 shows a thermal image with poor temperature distribution change (poor control result). The temperature in the upper right corner of the image capture area has not decreased, which is why it is judged to be a poor control command. As a result, the thermal image in Figure 5, which is the thermal image taken just before the control command was output, the judgment result that the thermal image change is poor, and the control command (location information, temperature and volume of the blown cool air) are registered in Database 4.

[0053] 6 and 7 are reference diagrams for explanation, and it goes without saying that a sufficiently large amount of control command information and thermal images are required to perform machine learning. As described above, if a trained anomaly detection machine learning model 9 has already been generated, the anomaly detection machine learning model 9 may be used to judge thermal images such as those shown in FIGS. 6 and 7. By performing machine learning using control command information and thermal image data, a control command machine learning model 10 that estimates appropriate control commands can be generated.

[0054] As described above, in this embodiment, an anomaly detection machine learning model 9 can be generated, and by using the anomaly detection machine learning model 9, it is possible to determine whether a thermal image is normal or abnormal, even if the thermal image shows a complex temperature distribution.

[0055] Furthermore, in this embodiment, a control command machine learning model 10 can be generated, and by using the control command machine learning model 10, it is possible to estimate what control should be performed to bring the thermal image back to a normal state.

[0056] The information processing device described in this embodiment can be realized by a computer equipped with a CPU (Central Processing Unit), a storage device, and an interface, and a program that controls these hardware resources. An example of the configuration of this computer is shown in Figure 8.

[0057] 8, the computer includes a CPU 300, a storage device 301, and an interface device (I / F) 302. The thermal image sensor 11, the controller 12, the PC 13, etc. are connected to the I / F 302. In such a computer, a program for realizing the information processing method of the present invention is stored in the storage device 301. The CPU 300 executes the processing described in this embodiment in accordance with the program stored in the storage device 301. [Industrial Applicability]

[0058] The present invention can be applied to an automatic determination technique that uses a machine learning model. [Explanation of symbols]

[0059] 1...control command identification unit, 2...mode setting unit, 3...database, 4...database, 5...anomaly detection data registration unit, 6...control command data registration unit, 7...anomaly detection learning unit, 8...control command learning unit, 9...anomaly detection machine learning model, 10...control command machine learning model, 11...thermal image sensor, 12...controller

Claims

1. a database configured to store the thermal images; an anomaly detection data registration unit configured to register a thermal image of a measurement target in the database as a thermal image of a normal state when the thermal image is determined to be normal, and to register the thermal image in the database as a thermal image of an abnormal state when the thermal image is determined to be abnormal; an anomaly detection learning unit configured to perform machine learning using the data registered in the database by the anomaly detection data registration unit and generate an anomaly detection machine learning model for determining whether a thermal image is normal or abnormal.

2. a control command identifying unit configured to determine whether a timing at which a thermal image of the measurement object is captured is the timing immediately before a time at which a control command from a controller controlling the measurement object is changed; a mode setting unit configured to set the control command learning mode when the timing at which the thermal image is captured is the timing immediately before the timing at which the control command is changed; a first database configured to store thermal images and control command information; a control command data registration unit configured to register, in the control command learning mode, a thermal image captured immediately before the control command is changed, a quality determination result of the thermal image after the control command is changed, and control command information indicating the content of the control command in the first database; and a control command learning unit configured to perform machine learning using the data registered in the first database by the control command data registration unit and generate a control command machine learning model for estimating a control command that will transition the thermal image to a normal state.

3. 3. The information processing device according to claim 2, a second database configured to store the thermal images; an anomaly detection data registration unit configured to, in an anomaly detection learning mode, when a thermal image of the measurement object is determined to be normal, register the thermal image in the second database as a thermal image of a normal state, and when the thermal image is determined to be abnormal, register the thermal image in the second database as a thermal image of an abnormal state; an anomaly detection learning unit configured to perform machine learning using the data registered in the second database by the anomaly detection data registration unit and generate an anomaly detection machine learning model for determining whether a thermal image is normal or abnormal; The information processing device is characterized in that the mode setting unit sets the abnormality detection learning mode when the timing at which the thermal image is captured is not immediately before the timing at which the control command changes.

4. 4. The information processing device according to claim 3, The information processing device is characterized in that, when a trained anomaly detection machine learning model has been generated, the control command data registration unit receives from the anomaly detection machine learning model a judgment result as to whether the thermal image is good or bad after the control command has changed.

5. 5. The information processing device according to claim 1, The information processing device, wherein the object to be measured is a mechanism including a sliding portion.

6. a first step of registering a thermal image of a measurement target in a database as a thermal image of a normal state when the thermal image is determined to be normal, and registering the thermal image in the database as a thermal image of an abnormal state when the thermal image is determined to be abnormal; and a second step of performing machine learning using the data registered in the database by the first step to generate an anomaly detection machine learning model for determining whether a thermal image is normal or abnormal.

7. a first step of determining whether a timing at which a thermal image of a measurement object is captured is the timing immediately before a change in a control command from a controller that controls the measurement object; a second step of setting a control command learning mode when the timing at which the thermal image was captured is the timing immediately before the time at which the control command was changed; a third step of registering, in a first database, in the control command learning mode, a thermal image captured immediately before the control command is changed, a result of determining whether the thermal image is good or bad after the control command is changed, and control command information indicating the content of the control command; and a fourth step of performing machine learning using the data registered in the first database by the third step to generate a control command machine learning model for estimating a control command that will transition the thermal image to a normal state.

8. 8. The information processing method according to claim 7, a fifth step of, in an anomaly detection learning mode, registering the thermal image of the measurement object as a thermal image of a normal state in a second database when the thermal image is determined to be normal, and registering the thermal image as a thermal image of an abnormal state in the second database when the thermal image is determined to be abnormal; a sixth step of performing machine learning using the data registered in the second database in the fifth step to generate an anomaly detection machine learning model for determining whether a thermal image is normal or abnormal; The information processing method is characterized in that the second step includes a step of setting the abnormality detection learning mode if the timing at which the thermal image was captured is not immediately prior to the time at which the control command changed.

9. 9. The information processing method according to claim 8, The third step is an information processing method characterized in that, when a trained anomaly detection machine learning model has been generated, a judgment result as to whether the thermal image is good or bad after the control command has changed is received from the anomaly detection machine learning model.

10. 10. The information processing method according to claim 6, An information processing method, wherein the object to be measured is a mechanism including a sliding part.

Images