Machine learning apparatus and controlled device abnormality analysis apparatus

Abstract

An objective of the present invention is to generate a trained model for accurately detecting an abnormality occurring in a controlled device.　A machine learning apparatus disclosed herein comprises: an input data acquisition unit that acquires information about a controlled device as input data in an environment in which a PLC controls the controlled device, the information including one or more monitoring images related to the controlled device during normal operation of the controlled device and / or sound information emitted by the controlled device; a label acquisition unit that acquires, as label data, the state of input / output signals related to the controlled device at the time when the information is acquired; and a training unit that executes supervised training using the input data acquired by the input data acquisition unit and the label data acquired by the label acquisition unit, and generates a trained model that is input with the information including the monitoring images or sound information of the controlled device controlled by the PLC, and outputs the state of the input / output signals.

Description

Machine learning device and control equipment abnormality analysis device

[0001] The present disclosure relates to a machine learning device and a control device abnormality analysis device that detect abnormalities in control devices controlled by PLCs.

[0002] Abnormalities may occur in controlled devices controlled by a PLC (Programmable Logic Controller). Therefore, maintenance personnel periodically inspect the controlled devices to determine whether any abnormalities have occurred. Also, a technology known as a dual check safety function is known that duplicates signals from the controlled devices controlled by the PLC to detect abnormalities. For example, see Patent Document 1.

[0003] Japanese Patent Application Laid-Open No. 2023-161769

[0004] The technology described in Patent Document 1 can detect abnormalities in the PLC's CPU and broken lines connecting the PLC to the controlled device, but may not be able to detect abnormalities that affect both duplicated signals in the controlled device.

[0005] Therefore, it is desirable to generate a trained model that can accurately detect abnormalities that occur in controlled devices.

[0006] (1) One aspect of the machine learning device of the present disclosure includes an input data acquisition unit that acquires, in an environment in which a PLC controls a controlled device, information including one or more monitoring images of the controlled device and / or sound information emitted by the controlled device when the controlled device is operating normally, as input data; a label acquisition unit that acquires, as label data, the states of input / output signals related to the controlled device at the time the information is acquired; and a learning unit that performs supervised learning using the input data acquired by the input data acquisition unit and the label data acquired by the label acquisition unit, inputs information including monitoring images or sound information of the controlled device controlled by the PLC, and generates a trained model that outputs the states of the input / output signals.

[0007] (2) One aspect of the control equipment abnormality analysis device of the present disclosure includes an input unit that accepts a trained model generated by the machine learning device of (1), information including one or more monitoring images related to controlled equipment controlled by a PLC and / or sound information emitted by the controlled equipment, and the state of input / output signals at the time the information was acquired, an abnormality determination unit that inputs the information accepted by the input unit to the trained model, compares the state of the input / output signals output from the trained model with the state of the input / output signals at the time the information was acquired, and determines whether the controlled equipment is abnormal based on the comparison result, and an abnormality notification unit that notifies the determination result of the abnormality determination unit.

[0008] 1 is a diagram showing an example of a functional block configuration of an anomaly monitoring system according to a first embodiment; FIG. 2 is a diagram showing an example of the state of input / output signals of a PLC (not shown) at the time when information is acquired; FIG. 3 is a diagram showing an example of the state of input / output signals output from a trained model; FIG. 4 is a flowchart illustrating the machine learning processing of a machine learning device; FIG. 5 is a flowchart illustrating the analysis processing of a control equipment abnormality analysis device; FIG. 6 is a diagram showing an example of a functional block configuration of an anomaly monitoring system according to a second embodiment; FIG. 7 is a diagram showing an example of sounds emitted by two fans; FIG. 8 is a diagram showing an example of the state of input / output signals at the time when information is acquired; FIG. 9 is a diagram showing an example of the state of input / output signals output from a trained model; FIG. 10 is a flowchart illustrating the machine learning processing of a machine learning device;

[0009] First Embodiment An anomaly monitoring system according to a first embodiment will be described in detail below with reference to the drawings. Here, the description will be given using, as an example, an operation panel for operating a machine tool as a controlled machine controlled by a PLC (not shown), a safety fence, and an LED that lights up to indicate the operating status of the machine tool. Note that the present invention is also applicable to controlled machines other than the operation panel, safety fence, and LED, such as a blower or a pump that circulates oil or coolant. FIG. 1 is a diagram showing an example of the functional block configuration of the anomaly monitoring system according to the first embodiment. As shown in FIG. 1, the anomaly monitoring system 1 includes a control equipment anomaly analysis device 10, a machine learning device 20, a machine tool 30, a safety fence 40, a monitoring camera 50, and an LED 60. The machine tool 30 also includes an operation panel 35. The safety fence 40 also includes a door 41 and an opening / closing sensor 45 that detects whether the door 41 is open or closed. The control equipment abnormality analysis device 10, the machine learning device 20, the machine tool 30, the opening / closing sensor 45 of the safety fence 40, and the LED 60 may be connected to each other and communicate via a network (not shown) such as a local area network (LAN) or the Internet. In this case, the control equipment abnormality analysis device 10, the machine learning device 20, the machine tool 30, the opening / closing sensor 45, and the LED 60 are provided with a communication unit (not shown) for communicating with each other via such a connection. Note that the control equipment abnormality analysis device 10, the machine learning device 20, the machine tool 30, the opening / closing sensor 45, and the LED 60 may be directly connected to each other via a connection interface (not shown). Furthermore, although the control equipment abnormality analysis device 10 is described as being a device separate from the machine learning device 20, it may include the machine learning device 20, as described below.

[0010] <Machine Tool 30> Machine tool 30 is, for example, a machine tool known to those skilled in the art. Machine tool 30 receives commands generated based on a machining program created by a CAD / CAM device (not shown) or an operation panel 35 included in machine tool 30 from a numerical control device (not shown) and operates in response to the commands. Machine tool 30 may also be configured to output the status of input / output signals between it and a PLC (not shown) to control equipment abnormality analysis device 10 (described below).

[0011] <Safety Fence 40> The safety fence 40 is intended to restrict people such as an operator from approaching the machine tool 30. The operator enters and exits the safety fence 40 through a door 41 to operate the control panel 35 of the machine tool 30, for example. An opening / closing sensor 45 is provided on the safety fence 40 to detect the entrance and exit of the operator, for example. The opening / closing sensor 45 may also be configured to output the state of an input / output signal to and from a PLC (not shown) in response to the opening and closing of the door 41 to the control equipment abnormality analysis device 10, which will be described later.

[0012] <Monitoring camera 50> The monitoring camera 50 is, for example, a digital camera known to those skilled in the art. The monitoring camera 50 captures images of the opening and closing of the door 41 of the safety fence 40, the lighting of the LED 60 (described later), etc., and generates monitoring images. The monitoring camera 50 outputs the generated monitoring images to the control equipment abnormality analysis device 10 (described later).

[0013] <LED 60> LED 60 is, for example, a light-emitting diode known to those skilled in the art, and lights up in response to the operating state or abnormality of machine tool 30. This allows the operator to know the operating state or abnormality of machine tool 30. LED 60 may also be configured to output the state of input / output signals between it and a PLC (not shown) to control equipment abnormality analysis device 10, which will be described later.

[0014] <Control Equipment Fault Analysis Device 10> In the operation phase, the control equipment fault analysis device 10 receives and acquires information on a monitoring image captured by a monitoring camera 50, including the controlled devices of the operation panel 35, safety fence 40, and LED 60 of the machine tool 30. The control equipment fault analysis device 10 inputs the acquired information into a trained model provided by a machine learning device 20 (described later), thereby acquiring the status of input / output signals of a PLC (not shown) between the operation panel 35, safety fence 40, and LED 60 and each of the controlled devices. Note that in FIG. 1 , the control equipment fault analysis device 10 may acquire the status of input / output signals of a PLC (not shown) from the operation panel 35, the opening / closing sensor 45 of the safety fence 40, and the LED 60 of the machine tool 30. The control equipment fault analysis device 10 may then output the acquired status of the input / output signals of the PLC to the machine learning device 20 (described later).

[0015] Before explaining the control equipment abnormality analysis device 10, we will explain machine learning for generating a trained model.

[0016] <Machine Learning Device 20> The machine learning device 20 acquires, as input data, a monitoring image captured by a monitoring camera 50 while the machine tool 30 is operating normally, including the control equipment of the operation panel 35, safety fence 40, and LED 60 of the machine tool 30. The machine learning device 20 also acquires, as labels (correct answers), data indicating the state of input / output signals of a PLC (not shown) between the operation panel 35 of the machine tool 30, the opening / closing sensor 45 of the safety fence 40, and the LED 60 from the control equipment fault analysis device 10. The machine learning device 20 performs supervised learning using training data consisting of pairs of the acquired input data and labels, and constructs a trained model (described below). In this way, the machine learning device 20 can provide the constructed trained model to the control equipment fault analysis device 10. The machine learning device 20 will now be described in detail.

[0017] 1 , the machine learning device 20 has an input data acquisition unit 210, a label acquisition unit 220, a learning unit 230, and a storage unit 240. In the learning phase, the input data acquisition unit 210 acquires, as input data, a monitoring image captured by the monitoring camera 50 during normal operation, the image including the controlled equipment of the operation panel 35, safety fence 40, and LED 60 of the machine tool 30. The input data acquisition unit 210 outputs the acquired input data to the storage unit 240.

[0018] The label acquisition unit 220 acquires, as a label (correct answer), data indicating the state of the input / output signals of the PLC (not shown) between the operation panel 35 of the machine tool 30, the open / close sensor 45 of the safety fence 40, and the LED 60 during normal operation, which corresponds to the respective operating states of the operation panel 35 of the machine tool 30, the safety fence 40, and the LED 60 in the monitoring image of the input data, via the control equipment abnormality analysis device 10. The label acquisition unit 220 outputs the acquired label data to the storage unit 240.

[0019] The learning unit 230 receives the above-described pairs of input data and labels as training data, and performs supervised learning using the received training data to generate a trained model 141 that receives information including monitoring images of controlled devices controlled by a PLC (not shown) and outputs the status of input / output signals of the PLC. The learning unit 230 then provides the constructed trained model 141 to the control equipment abnormality analysis device 10. Note that supervised learning can use a neural network, a convolutional neural network (CNN), a support vector machine (SVM), or the like.

[0020] Furthermore, when new teacher data is acquired after the learning unit 230 has constructed the trained model 141, the learning unit 230 may update the trained model 141 that has been constructed by further performing supervised learning on the trained model 141. Furthermore, the trained model 141 may be shared with other control equipment fault analysis devices 10. If the trained model 141 is shared by a plurality of control equipment fault analysis devices 10, it becomes possible to perform supervised learning in a distributed manner in each control equipment fault analysis device 10, thereby improving the efficiency of supervised learning.

[0021] The above-described supervised learning may be performed by online learning, batch learning, or mini-batch learning. Online learning is a learning method in which the machine learning device 20 immediately performs supervised learning each time it acquires a monitoring image captured by the monitoring camera 50 and the state of an input / output signal of a PLC (not shown). Batch learning is a learning method in which the machine learning device 20 repeatedly acquires a monitoring image captured by the monitoring camera 50 and the state of an input / output signal of a PLC (not shown) and creates training data, collects multiple training data corresponding to the repetitions, and performs supervised learning using all of the collected training data. Mini-batch learning is a learning method that is intermediate between online learning and batch learning, in which supervised learning is performed each time a certain amount of training data is accumulated.

[0022] The storage unit 240 is a RAM (Random Access Memory) or the like, and stores the input data acquired by the input data acquisition unit 210, the label data acquired by the label acquisition unit 220, and the trained model 141 constructed by the learning unit 230. The machine learning for generating the trained model 141 included in the control equipment fault analysis device 10 has been described above. Next, the control equipment fault analysis device 10 in the operation phase will be described.

[0023] <Control Equipment Fault Analysis Device 10 in Operation Phase> As shown in Fig. 1 , the control equipment fault analysis device 10 in the operation phase is configured to include an input unit 110, an fault determination unit 120, an fault notification unit 130, and a storage unit 140. Note that the control equipment fault analysis device 10 is equipped with a calculation processing device (not shown) such as a CPU (Central Processing Unit) in order to realize the operations of the functional blocks in Fig. 1. The control equipment fault analysis device 10 is also equipped with auxiliary storage devices (not shown) such as a ROM (Read Only Memory), SSD (Solid State Drive), or HDD (Hard Disk Drive) that store various control programs, and a main storage device (not shown) such as a RAM for storing data temporarily required for the calculation processing device to execute the programs.

[0024] In the control equipment fault analysis device 10, the arithmetic processing device reads the OS and application software from the auxiliary storage device, and performs arithmetic processing based on the OS and application software while expanding the read OS and application software into the main storage device. Based on the results of this calculation, the control equipment fault analysis device 10 controls each piece of hardware. In this way, the processing by the functional blocks in Figure 1 is realized. In other words, the control equipment fault analysis device 10 can be realized by the cooperation of hardware and software.

[0025] The input unit 110 receives, from the monitoring camera 50 and the PLC (not shown), information including monitoring images (and / or sound information emitted by the machine tool 30, etc.) related to the control panel 35, safety fence 40, and LED 60 of the machine tool 30, which are controlled devices controlled by the PLC (not shown), as well as the state of input / output signals of the PLC (not shown) at the time the information was acquired. FIG. 2 is a diagram showing an example of the state of the input / output signals of the PLC (not shown) at the time the information was acquired. In FIG. 2, the state of the input / output signals is represented by an 8-bit signal. For example, the input / output signal "X10" indicates the state of the door 41, the input / output signal "X11" indicates the state of the control panel 35 of the machine tool 30, and the input / output signal "Y20" indicates the state of the LED 60. The input unit 110 outputs the received information including the monitoring images to the abnormality determination unit 120 and stores the received state of the input / output signals of the PLC (not shown) in the memory unit 140.

[0026] The abnormality judgment unit 120 inputs information received by the input unit 110 to the trained model 141, compares the state of the input / output signals output from the trained model 141 with the state of the input / output signals at the time the information was acquired, and judges whether or not the controlled device is abnormal based on the comparison result. Specifically, the abnormality judgment unit 120 inputs, for example, a monitoring image included in the information received by the input unit 110 to the trained model 141 and receives the state of the input / output signals output from the trained model 141. FIG. 3 is a diagram showing an example of the state of the input / output signals output from the trained model 141. The abnormality judgment unit 120 reads out from the memory unit 140 the state of the input / output signals of a PLC (not shown) at the time the information including the monitoring image was acquired, and compares the read state of the input / output signals with the state of the input / output signals output from the trained model 141. If the state of the read input / output signal matches the state of the input / output signal output from the trained model 141, the abnormality judgment unit 120 judges that the operation of the controlled equipment, namely the operation panel 35 of the machine tool 30, the door 41 of the safety fence 40, and the LED 60, is normal. On the other hand, for example, since the state of the read input / output signal shown in Figure 2 and the state of the input / output signal output from the trained model 141 shown in Figure 3 differ in the first bit of the input / output signal "X11", the abnormality judgment unit 120 judges that an abnormality has occurred in the operation of at least the controlled equipment, namely the door 41 of the safety fence 40.

[0027] The abnormality notification unit 130 notifies the judgment result of the abnormality judgment unit 120 to an operator's smartphone, tablet terminal, personal computer, etc. (not shown), and displays it on the display of the smartphone, tablet terminal, personal computer, etc. (not shown).

[0028] <Machine Learning Process of Machine Learning Device 20> Next, the flow of the machine learning process of the machine learning device 20 will be described with reference to Fig. 4. Fig. 4 is a flowchart illustrating the machine learning process of the machine learning device 20.

[0029] In step S11 , the input data acquisition unit 210 acquires, as input data, a monitoring image including the controlled device captured by the monitoring camera 50 via the control device abnormality analysis device 10 .

[0030] In step S12, the label acquisition unit 220 acquires, via the control equipment abnormality analysis device 10, the state of the input / output signals of the PLC (not shown) that corresponds to the operating state of each controlled device in the monitoring image of the input data acquired in step S11, as a label (correct answer).

[0031] In step S13, the learning unit 230 accepts the pair of input data acquired in step S11 and the label acquired in step S12 as training data, and performs supervised learning using the accepted training data to generate a trained model 141 that inputs information including monitoring images of controlled equipment controlled by a PLC (not shown) and outputs the state of input / output signals of the PLC (not shown).

[0032] <Analysis Processing of Control Equipment Fault Analysis Device 10> Next, the flow of the analysis processing of the control equipment fault analysis device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart illustrating the analysis processing of the control equipment fault analysis device 10.

[0033] In step S21, the input unit 110 receives information from the monitoring camera 50, including a monitoring image relating to a controlled device controlled by a PLC (not shown).

[0034] In step S22, the input unit 110 receives from the PLC (not shown) the state of the input / output signals of the PLC at the time when the information received in step S21 was acquired.

[0035] In step S23, the abnormality judgment unit 120 inputs the monitoring image contained in the information received in step S21 into the trained model 141 generated by the machine learning processing of the machine learning device 20 in Figure 4, and acquires the state of the PLC input / output signals output from the trained model 141.

[0036] In step S24, the abnormality judgment unit 120 compares the state of the input / output signals output from the trained model 141 in step S23 with the state of the input / output signals at the time the information received in step S21 was acquired.

[0037] In step S25, if the state of the input / output signals output from the trained model 141 matches the state of the input / output signals at the time the information received in step S21 was acquired, the abnormality determination unit 120 determines that the controlled device is normal. On the other hand, if the state of the input / output signals output from the trained model 141 does not match the state of the input / output signals at the time the information received in step S21 was acquired, the abnormality determination unit 120 determines that the controlled device is abnormal.

[0038] In step S26, the abnormality notification unit 130 notifies and displays the determination result of step S25 on the operator's smartphone, tablet terminal, personal computer, or the like (not shown).

[0039] As described above, the machine learning device 20 according to the first embodiment can generate a trained model for detecting an abnormality that has occurred in a control device with high accuracy, and the control device abnormality analysis device 10 can detect an abnormality that has occurred in a control device with high accuracy from device monitoring information including a monitoring image captured by the monitoring camera 50. This completes the description of the first embodiment.

[0040] Second Embodiment Next, the second embodiment will be described. In the first embodiment, the machine learning device 20 acquires, as input data, information including one or more monitoring images of a controlled device when the controlled device is operating normally. The control equipment fault analysis device 10 inputs, to the trained model 141, information including monitoring images of the controlled device captured by the monitoring camera 50, and acquires the state of input / output signals output from the trained model 141. In contrast, the second embodiment differs from the first embodiment in that the machine learning device 20A acquires, as input data, information including sound information emitted by the controlled device when the controlled device is operating normally. The control equipment fault analysis device 10A inputs, to the trained model 141a, information including sound information related to the controlled device collected by the microphone 90, and acquires the state of input / output signals output from the trained model 141a. As a result, according to the second embodiment, the machine learning device 20A can generate a trained model for detecting an abnormality occurring in the control device with high accuracy. The second embodiment will be described below.

[0041] FIG. 6 is a diagram showing an example of a functional block configuration of an anomaly monitoring system 1 according to the second embodiment. Elements having similar functions to those of the anomaly monitoring system 1 in FIG. 1 are designated by the same reference numerals, and detailed description thereof will be omitted. As shown in FIG. 6 , the anomaly monitoring system 1 includes a control equipment anomaly analysis device 10A, a machine learning device 20A, two fans 70(1) and 70(2), a control panel 80 for the fans 70(1) and 70(2), and a microphone 90. While FIG. 6 illustrates two fans 70(1) and 70(2) as sound sources, the same applies to a case where two or more machine tools are installed, since machine tools also emit sounds during machining or other operations.

[0042] <Blowers 70(1), 70(2)> The blowers 70(1), 70(2) are general fans that are placed inside a building such as a factory and used for ventilation, air circulation, etc. The blowers 70(1), 70(2) operate based on the operation of a control panel 80 by an operator or the like.

[0043] <Microphone 90> The microphone 90 is a general microphone that is placed in a building such as a factory to collect sounds from the fans 70(1) and 70(2). Although one microphone 90 is placed in Fig. 6, two or more microphones 90 may be placed, and a microphone 90 may be placed for each of the controlled devices such as the fans 70(1) and 70(2).

[0044] <Machine Learning Device 20A> The machine learning device 20A acquires, as input data, information including sound information of controlled devices such as the fans 70(1) and 70(2) collected by a microphone 90 when the fans 70(1) and 70(2) are operating normally from the control equipment fault analysis device 10A. The machine learning device 20A also acquires, as labels (correct answers), data indicating the states of input / output signals of a PLC (not shown) between the fans 70(1) and 70(2) from the control equipment fault analysis device 10A. The machine learning device 20A performs supervised learning using training data consisting of pairs of the acquired input data and labels, and constructs a trained model 141a. In this way, the machine learning device 20A can provide the constructed trained model 141a to the control equipment fault analysis device 10A. The machine learning device 20A will now be described in detail.

[0045] 6, the machine learning device 20A includes an input data acquisition unit 210a, a label acquisition unit 220, a learning unit 230a, and a storage unit 240. The label acquisition unit 220 and the storage unit 240 have the same functions as the label acquisition unit 220 and the storage unit 240 of the first embodiment.

[0046] During the learning phase, the input data acquisition unit 210a acquires, as input data via the control equipment abnormality analysis device 10A, information including sound information of the controlled devices, the fans 70(1) and 70(2), collected by the microphone 90. FIG. 7 is a diagram illustrating an example of sounds emitted by the two fans 70(1) and 70(2). The first row of FIG. 7 shows an example of the change over time of a sound with a frequency of 295 Hz emitted by the fan 70(1). The second row shows an example of the change over time of a sound with a frequency of 605 Hz emitted by the fan 70(2). The third row shows an example of the change over time of a sound obtained by superimposing the sounds of the fans 70(1) and 70(2) collected by the microphone 90. The fourth row shows an example of a spectrum obtained by Fourier transforming the sound collected by the microphone 90 in the third row, which shows that the natural frequencies of 295 Hz and 605 Hz of the fans 70(1) and 70(2) are separated. Therefore, the input data acquisition unit 210a performs a Fourier transform on the acquired input data and stores the spectral data in the storage unit 240. Note that the input data acquisition unit 210a may store the acquired input data directly in the storage unit 240.

[0047] The learning unit 230a receives the above-described sets of input data and labels as training data and performs supervised learning using the received training data to generate a trained model 141a that inputs information including sound information of a controlled device controlled by a PLC (not shown) and outputs the state of an input / output signal of the PLC. When performing machine learning, the learning unit 230a reads Fourier-transformed sound spectrum data from the storage unit 240 as input data. This allows the generated trained model 141a to distinguish between the fans 70(1) and 70(2) and output the state of the input / output signals of the fans 70(1) and 70(2). The learning unit 230a provides the constructed trained model 141a to the control equipment abnormality analysis device 10A.

[0048] <Control equipment fault analysis device 10A in operation phase> As shown in Fig. 6, the control equipment fault analysis device 10A in the operation phase is configured to include an input unit 110a, an fault determination unit 120a, an fault notification unit 130, and a storage unit 140. Note that the control equipment fault analysis device 10A is equipped with an arithmetic processing device (not shown) such as a CPU in order to realize the operations of the functional blocks in Fig. 6. The control equipment fault analysis device 10A is also equipped with an auxiliary storage device (not shown) such as a ROM, SSD, or HDD that stores various control programs, and a main storage device (not shown) such as a RAM for storing data temporarily required when the arithmetic processing device executes the programs.

[0049] In the control equipment fault analysis device 10A, the arithmetic processing unit reads the OS and application software from the auxiliary storage device, and executes arithmetic processing based on the OS and application software while expanding the read OS and application software into the main storage device. Based on the results of this calculation, the control equipment fault analysis device 10A controls each piece of hardware. This realizes the processing represented by the functional blocks in Figure 6. In other words, the control equipment fault analysis device 10A can be realized by the cooperation of hardware and software. The fault notification unit 130 and the memory unit 140 have the same functions as the fault notification unit 130 and the memory unit 140 of the first embodiment.

[0050] The input unit 110a receives, from the microphone 90 and the PLC (not shown), information including sounds emitted by the fans 70(1) and 70(2), which are controlled devices controlled by a PLC (not shown), and the state of the input / output signals at the time the information was acquired. FIG. 8 is a diagram showing an example of the state of the input / output signals at the time the information was acquired. In FIG. 8, the state of the input / output signals is represented by an 8-bit signal. For example, the input / output signals "X0" and "Y0" indicate the operating state of the fan 70(1), and the input / output signals "X10" and "Y10" indicate the state of the fan 70(2). The input unit 110 outputs the received information including the sounds to the abnormality determination unit 120a and stores it in the memory unit 140.

[0051] The abnormality determination unit 120a performs a Fourier transform on the sound included in the information received by the input unit 110 to convert it into sound spectrum data, and inputs the converted sound spectrum data to the trained model 141a. The abnormality determination unit 120a receives the state of the input / output signals output from the trained model 141a. FIG. 9 is a diagram showing an example of the state of the input / output signals output from the trained model 141a. The abnormality determination unit 120a reads the state of the input / output signals at the time the information including the sound was acquired from the storage unit 140, and compares the read state of the input / output signals with the state of the input / output signals output from the trained model 141a. As shown in FIGS. 8 and 9 , if the read state of the input / output signals matches the state of the input / output signals output from the trained model 141a, the abnormality determination unit 120a determines that the operation of the controlled devices, i.e., the fans 70(1) and 70(2), is normal. On the other hand, if the state of the read input / output signal does not match the state of the input / output signal output from the trained model 141a, the abnormality judgment unit 120a judges that the operation of blower 70(1) or blower 70(2) whose input / output signal state does not match is abnormal.

[0052] <Machine Learning Processing of Machine Learning Device 20A> Next, the flow of the machine learning processing of the machine learning device 20A will be described with reference to Fig. 10. Fig. 10 is a flowchart illustrating the machine learning processing of the machine learning device 20A. Note that the processing of step S32 is similar to the processing of step S12 in Fig. 4, and therefore description thereof will be omitted.

[0053] In step S31, the input data acquisition unit 210a receives, via the control equipment abnormality analysis device 10A, the sound emitted by the controlled equipment and collected by the microphone 90. The input data acquisition unit 210a performs a Fourier transform on the received sound, and acquires the spectrum data of the converted sound as input data.

[0054] In step S33, the learning unit 230a accepts the pair of input data acquired in step S31 and the label acquired in step S32 as training data, and performs supervised learning using the accepted training data to generate a trained model 141a that inputs information including the sound of a controlled device controlled by a PLC (not shown) and outputs the state of the input / output signals of the PLC (not shown).

[0055] <Analysis Processing of Control Equipment Fault Analysis Device 10A> Next, the flow of the analysis processing of the control equipment fault analysis device 10A will be described with reference to Fig. 11. Fig. 11 is a flowchart illustrating the analysis processing of the control equipment fault analysis device 10A. Note that the processing of step S42 and steps S44 to S46 is the same as the processing of step S22 and steps S24 to S26 in Fig. 5, and therefore description thereof will be omitted.

[0056] In step S41, the input unit 110a receives, from the microphone 90, information including a sound emitted by a controlled device controlled by a PLC (not shown).

[0057] In step S43, the abnormality determination unit 120a performs a Fourier transform on the sound included in the information received in step S41, and inputs the spectrum data of the converted sound to the trained model 141a generated by the machine learning process of the machine learning device 20A in Fig. 10. The abnormality determination unit 120a acquires the state of the input / output signal output from the trained model 141a.

[0058] As described above, the machine learning device 20A according to the second embodiment can generate a trained model for detecting an abnormality that has occurred in a control device with high accuracy, and the control device abnormality analysis device 10A can detect an abnormality that has occurred in a control device with high accuracy from the device monitoring information that includes sound collected by the microphone 90. This completes the description of the second embodiment.

[0059] Modification of Second Embodiment In the second embodiment, one microphone 90 is provided, but this is not limiting. For example, a microphone 90 may be provided for each controlled device, such as the fans 70(1) and 70(2) or a machine tool (not shown). In this case, the machine learning device 20A may acquire information including sound from the microphone 90 for each controlled device during normal operation and accept, as training data, a set of input data of the acquired sound information (e.g., sound volume, pitch, etc.) for each controlled device and labels of the state of the input / output signals. The machine learning device 20A may perform supervised learning for each controlled device using the accepted training data for each controlled device, thereby generating, for each controlled device, a trained model that inputs information including sound of the controlled device controlled by a PLC (not shown) and outputs the state of the input / output signals of the PLC (not shown). In this way, the control equipment abnormality analysis device 10A can determine whether each controlled equipment is abnormal by inputting the sound information (e.g., sound volume (dB) and pitch (Hz)) of each controlled equipment into the trained model for each controlled equipment without performing a Fourier transform on the sound information acquired from the microphone 90.

[0060] As described above, as described in the first embodiment, the second embodiment, and the modified example of the second embodiment, the machine learning devices 20 and 20A of the present disclosure can generate trained models for detecting abnormalities that occur in control devices with high accuracy.

[0061] <Modification> In the first embodiment, the second embodiment, and the modification of the second embodiment described above, the control equipment fault analysis device 10, 10A is a device different from the machine learning device 20, but is not limited to this. For example, the control equipment fault analysis device 10, 10A may include the machine learning device 20.

[0062] Note that the functions included in the machine learning devices 20 and 20A and the control equipment abnormality analysis devices 10 and 10A in the first embodiment, the second embodiment, and the modified example of the second embodiment can be realized by hardware, software, or a combination thereof. Here, "realized by software" means that the functions are realized by a computer reading and executing a program.

[0063] The program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs). The program may be provided to the computer by various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable media can provide the program to the computer via a wired communication path such as an electrical wire or optical fiber, or via a wireless communication path.

[0064] The step of executing the program recorded on the recording medium includes not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually. Also, the step of writing the program may be performed by cloud computing.

[0065] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0066] The following supplementary note is further disclosed regarding the above-described embodiment and modified examples. (Supplementary Note 1) The machine learning device (20, 20A) includes an input data acquisition unit (210, 210a) that acquires, as input data, information including one or more monitoring images of the controlled devices (30, 35, 40, 45, 60, 70(1), 70(2)) and / or sound information emitted by the controlled devices (30, 70(1), 70(2)) when the controlled devices (30, 35, 40, 45, 60, 70(1), 70(2)) are operating normally in an environment where a PLC controls the controlled devices (30, 35, 40, 45, 60, 70(1), 70(2)), and an input data acquisition unit (210, 210a) that acquires, as input data, information including one or more monitoring images of the controlled devices (30, 35, 40, 45, 60, 70(1), 70(2)) and / or sound information emitted by the controlled devices (30, 70(1), 70(2)). , 40, 45, 60, 70(1), 70(2)) as label data, and learning units (230, 230a) that perform supervised learning using the input data acquired by the input data acquisition units (210, 210a) and the label data acquired by the label acquisition unit (220), input information including monitoring images or sound information of controlled devices (30, 35, 40, 45, 60, 70(1), 70(2)) controlled by the PLC, and generate learned models (141, 141a) that output the states of the input and output signals. (Appendix 2) The control equipment abnormality analysis device (10, 10A) includes an input unit (110, 110a) that receives a trained model (141, 141a) generated by the machine learning device (20, 20A) of Supplementary Note 1, information including one or more monitoring images related to controlled equipment (30, 35, 40, 45, 60, 70(1), 70(2)) controlled by a PLC and / or sound information emitted by the controlled equipment (30, 70(1), 70(2)), and the state of an input / output signal at the time the information is acquired; and an input unit (110, 110a) that receives a trained model (141, 141a) generated by the machine learning device (20, 20A) of Supplementary Note 1. The control system includes an abnormality determination unit (120, 120a) that inputs information received by the trained model (141, 141a) and compares the state of the input / output signal output from the trained model (141, 141a) with the state of the input / output signal at the time the information is acquired, and determines whether or not the controlled device (30, 35, 40, 45, 60, 70(1), 70(2)) is abnormal based on the comparison result, and an abnormality notification unit (130) that notifies the determination result of the abnormality determination unit (120, 120a).(Supplementary Note 3) The control equipment abnormality analysis device (10, 10A) of Supplementary Note 2 includes the machine learning device (20, 20A) of Supplementary Note 1.

[0067] 1 Abnormality monitoring system 10, 10A Control equipment abnormality analysis device 110, 110a Input unit 120, 120a Abnormality judgment unit 130 Abnormality notification unit 140 Memory unit 141, 141a Trained model 20, 20A Machine learning device 210, 210a Input data acquisition unit 220 Label acquisition unit 230, 230a Learning unit 240 Memory unit 30 Machine tool 35 Operation panel 40 Safety fence 41 Door 45 Opening / closing sensor 50 Surveillance camera 60 LED 70 (1), 70 (2) Fan 80 Operation panel 90 Microphone

Claims

1. A machine learning device comprising: an input data acquisition unit that acquires, in an environment in which a PLC controls a controlled device, information including one or more monitoring images of the controlled device and / or sound information emitted by the controlled device when the controlled device is operating normally, as input data; a label acquisition unit that acquires, as label data, the state of input / output signals related to the controlled device at the time the information is acquired; and a learning unit that performs supervised learning using the input data acquired by the input data acquisition unit and the label data acquired by the label acquisition unit, inputs information including monitoring images or sound information of the controlled device controlled by the PLC, and generates a trained model that outputs the state of the input / output signals.

2. A control equipment abnormality analysis device comprising: a trained model generated by the machine learning device according to claim 1; an input unit that receives information including one or more monitoring images of controlled equipment controlled by a PLC and / or sound information emitted by the controlled equipment, and the state of input / output signals at the time the information was acquired; an abnormality judgment unit that inputs the information received by the input unit into the trained model and compares the state of the input / output signals output from the trained model with the state of the input / output signals at the time the information was acquired, and judges whether the controlled equipment is abnormal based on the comparison result; and an abnormality notification unit that notifies the judgment result of the abnormality judgment unit.

3. A control equipment abnormality analysis device according to claim 2, comprising the machine learning device according to claim 1.

Images