Judgment system, judgment method, and computer program

Abstract

To achieve more accurate anomaly detection. [Solution] A determination system comprising: an image acquisition unit that acquires multiple images obtained by imaging the equipment to be monitored using multiple different imaging methods; a trained model storage unit that stores a trained model obtained by performing unsupervised learning on the equipment to be monitored using the multiple images; and a determination unit that determines whether or not an abnormality has occurred in the equipment to be monitored using the trained model and the multiple images newly obtained by the image acquisition unit.

Description

Technical Field

[0001] The present invention relates to a determination system, a determination method, and a computer program.

Background Art

[0002] In recent years, technologies for imaging equipment to be monitored and performing anomaly detection using unsupervised learning have been proposed. For example, Patent Document 1 discloses a technique for performing anomaly detection using unsupervised learning on images obtained by imaging the same monitoring target with a plurality of cameras in order to improve the detection accuracy.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, anomaly detection may not be achieved with sufficient accuracy. The present invention has been made in view of the above circumstances, and provides a technology that enables anomaly detection with higher accuracy.

Means for Solving the Problems

[0005] One aspect of the present invention is a determination system including an image acquisition unit that acquires a plurality of images obtained by imaging equipment to be monitored by a plurality of different imaging methods, a learned model storage unit that stores a learned model obtained by performing unsupervised learning on the equipment to be monitored using the plurality of images, a determination unit that determines whether an anomaly has occurred in the equipment to be monitored using the learned model and the plurality of newly obtained images by the image acquisition unit.

[0006] One aspect of the present invention is a determination method comprising: an image acquisition step of acquiring a plurality of images obtained by imaging a monitored facility using a plurality of different imaging methods; a determination step of determining whether or not an abnormality has occurred in the monitored facility using a trained model obtained by performing unsupervised learning on the monitored facility using the plurality of images, and a plurality of images newly obtained in the image acquisition step.

[0007] One aspect of the present invention is a computer program for causing a computer to function as a determination system comprising: an image acquisition unit that acquires a plurality of images obtained by imaging a monitored facility using a plurality of different imaging methods; a trained model storage unit that stores a trained model obtained by performing unsupervised learning on the monitored facility using the plurality of images; and a determination unit that determines whether or not an abnormality has occurred in the monitored facility using the trained model and the plurality of images newly obtained by the image acquisition unit. [Effects of the Invention]

[0008] This invention makes it possible to achieve anomaly detection with higher accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic block diagram showing the system configuration of the determination system 100 of the present invention. [Figure 2] This figure shows a specific example of the installation of the imaging device 10. [Figure 3] This is a schematic block diagram showing a specific example of the functional configuration of the learning device 20. [Figure 4] This is a diagram showing one specific example of image data. [Figure 5] This figure shows a specific example of image data containing feature information (attribute information) generated by preprocessing. [Figure 6] This is a diagram showing one specific example of image data. [Figure 7]This figure shows the results of detecting attribute information of objects present in the image data. [Figure 8] This flowchart shows a specific example of the processing performed by the learning device 20. [Figure 9] This is a schematic block diagram showing a specific example of the functional configuration of the determination device 30. [Figure 10] This figure shows a specific example of image data from monitored equipment 1 where an abnormality has occurred. [Figure 11] This flowchart shows a specific example of the processing performed by the determination device 30. [Figure 12] This figure shows a schematic example of the hardware configuration of the information processing device 90 applied to this embodiment. [Figure 13] This figure shows a modified example of the determination device 30. [Modes for carrying out the invention]

[0010] Figure 1 is a schematic block diagram showing the system configuration of the determination system 100 of the present invention. The determination system 100 is used to determine abnormalities occurring in the monitored equipment 1 based on multiple images obtained by capturing the monitored equipment 1 with multiple imaging devices 10.

[0011] The monitored equipment 1 can be any type of equipment. For example, the monitored equipment 1 may be equipment that manufactures products, or equipment that packs products into containers (bags, boxes, or containers). The monitored equipment 1 can be any type of equipment as long as any abnormalities that may occur in that equipment can be reflected as changes in pixel values ​​in the image captured by the imaging device 10 described later.

[0012] The judgment system 100 includes a plurality of imaging devices 10, a learning device 20, and a judgment device 30. Each imaging device 10 and judgment device 30 is connected to each other via a network 70. The learning device 20 and the judgment device 30 may also be connected to each other via the network 70. The network 70 may be a wireless communication network or a wired communication network. The network 70 may be configured using, for example, the Internet or a local area network (LAN). The network 70 may be configured by combining multiple networks.

[0013] The imaging device 10 is composed of devices for capturing still images and devices for capturing moving images. The imaging device 10 captures the equipment to be monitored 1. Multiple imaging devices 10 may be installed, for example, to capture the same equipment to be monitored 1 from different angles. The multiple imaging devices 10 may include devices with different imaging methods. For example, some or all of the following may be used: an imaging device that captures using visible light, an imaging device that captures using electromagnetic waves of different wavelengths than visible light (e.g., near-infrared, far-infrared), an imaging device that acquires polarization direction and polarization degree for each pixel (polarization camera), an imaging device that acquires spectral information for each wavelength for each pixel (hyperspectral camera), an imaging device that captures using sound waves (e.g., acoustic camera or ultrasonic camera), a distance measuring camera that acquires three-dimensional spatial information (e.g., ToF camera or stereo camera, etc.), and an imaging device that captures using an event sensor that reacts to changes in brightness (event camera). The devices with different imaging methods may be installed to capture the equipment to be monitored 1 from viewpoint positions close to each other (including coaxial), or they may be installed to capture the equipment to be monitored 1 from different positions. The imaging device 10 may be configured using an imaging device with a changeable field of view (for example, a PTZ camera: Pan-Tilt-Zoom camera).

[0014] FIG. 2 is a diagram showing a specific example of the installation of the imaging device 10. In FIG. 2, a flexible container filling facility is shown as a specific example of the monitoring target facility 1. In the flexible container filling facility, an operation of filling a soft and deformable bag with a powder or granular object is performed. A plurality of imaging devices 10 are installed at a plurality of positions (from a plurality of angles) so as to image one flexible container filling facility. Among the imaging devices 10 that image the same flexible container filling facility, some imaging devices may be configured using a first imaging method (for example, a visible light imaging device), and other imaging devices may be configured using a second imaging method (for example, a near-infrared imaging device). The imaging device 10 transmits an image obtained by imaging to the determination device 30.

[0015] For example, an event camera may be applied in the first imaging method, and another camera (for example, a high-resolution visible light camera, etc.) may be applied in the second imaging method. Since the event camera can operate at high speed and with low power consumption, it is possible to always monitor with the event camera and intermittently execute imaging with a high cost (large data volume, high power consumption, high processing load) by other cameras only when a specific event occurs. As a result, an inspection system with excellent efficiency (low cost and high performance) can be constructed. More specifically, by using the event camera as an external trigger device for the second imaging method, it is not necessary to extract a trigger signal from the flexible container filling facility. Therefore, it is possible to suppress the overall power consumption. In addition, since the physical vibration frequency of the flexible container filling facility can be visualized from the data of the event camera, there is also a possibility that abnormal prediction can be achieved.

[0016] For example, a visible light camera may be applied in the first imaging method, and an acoustic camera may be applied in the second imaging method. By superimposing images obtained from different physical phenomena of light and sound in the same imaging space, it becomes possible to perform more highly functional (faster, wider, and more detailed) abnormality detection. For example, it becomes possible to realize abnormality detection and abnormal prediction by abnormal sounds.

[0017] Figure 3 is a schematic block diagram showing a specific example of the functional configuration of the learning device 20. The learning device 20 is configured using information processing equipment such as a personal computer or a server device. The learning device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0018] The communication unit 21 is a communication device. The communication unit 21 may be configured, for example, as a network interface. The communication unit 21 communicates data with other devices via the network 70 in accordance with the control of the control unit 23. The communication unit 21 may be a wireless communication device or a wired communication device.

[0019] The storage unit 22 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as, for example, an image data storage unit 221, a pre-processed data storage unit 222, and a trained model storage unit 223.

[0020] The image data storage unit 221 stores multiple images used in the learning process performed in the learning device 20. The image data stored in the image data storage unit 221 is an image obtained by imaging the monitored equipment 1 with the imaging device 10. The image data stored in the image data storage unit 221 may also be an image obtained by imaging equipment of the same type as the monitored equipment 1. The images stored in the image data storage unit 221 are images obtained by imaging the monitored equipment 1 when no abnormalities are occurring. It is desirable that the majority of the images stored in the image data storage unit 221 are images obtained by imaging the monitored equipment 1 when no abnormalities are occurring. The extent of "majority" in this case will also differ depending on the learning algorithm applied to the learning device 20. It is necessary that a sufficient proportion of the images to determine whether an abnormality exists in the monitored equipment 1 using the trained model are images of the monitored equipment 1 when no abnormalities are occurring (normal).

[0021] The preprocessed data storage unit 222 stores preprocessed data. Preprocessed data is data that includes information obtained by performing preprocessing on image data stored in the image data storage unit 221. For example, preprocessed data may be image data obtained by performing predetermined image processing on the image data. Specific examples of such image processing include processing to mitigate the effects caused by changes in ambient light (e.g., sunlight or light from lighting fixtures). For example, median filtering or shading correction may be performed as preprocessing. For example, binarization may be performed as preprocessing. For example, a combination of object detection processing and processing to display each detected object in a different color or pattern may be performed as preprocessing.

[0022] For example, preprocessed data may be data that represents feature information obtained from image data. Specific examples of such feature information include values ​​indicating the shape of each object recognized by object detection processing, and values ​​indicating the size of each object. Other specific examples of feature information include values ​​obtained by performing blob analysis on image data (e.g., the number of blobs, the area of ​​the blobs, the shape of the blobs, the position of the blobs). Still other specific examples of feature information include values ​​obtained by taking the difference between multiple images obtained at predetermined time differences (e.g., the number of pixels whose pixel value difference is greater than or equal to a predetermined threshold, the size of the region formed by such pixels, the area, etc.). Such feature information may be obtained from images before the image processing described above is performed, or from images after the image processing is performed. Such feature information may be obtained from images obtained using multiple different imaging methods. The type of feature information may differ depending on the imaging method.

[0023] The trained model storage unit 223 stores the trained model obtained by a training process using the preprocessed data stored in the preprocessed data storage unit 222.

[0024] The control unit 23 is composed of a processor such as a CPU and memory. The control unit 23 functions as an information control unit 231, a preprocessing control unit 232, and a learning control unit 233 when the processor executes a program. Note that all or part of each function of the control unit 23 may be implemented using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, optical disks, magnetic disks (e.g., HDDs), semiconductor memory devices (e.g., USB memory, SSDs), and storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may be transmitted via a telecommunications line.

[0025] The information control unit 231 controls the input and output of information. For example, the information control unit 231 acquires image data from other devices (information processing devices or storage media) and records it in the image data storage unit 221. For example, the information control unit 231 transmits the trained model stored in the trained model storage unit 223 to another device (for example, the judgment device 30).

[0026] The preprocessing control unit 232 generates preprocessed data by performing predetermined preprocessing on the image data. Specific examples of the preprocessing performed by the preprocessing control unit 232 are as described above.

[0027] The learning control unit 233 performs unsupervised learning using the preprocessed data stored in the preprocessed data storage unit 222. The learning control unit 233 generates a trained model for determining the presence or absence of anomalies based on input image data, for example, by performing unsupervised learning. The learning control unit 233 records the generated trained model in the trained model storage unit 223. The trained model obtained by the learning control unit 233 may be transmitted to the determination device 30 and recorded in the determination model storage unit 321 of the determination device 30.

[0028] Figure 4 shows one specific example of image data. The image data shown in Figure 4 is image data captured when the flexible container filling equipment is designated as the monitored equipment 1, as shown in Figure 2. The image data shown in Figure 4 shows two arms 801 and a bag 802.

[0029] Figure 5 shows a specific example of image data containing feature information (attribute information) generated by preprocessing. In preprocessing, attribute information of objects present in the image data may be detected. The attribute information may include the location of the object and the attribute name of that object. The attribute name may be, for example, the name of the object's type or a model number. For example, attribute information may be obtained by dividing the image into regions for each object using image segmentation techniques (e.g., SAM (Segment Anything Model)) and recognizing each object, or the location and attribute information of objects in the image may be obtained using other algorithms. In the example in Figure 5, each of the two arms 801 and the bag 802 are detected. Thus, the preprocessed data may further contain the attribute information of each object detected in the image as feature information.

[0030] Figure 6 shows one specific example of image data. The image data shown in Figure 6 was captured while the powder was being filled into bag 802, following the situation shown in the image data of Figure 4. As a result, the bottom of bag 802 is bulging and its width is increasing. By using image data captured at various timings when the monitored equipment 1 is operating normally in this way for the learning process, it becomes possible to detect abnormalities with higher accuracy. For example, in the example in Figure 6, the image was taken while bag 802 was being filled, and the bottom of bag 802 is bulging and its width is increasing. Figure 7 shows the results of detecting attribute information of objects present in the image data, using the image in Figure 6 as the processing target.

[0031] Figure 8 is a flowchart illustrating a specific example of the processing performed by the learning device 20. First, the information control unit 231 acquires image data to be used for the learning process (step S101). The image data may be, for example, image data obtained by actually imaging the monitored equipment 1 that is the target of processing by the determination device 30, or image data obtained by imaging a device similar to the monitored equipment 1 (for example, a device of the same model). The image data may be acquired by communication from other information devices, or from a recording medium connected to the learning device 20. The preprocessing control unit 232 performs predetermined preprocessing on the image data (step S102). The learning control unit 233 performs the learning process using the preprocessed data and records the learned model in the learned model storage unit 223 (step S103).

[0032] Figure 9 is a schematic block diagram showing a specific example of the functional configuration of the determination device 30. The determination device 30 is configured using information processing equipment such as a smartphone, tablet, personal computer, or server device. The determination device 30 includes a communication unit 31, a storage unit 32, a control unit 33, and an output unit 34.

[0033] The communication unit 31 is a communication device. The communication unit 31 may be configured, for example, as a network interface. The communication unit 31 communicates data with other devices via the network 70 in accordance with the control of the control unit 33. The communication unit 31 may be a device that performs wireless communication or a device that performs wired communication.

[0034] The storage unit 32 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 32 stores data used by the control unit 33. The storage unit 32 may also function as, for example, a decision model storage unit 321.

[0035] The judgment model storage unit 321 stores the judgment model used by the judgment unit 333 when performing judgment processing. The judgment model is constructed using information from a pre-trained model generated by a learning process. Such a learning process is performed by the learning device 20 described above.

[0036] The control unit 33 is configured using a processor such as a CPU and memory. The control unit 33 functions as an information control unit 331, a pre-processing control unit 332, and a determination unit 333 when the processor executes a program. Note that all or part of the functions of the control unit 33 may be implemented using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, optical disks, magnetic disks (e.g., HDDs), semiconductor memory devices (e.g., USB memory, SSDs), and memory devices such as hard disks and semiconductor memory devices built into computer systems. The above program may be transmitted via a telecommunications line.

[0037] The information control unit 331 acquires image data from multiple imaging devices 10 that are imaging the equipment to be monitored 1. The information control unit 331 outputs information indicating the determination result obtained by the determination unit 333 via the output unit 34. Such information exchange between the information control unit 331 and other devices may be performed, for example, by communication via the communication unit 31.

[0038] The preprocessing control unit 332 obtains preprocessed data by performing predetermined preprocessing on the image data acquired by the information control unit 331. The preprocessing performed by the preprocessing control unit 332 is the same as the preprocessing performed when generating the trained model used by the determination unit 333. In other words, the preprocessing performed by the preprocessing control unit 332 on the image data is the same as the processing performed by the preprocessing control unit 232 of the learning device 20 on the image data. The preprocessing control unit 332 obtains preprocessed data by performing such preprocessing.

[0039] The determination unit 333 performs a determination process using the determination model stored in the determination model storage unit 321 and the pre-processed data. The determination process determines whether or not an abnormality has occurred in the monitored equipment 1 from which image data has been obtained. Figure 10 shows a specific example of image data of the monitored equipment 1 in which an abnormality has occurred. In the case shown in Figure 10, the bag 802 has come off the right arm 801. Such image data is image data that was captured in a state that was not used when the learning process was performed in advance. Therefore, by making a determination using the determination model, it can be determined that an abnormality has occurred. Note that the types of abnormalities that can occur in the monitored equipment 1 are not limited to those shown in Figure 10, and various types of abnormalities can occur. The determination unit 333 is capable of determining that many of these types of abnormalities are abnormal.

[0040] The output unit 34 outputs information in a format that the user can recognize. The output unit 34 may be an image display device such as a liquid crystal display or an organic EL (electroluminescence) display. The output unit 34 may also be an interface for connecting the image display device to the judgment device 30. In this case, the output unit 34 generates a video signal for displaying image data and outputs the video signal to the image display device connected to it. The output unit 34 may also be a device that outputs sound, such as a speaker. The output unit 34 may also be an interface for connecting an audio output device such as a speaker or headphones to the judgment device 30. In this case, the output unit 34 generates an audio signal for playing audio data and outputs the audio signal to the audio output device connected to it. The output unit 34 may also be configured as a touch panel integrated with the input device.

[0041] Figure 11 is a flowchart illustrating a specific example of the processing performed by the determination device 30. First, the information control unit 331 acquires image data of the equipment to be monitored 1 from multiple imaging devices 10 (step S201). The preprocessing control unit 332 acquires preprocessed data by performing preprocessing on the image data (step S202). The determination unit 333 performs determination processing using at least the preprocessed data (step S203). The determination unit 333 outputs information indicating the determination result from the output unit 34 (step S204).

[0042] In the judgment system 100 configured in this way, by using image data obtained by imaging the same monitored equipment 1 using multiple types of imaging methods, it becomes possible to determine with greater accuracy whether or not an abnormality has occurred in the monitored equipment 1. Images obtained by visible light cameras and infrared cameras (far-infrared cameras, near-infrared cameras) are close to the actual shape of the object, and therefore have a correlation with abnormalities that appear in shape (for example, the abnormality shown in the example in Figure 10) and abnormalities that appear in wavelength information (color in the case of visible light).

[0043] Furthermore, acoustic and ultrasonic cameras produce images with different pixels in the locations where acoustic and ultrasonic sounds occur compared to other locations where they do not. Generally, in monitored equipment 1 that repeatedly performs normal operations, the location, loudness, and frequency range of sound fall within a certain range, and it is unlikely that sound will occur from places where it does not normally occur, or that sound will occur at a different loudness or frequency range than usual. Therefore, images captured by acoustic and ultrasonic cameras have a correlation with anomalies accompanied by sound. For example, in the example in Figure 10, there is a high possibility that sound will occur from locations where it does not normally occur, or that sound will occur at a different loudness or frequency range than usual, such as when the bag 802 detaches from the arm 801, or when the arm 801 fails to catch the bag 802. Therefore, anomalies are determined based on these values.

[0044] Furthermore, a distance measuring camera obtains an image in which the length of the distance from the camera's viewpoint to the object being imaged is represented by the pixel value. Generally, in monitored equipment 1 that repeatedly performs normal operation, deviations are unlikely to occur in the distance between the camera's viewpoint and each part of monitored equipment 1. Therefore, the distance image has a correlation with abnormalities that appear in position and shape. For example, in the example in Figure 10, although the distance to the part where the bag 802 is located is the normal value, in the part that has come off the arm 801, the distance obtained is not to the bag 802 but to equipment or a wall located beyond it. Therefore, abnormalities are determined based on such values.

[0045] Furthermore, event cameras produce images with different pixel patterns in locations where changes have occurred and locations where no changes have occurred, compared to the time before and after the event. Generally, in monitored equipment 1 that repeatedly performs normal operation, the location, shape, and time pattern (frequency spectrum) of changes fall within a certain range of region, shape, and value. Therefore, images captured by event cameras correlate with anomalies that involve time changes in the state of the monitored equipment and its surroundings (disturbances involving movement, shape changes, and brightness changes). For example, in the example in Figure 10, there is a high probability that changes will occur in locations that do not normally occur, such as when the bag 802 detaches from the arm 801 or when the arm 801 fails to catch the bag 802. Therefore, anomalies are determined based on these values.

[0046] In this way, by performing unsupervised learning and making judgments using multiple types of image data that have different properties but are correlated with anomalies, it becomes possible to detect the occurrence of anomalies with greater accuracy.

[0047] Figure 12 is a schematic diagram of an example hardware configuration of an information processing device 90 applied to this embodiment. The information processing device 90 comprises a processor 91, main memory 92, communication interface 93, auxiliary storage device 94, input / output interface 95, and internal bus 96. The processor 91, main memory 92, communication interface 93, auxiliary storage device 94, and input / output interface 95 are connected to each other via the internal bus 96 so as to be able to communicate with each other. The information processing device 90 may be applied to, for example, a learning device 20 and a determination device 30. In this case, for example, the communication unit 21 and the communication unit 31 may be configured using the communication interface 93. For example, the storage unit 22 and the storage unit 32 may be configured using the auxiliary storage device 94. Furthermore, the control unit 23 and the control unit 33 may be configured using the processor 91 and the main memory 92.

[0048] (modified version) In this embodiment, the learning device 20 and the determination device 30 are configured as separate devices, but they may be configured as a single integrated device. Figure 13 shows a modified example of the determination device 30 configured in this way. The storage unit 32 of the determination device 30 shown in Figure 13 also functions as an image data storage unit 322 and a pre-processed data storage unit 323. The control unit 33 of the determination device 30 shown in Figure 13 also functions as a pre-processing control unit 332 and a learning control unit 334. The image data storage unit 322 and the pre-processed data storage unit 323 function similarly to the image data storage unit 221 and the pre-processed data storage unit 222 of the learning device 20, respectively. The pre-processing control unit 332 performs not only pre-processing of the determination device 30 (pre-processing of the image data to be determined) but also pre-processing of the learning device 20 (pre-processing of the image data used for learning). The learning control unit 334 functions similarly to the learning control unit 233 of the learning device 20.

[0049] The learning device 20 may be implemented using multiple information processing devices. For example, the learning device 20 may be implemented using a cloud or other device. For example, in the learning device 20, the storage unit 22 and the control unit 23 may be implemented on different information processing devices. For example, the storage unit 22 of the learning device 20 may be distributed and implemented across multiple information processing devices. The determination device 30 may be implemented using multiple information processing devices. For example, the determination device 30 may be implemented using a cloud or other device. For example, in the determination device 30, the storage unit 32 and the control unit 33 may be implemented on different information processing devices. For example, the storage unit 32 of the determination device 30 may be distributed and implemented across multiple information processing devices.

[0050] Although the learning device 20 and the determination device 30 have been described as performing learning and determination processing using pre-processed data obtained by performing pre-processing, learning and determination processing may also be performed using image data without performing pre-processing. In this case, the learning device 20 and the determination device 30 may be configured without a pre-processed data storage unit 222, a pre-processing control unit 232, and a pre-processing control unit 332, respectively.

[0051] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of symbols]

[0052] 100…Determination system, 10…Imaging device, 20…Learning device, 21…Communication unit, 22…Storage unit, 221…Image data storage unit, 222…Preprocessed data storage unit, 223…Trained model storage unit, 23…Control unit, 231…Information control unit, 232…Preprocessing control unit, 233…Learning control unit, 30…Determination device, 31…Communication unit, 32…Storage unit, 321…Determination model storage unit, 33…Control unit, 331…Information control unit, 332…Preprocessing control unit, 333…Determination unit, 34…Output unit

Claims

1. An image acquisition unit that acquires multiple images obtained by imaging the equipment to be monitored using multiple different imaging methods, A trained model storage unit stores a trained model obtained by performing unsupervised learning on the equipment to be monitored using the multiple images, A determination unit that determines whether or not an abnormality has occurred in the equipment to be monitored, using the trained model and a plurality of images newly obtained by the image acquisition unit, A judgment system equipped with the following features.

2. An image acquisition step involves acquiring multiple images obtained by imaging the equipment to be monitored using multiple different imaging methods, A determination step that determines whether or not an abnormality has occurred in the monitored equipment using a trained model obtained by performing unsupervised learning on the monitored equipment using the multiple images, and multiple images newly obtained in the image acquisition step. A method for determining the thumbnail.

3. An image acquisition unit that acquires multiple images obtained by imaging the equipment to be monitored using multiple different imaging methods, A trained model storage unit stores a trained model obtained by performing unsupervised learning on the equipment to be monitored using the multiple images, A determination unit that determines whether or not an abnormality has occurred in the equipment to be monitored, using the trained model and a plurality of images newly obtained by the image acquisition unit, A computer program that allows a computer to function as a judgment system equipped with such a system.

Images