Abnormality detection system, abnormality detection method, and program

Abstract

To reduce the occurrence of undesired detection when detecting an abnormality by picking up an image of an object.SOLUTION: An abnormality detection system has: imaging means that picks up an image of an object; a control unit that detects an abnormality in the object from picked-up image data picked up by the imaging means, and performs image processing on the picked-up image data; information processing means that includes a storage unit, acquires the picked-up image data and stores it in the storage unit, and outputs the picked-up image data to the control unit; and a display unit that displays detected abnormality information. The control unit combines selected areas input from an input unit to set a mask area to the picked-up image data read out from the storage unit and displayed on the display unit, and does not detect the abnormality occurring in the object within a range of the mask area set to the picked-up image data.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an anomaly detection system, an anomaly detection method, and a program. [Background technology]

[0002] Conventionally, various techniques have been proposed for detecting abnormalities occurring in factories or facilities (Patent Documents 1 and 2).

[0003] Patent Document 1 discloses a mechanism for detecting anomalies using background subtraction, in which a binary image obtained from a difference image, which is the brightness difference between the monitoring image of the leak monitoring area and a reference image, is divided into multiple sections, a counter is provided for storing the degree of continuous detection for each section, the current degree of continuous detection for a section in which detection occurred in the binary image is calculated by adding 1 to the previous degree of continuous detection stored in the counter corresponding to that section, the current degree of continuous detection for sections in which no detection occurred is set to zero, the current degree of continuous detection for each section is stored in the corresponding counter, and a leak is detected by detecting a section in which the current degree of continuous detection is above a predetermined upper limit.

[0004] Patent Document 2 discloses a system including a solid-liquid separation device that separates water to be treated into solids and liquids to produce a separated product, an imaging device that generates image data, an image analysis system that analyzes an abnormal state of the separated product based on the image data, an edge server, and a cloud server, the cloud server having a web application that displays the analysis results of the abnormal state and the image data on the web application, the edge server having an inference model constructed by machine learning using training data that includes at least the image data, the image data being input into the inference model, and the system infers an abnormal state of the separated product according to the inference model. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 8-122197 [Patent Document 2] Japanese Patent Publication No. 2022-137973 Summary of the Invention [Problem to be solved by the invention]

[0006] The technologies described in Patent Documents 1 and 2 do not consider narrowing down the targets for detecting abnormalities, and do not consider a method for setting targets to be excluded from detection, i.e., a method for a user to set certain targets to be excluded from detection so that they are not detected as abnormal. In particular, the technology described in Patent Document 2 displays an image of the wastewater that is the target of abnormality detection over the entire display screen, making it possible to determine the presence or absence of an abnormality in the entire area of ​​the display screen.

[0007] However, when detecting anomalies in equipment within a plant, images of machines, piping, and the like outside the target area of ​​monitoring that is the object of detection are also captured simultaneously. Therefore, when an anomaly detection system based on differential images recognizes anomalies using a display unit used in the anomaly detection system according to the above-mentioned conventional technology, there is a possibility that an anomaly will be detected outside the area where an anomaly may occur. Therefore, there has been a demand for technology that can reduce the occurrence of unwanted detection by excluding areas where detection is not desired when detecting anomalies related to the operating status or operational status of an object, such as equipment installed in a plant such as a factory or facility, by capturing images of the object.

[0008] The present invention has been made in consideration of such problems, and its purpose is to provide an anomaly detection system, an anomaly detection method, and a program that can reduce the occurrence of undesired detections when detecting anomalies by imaging an object. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the above-mentioned object, one aspect of the present invention provides an anomaly detection system having an imaging means configured to capture images of an object at predetermined time intervals, a control unit configured to detect an abnormality that has occurred in the object based on captured image data generated by the imaging means and to perform predetermined image processing on the captured image data, a memory unit for storing information, an information processing means configured to acquire captured image data of the object captured by the imaging means and store it in the memory unit, and output the captured image data to the control unit, a display unit configured to display information related to the detected abnormality together with the captured image data, and an input unit configured to allow a user to input information, wherein the control unit sets a mask area for the captured image data read from the memory unit and displayed on the display unit by combining a selection area input from the input unit, and sets the range of the set mask area in the captured image data as outside the scope of detection of an abnormality that has occurred in the object.

[0010] In the anomaly detection system according to one aspect of the present invention, in the above invention, the selection area is set to a rectangular shape, and the mask area is made up of a plurality of the selection areas.

[0011] In the anomaly detection system according to one aspect of the present invention, the selection area is configured to be changeable based on information input from the input unit.

[0012] An anomaly detection method according to one aspect of the present invention is an anomaly detection method executed by a control unit in an anomaly detection system having: an imaging means configured to capture images of an object at predetermined time intervals; a control unit configured to detect an abnormality that has occurred in the object based on captured image data generated by the imaging means and configured to perform predetermined image processing on the captured image data; a memory unit for storing information, and configured to acquire captured image data of the object captured by the imaging means and store it in the memory unit, and output the captured image data to the control unit; a display unit configured to display information related to the detected abnormality together with the captured image data; and an input unit configured to allow a user to input information.The control unit sets a mask area by combining a selection area input from the input unit with the captured image data read from the memory unit and displayed on the display unit, and sets the range of the set mask area in the captured image data as outside the scope of detection of an abnormality that has occurred in the object.

[0013] A program according to one aspect of the present invention is an anomaly detection system including: an imaging means configured to be able to capture images of an object at predetermined time intervals; a control unit configured to be able to detect an abnormality that has occurred in the object based on captured image data generated by the imaging means and to be able to perform predetermined image processing on the captured image data; an information processing unit including a storage unit for storing information, the information processing unit being configured to acquire captured image data of the object captured by the imaging means and store the captured image data in the storage unit, and to be able to output the captured image data to the control unit; a display unit configured to display information related to the detected abnormality together with the captured image data; and an input unit configured to allow a user to input information. The program includes: an imaging means configured to be able to capture an image of an object at predetermined time intervals; a control unit configured to be able to detect an abnormality that has occurred in the object based on captured image data generated by the imaging means and to perform predetermined image processing on the captured image data; an information processing unit configured to acquire captured image data of the object captured by the imaging means and store the captured image data in the storage unit, and output the captured image data to the control unit; The image data is read from the memory unit and displayed on the display unit, and a mask area is set by combining the selected area input from the input unit, and the range of the set mask area in the image data is set outside the scope of detection of abnormalities occurring in the object. [Effects of the Invention]

[0014] According to the anomaly detection system, anomaly detection method, and program of the present invention, when detecting an anomaly by capturing an image of an object, it is possible to reduce the occurrence of undesired detection. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram that schematically shows an anomaly detection system to which an information processing device according to an embodiment of the present invention can be applied. [Figure 2] FIG. 2 is a block diagram schematically showing the configuration of the information processing device shown in FIG. [Figure 3] FIG. 3 is a block diagram schematically showing the configuration of the plant control device shown in FIG. [Figure 4] FIG. 4 is a block diagram schematically showing the configuration of the image processing server shown in FIG. [Figure 5] FIG. 5 is a block diagram schematically showing the configuration of the imaging device shown in FIG. [Figure 6] FIG. 6 is a block diagram schematically showing the configuration of the user terminal shown in FIG. [Figure 7] FIG. 7 is a diagram illustrating an example of a display method displayed on the output unit of the information processing device according to one embodiment of the present invention. [Figure 8] FIG. 8 is a diagram illustrating an example of a masking method in the output unit of the information processing device according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings of the embodiment below, the same or corresponding parts are designated by the same reference numerals. Furthermore, the present invention is not limited to the embodiment described below.

[0017] First, before describing the embodiments of the present invention, the problems that the inventors have found and the intensive studies that have been carried out to solve the problems will be described.

[0018] First, the present inventor conducted various studies on technologies for detecting abnormalities occurring in factories or facilities through imaging. The inventor reviewed the technologies described in Patent Documents 1 and 2 and investigated a method for deriving a difference image, which serves as a criterion for determining whether an abnormality exists, by capturing images of the interior of a factory or facility using a predetermined imaging unit such as a camera. The inventor discovered that, among anomaly detection based on pixel areas with large brightness changes (hereinafter referred to as change areas) obtained from a difference image, there is room for further reduction in false detections in areas of plant equipment that do not require detection. For example, even if the technologies described in Patent Documents 1 and 2 pre-set areas that are excluded from anomaly detection (hereinafter referred to as excluded areas), unexpected differences may be detected during plant operation, specifically, due to, for example, the operation of cylinders or the shadows of operating equipment, irregular movement of fixtures, the on / off of lights in a control panel, or the movement of hoses or power cords. Therefore, the inventor conducted extensive research to develop a technology that allows users to arbitrarily edit the excluded areas for anomaly detection.

[0019] Specifically, the above-mentioned Patent Documents 1 and 2 do not disclose any setting method that allows a user to easily configure or edit anomaly detection settings. Furthermore, the technology described in Patent Document 2 includes the entire displayed image, i.e., the entire captured image data that is the detection target, including the wastewater. It is possible to determine the presence or absence of anomalies across the entire image. However, the present inventors have found that when applying the technology described in Patent Document 2 to anomaly detection in plant equipment, images of machinery and piping other than the location where the desired anomaly may occur are often captured. Therefore, the present inventors have discovered a problem: when capturing images and detecting anomalies using an anomaly detection system based on the presence or absence of change areas described in Patent Documents 1 and 2, etc., there is a high possibility that anomalies outside the desired area may be detected due to change areas other than the location where the desired anomaly may occur. Furthermore, according to the present inventors' findings, unexpected differences such as those described above may be detected during plant operation. From this perspective, the present inventors have discovered a need for a technology that allows users to arbitrarily edit the exclusion areas that are excluded from anomaly detection. However, with the conventional techniques described in Patent Documents 1 and 2, it is extremely difficult to accommodate techniques that allow users to arbitrarily edit non-target areas.

[0020] Therefore, the present inventors conducted extensive research based on the problems identified above and devised a method for masking areas on the display screen of a user device used by a user, whereby areas where changes occur steadily during normal operation (i.e., a non-abnormal state) are excluded from the abnormality detection area, so that even if a change area occurs, it is not determined to be an abnormality. The inventors also devised a method for allowing a user to easily check and edit an area on the display screen that is excluded from the abnormality detection area (hereinafter referred to as a mask area) in cases where false detections occur frequently. Furthermore, the inventors devised a method for allowing a user to set a mask area on the display screen of a user device used by a user as a box-shaped area that is enclosed in a predetermined color, such as blue, and a predetermined shape, such as a rectangular area. This allows the mask area to be displayed as a blue box, so that even if a change area occurs, it is not detected as an abnormality within the mask area. Furthermore, the inventors devised a method for setting a mask area so that the user can add, delete, or adjust the size of the mask area as desired. This reduces the occurrence of undesired detections when detecting abnormalities by capturing an image of an object. The embodiment described below was devised by the inventors through the above-mentioned intensive studies.

[0021] (Anomaly detection system) First, an anomaly detection system to which an information processing device according to one embodiment of the present invention can be applied will be described. Fig. 1 is a schematic diagram showing an anomaly detection system 1 to which an information processing device according to this embodiment can be applied. As shown in Fig. 1, the anomaly detection system 1 according to this embodiment includes an information processing device 10, a plant control device 20, an image processing server 30, an imaging device 40, and a user terminal 50, which are capable of communicating with each other via a network 2. In the following description, information is transmitted and received between the respective components via the network 2, but a detailed description of this point will be omitted.

[0022] The network 2 is configured from an internet network, a mobile phone network, etc. The network 2 is, for example, a public communication network such as the internet, and may also include other communication networks such as a wide area network (WAN), a telephone communication network such as a mobile phone, or a wireless communication network such as Wi-Fi.

[0023] (Information processing device) 2 is a block diagram showing a schematic configuration of information processing device 10. As shown in Fig. 2, information processing device 10 has the configuration of a general computer that is capable of communicating via network 2. Information processing device 10, which serves as information processing means, includes control unit 11, storage unit 12, communication unit 13, output unit 14, and input unit 15.

[0024] Specifically, the control unit 11 includes a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array), and a main memory unit such as a RAM (Random Access Memory) or a ROM (Read Only Memory) (none of which are shown).

[0025] The storage unit 12 is configured with a storage medium selected from an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), and removable media. The removable media is, for example, a Universal Serial Bus (USB) memory or a disc storage medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a Blu-ray (registered trademark) Disc (BD). The storage unit 12 can store an operating system (OS), various programs, various tables, various databases, and the like.

[0026] The control unit 11 loads a program stored in the storage unit 12 into a working area of ​​the main storage unit, executes it, and controls each component through the execution of the program, thereby realizing a function that meets a predetermined purpose. The control unit 11 functions as an image processing unit 111 through the execution of the program.

[0027] The storage unit 12 stores an image database 121 in which various types of data are stored in a searchable manner. The image database 121 is, for example, a relational database (RDB). The database (DB) described below is constructed by a database management system (DBMS) program executed by the above-mentioned processor controlling the data stored in the storage unit 12. The image database 121 stores searchable image information including captured images of equipment installed in factories, facilities, etc. When the information processing device 10 communicates with a user terminal 50, it is also possible to associate unique user identification information for identifying the user terminal 50 with user input information input by a user to the user terminal 50 and store the associated information in the image database 121.

[0028] The communication unit 13 is, for example, a LAN (Local Area Network) interface board or a wireless communication circuit for wireless communication. The LAN interface board or the wireless communication circuit is connected to a network 2 such as the Internet, which is a public communication network. The communication unit 13 is connected to the network 2 and communicates with the plant control device 20, the image processing server 30, the imaging device 40, and the user terminal 50. The communication unit 13 receives image identification information and image information unique to each image processing server 30 from each image processing server 30, and transmits various instruction signals and confirmation signals to the image processing server 30.

[0029] Furthermore, the communication unit 13 transmits information to the user terminal 50 owned by the user when the user uses the image processing server 30, and receives user identification information and various information for identifying the user from the user terminal 50. The user identification information includes various information for identifying individual users from each other. The user identification information is, for example, a user ID that can identify each individual user terminal 50, and includes information necessary for accessing the information processing device 10 when transmitting information related to the user terminal 50. When the user terminal 50 transmits predetermined information such as user input information together with the user identification information to the information processing device 10, the information processing device 10 associates the predetermined information with the user identification information and stores it in a searchable state in the image database 121 of the storage unit 12.

[0030] The output unit 14 as an output means is configured to notify the outside of predetermined information by displaying characters, figures, etc. on the screen of a display such as a liquid crystal display or a plasma display, or by outputting sound from a speaker, under the control of the control unit 11. Furthermore, the output unit 14 includes a printer that outputs predetermined information by printing it on printing paper, etc. The various information stored in the memory unit 12 can be confirmed, for example, on a monitor of the output unit 14 installed in a predetermined office, etc.

[0031] The input unit 15 as an input means is configured, for example, with a keyboard or a touch panel keyboard that is incorporated inside the output unit 14 and detects touch operations on the display panel, or a voice input device that enables calls with the outside, etc. The output unit 14 and the input unit 15 may be integrated into one unit to form an input / output unit such as a touch panel display or a speaker / microphone.

[0032] (Plant control device) FIG. 3 is a block diagram showing a schematic configuration of a plant control device 20. As shown in FIG. 3, the plant control device 20 has a configuration capable of communicating via a network 2. The plant control device 20 is configured, for example, as a distributed control system (DCS) made up of at least one computer. The plant control device 20 is installed in a plant such as a factory or facility, and is configured to be able to control equipment installed in the plant. The plant control device 20 includes a plant control unit 21, a memory unit 22, a communication unit 23, and an input / output unit 24. The plant control unit 21, the memory unit 22, and the communication unit 23 have physical configurations similar to those of the control unit 11, the memory unit 12, and the communication unit 13 described above, respectively.

[0033] The storage unit 22 is configured to be able to store various types of information (plant information) supplied from equipment installed in the plant. The plant control unit 21 may further collect various types of information from the imaging device 40 and individual user terminals 50 and store the information in the storage unit 22. The communication unit 23 is connected to the network 2 and is capable of communicating with the information processing device 10, the image processing server 30, the imaging device 40, and the user terminal 50. The input / output unit 24 has a configuration similar to the above-mentioned configuration in which the output unit 14 and the input unit 15 are integrated together, but the input unit and the output unit may be configured separately.

[0034] (Image processing server) Fig. 4 is a block diagram showing a schematic configuration of the image processing server 30. As shown in Fig. 4, the image processing server 30 has a configuration capable of communicating via the network 2. The image processing server 30 is provided in a cloud connected to the network 2. The image processing server 30 is an image processing system server that performs predetermined image processing on image information received via the network 2. Here, the image processing system is a series of system configurations that process, synthesize, and read the characteristics of two-dimensional or three-dimensional images or data.

[0035] The image processing server 30 includes a control unit 31, a storage unit 32, and a communication unit 33. The control unit 31, the storage unit 32, and the communication unit 33 have the same physical configuration as the above-described control unit 11, the storage unit 12, and the communication unit 13, respectively. The control unit 31 of the image processing server 30 includes an image processing unit 311 and a difference processing unit 312. The communication unit 33 is connected to the network 2 and is capable of communicating with the information processing device 10, the plant control device 20, the imaging device 40, and the user terminal 50.

[0036] The storage unit 32 is configured to be able to store image information relating to captured images transmitted from the information processing device 10 or an imaging device 40 (described later). The storage unit 32 is able to store various programs, various tables, various databases, etc. The storage unit 32 of the image processing server 30 stores databases including an image database 321, etc., and programs including an image processing program 322 and a difference processing program 323, etc.

[0037] The control unit 31 loads the program stored in the storage unit 32 into the working area of ​​the main storage unit, executes it, and controls each component unit through the execution of the program, thereby realizing functions that meet a predetermined purpose. The control unit 31 functions as an image processing unit 311 through the execution of an image processing program 322, and as a difference processing unit 312 through the execution of a difference processing program 323.

[0038] The image processing unit 311 of the control unit 31 performs predetermined processing on image information such as captured image data stored in the image database 321. The difference processing unit 312 is configured to be able to perform difference processing, which compares multiple pieces of image data and derives and identifies differences between the compared pieces of image data. As described above, various data stored in the image database 121 is appropriately stored in corresponding databases in the image processing server 30. Specifically, for example, image information including various captured images acquired from the camera group 45 and stored in the image database 121 of the information processing device 10 is transmitted to the image processing server 30 and stored sequentially or appropriately in the corresponding image database 321.

[0039] (imaging device) Fig. 5 is a block diagram showing a schematic configuration of the imaging device 40. As shown in Fig. 5, the imaging device 40 is configured to include a group of cameras that can communicate via a network 2. The imaging device 40 includes a camera control unit 41, a storage unit 42, and a communication unit 43. The camera group 45 is provided in a plant 44. The imaging device 40 captures images of equipment provided in the plant 44 using the camera group 45 and transmits the images to the communication unit 43 of the imaging device 40. Various information may be supplied to the imaging device 40 from the information processing device 10 or the plant control device 20.

[0040] The camera control unit 41 and the storage unit 42 have the same physical configuration as the above-described control unit 11 and storage unit 12, respectively. The communication unit 43 has the same configuration as the above-described communication unit 13, is connected to the network 2, and transmits images captured at predetermined time intervals, such as one-minute intervals, to the information processing device 10 and the plant control device 20 sequentially or as needed. Note that the time intervals and frequency of capturing images are not limited to these, and the number of image data to be acquired can also be set appropriately depending on the time intervals and frequency of capturing images.

[0041] The communication unit 43 transmits and receives captured image information including image data captured by the camera group 45 and at least a part of the above-mentioned image identification information to and from at least the information processing device 10. The communication unit 43 may also communicate with the image processing server 30 or the user terminal 50 as necessary. The communication unit 43 may also transmit and receive captured image information and image identification information between the plant control device 20 and the imaging device 40.

[0042] The plant 44 may be, for example, a facility that separates recyclable resources, incinerates combustible waste, or detoxifies residue, or may be a facility for composting food waste or a sludge treatment facility for a sewage treatment plant. That is, the plant 44 may be, for example, a factory such as an incineration plant or a manufacturing plant, or a facility such as a waste incineration plant, but is not necessarily limited to these facilities.

[0043] Here, conditions applicable to the anomaly detection system 1 according to this embodiment include places where there is very little change in appearance under normal conditions and where it is necessary to detect a sudden change in appearance, places where it is necessary to detect when an animal or person remains in a restricted area, not limited to liquid leakage or falling off of parts, and places where it is necessary to detect when an insect or other insect enters a place where the intrusion of insects or other insects is undesirable.

[0044] The camera group 45 is a group of cameras installed inside these plants 44 to capture images of the equipment inside various facilities and factories, as well as the conditions inside the facilities and factories. The camera group 45 is composed of, for example, approximately 30 cameras serving as imaging means, arranged in various locations. The camera group 45 is composed of, for example, multiple cameras, such as CCD cameras or CMOS cameras, but is not necessarily limited to these. The individual cameras constituting the camera group 45 are preferably installed near pumps, tanks, conveyors, etc. to detect leaks such as liquid leaks and ash leaks, but are not necessarily limited to these. Furthermore, when detecting flames, smoke, or fallen parts in a plant, each camera constituting the camera group 45 is installed near a location where fire, smoke, or fallen parts are expected.

[0045] (user terminal) The user terminal 50, which serves as a user terminal, is operated by a worker who is a user. The user terminal 50 transmits various information, such as user information including user identification information and user input information, to the information processing device 10, for example, by a call using various data and voice via a communication application. The user terminal 50 is configured to be able to receive various information, such as captured image information, information related to changed areas (changed area information), and time-series graphs, from the image processing server 30. FIG. 6 is a block diagram showing a schematic configuration of the user terminal 50.

[0046] 6, user terminal 50 includes electronic control unit 51, memory unit 52, communication unit 53, and input / output unit 54, which are communicatively connected to one another. Electronic control unit 51, memory unit 52, and communication unit 53 have the same physical configurations as control unit 11, memory unit 12, and communication unit 13 described above, respectively.

[0047] The electronic control unit 51 executes an operating system (OS) and various application programs stored in the storage unit 52 to comprehensively control the operations of the storage unit 52, the communication unit 53, and the input / output unit 54. The communication unit 53 transmits and receives various information, such as user input information, to and from an external server such as the information processing device 10 via the network 2.

[0048] The input / output unit 54 has a display unit 541 capable of displaying images. The display unit 541 of the input / output unit 54 serving as output means is configured to be capable of displaying captured images and graphs accompanied by figures on the display screen under control of the electronic control unit 51. The input / output unit 54 serving as input means is configured with a touch panel display that allows touch operation of the display panel and a speaker / microphone that allows input and output by voice. Here, in the user terminal 50, communication with the outside world also includes communication with an operator resident in the information processing device 10 and an artificial intelligence system connected to the network 2. The input / output unit 54 may be configured as a separate entity, similar to the output unit 14 and input unit 15 described above.

[0049] Specifically, the above-mentioned user terminal 50 may be a mobile phone such as a smartphone, a notebook or tablet type information terminal, or a notebook or desktop type personal computer.

[0050] (Information processing method) Next, an anomaly detection method executed by the information processing device 10 of the above-described anomaly detection system 1 will be described.

[0051] First, the information processing device 10 acquires captured images from the imaging device 40 and stores them in the image database 121 of the storage unit 12. The information processing device 10 transmits the acquired captured images to the image processing server 30. Here, each camera constituting the camera group 45 in the imaging device 40 is installed near, for example, a pump, a tank, or a conveyor. The camera group 45 captures images of these objects at intervals of, for example, one minute to generate captured images every minute and transmit the images to the information processing device 10. The information processing device 10 stores the acquired captured images in the image database 121 of the storage unit 12 and transmits them to the image processing server 30.

[0052] In the image processing server 30, the control unit 31 loads the difference processing program 323 and executes difference processing by the difference processing unit 312. That is, the difference processing unit 312 first acquires, as still image data, a series of captured images taken by the camera group 45 over a predetermined period of time, such as the most recent five minutes, from the information processing device 10. Here, if the imaging device 40 captures images of an object at predetermined intervals of one minute, five still image data sets are acquired for five minutes of still image data.

[0053] The differential processor 312 of the image processing server 30 extracts the median value of the pixels in the acquired image data for each pixel coordinate (x, y) of each image data. The differential processor 312 places the median pixel at the corresponding pixel coordinate (x, y). This generates an image (hereinafter, a median image) that aggregates the median pixels. The generated median image is stored in the image database 321 of the storage unit 32. Here, by generating a median image from multiple still image data captured at a predetermined time interval, such as one minute, for example, five still image data captured over a five-minute period, the influence of a momentary brightness change in part of the still image data, such as when a user or other worker passes in front of a camera in the camera group 45 within a factory or facility, can be eliminated from the median image. Using this median image as the image to be monitored can reduce the likelihood of falsely detecting a change in the situation, such as when a user or other worker passes in front of a camera within a facility or facility, as an abnormality, compared to using still image data as is. In contrast, liquid leaks and ash leaks from equipment often occur continuously, and when generating a median image, it is highly likely that the majority of still images will contain images of these liquid leaks and ash leaks.Therefore, by using the median image as a monitoring image, it is possible to detect continuous abnormalities such as liquid leaks and ash leaks.

[0054] Once the median image is generated, the subtraction processing unit 312 compares the generated median image with a still image in a normal state (hereinafter referred to as the normal image). Here, the normal image is typically composed of a median image of image data captured in a normal state, but any median image can also be registered as the normal image. Furthermore, when a normal state is restored after an abnormality has been detected, a median image taken at a predetermined time after the restoration can also be registered as the normal image. Figure 7 is a diagram showing a display example for explaining a display method executed by an information processing method according to an embodiment of the present invention. As shown in Figure 7, the subtraction processing unit 312 compares the normal image with the generated median image, and marks, for example, a rectangular area, a change area, which is a pixel area where the difference in brightness is greater than a predetermined difference in brightness.

[0055] In the example shown in FIG. 7 , on the display screen 100 of the output unit 14 of the information processing device 10 or the display unit 541 of the user terminal 50, which serves as an information processing device, the subtraction processing unit 312 detects an abnormal portion of the median image 102 at the time "YYYY-MM-DD 16:25:00" in the screen area, relative to the median image 101, which is a normal image at the time "YYYY-MM-DD 16:20:00." Note that "YYYY-MM-DD" refers to the "year-month-day" format, which is the date and time of image capture. Images (median images) every 5 minutes, for example, are displayed horizontally on the display unit 541 of the output unit 14 or the user terminal 50. Here, by tapping a predetermined arrow 107a (the left and right circled arrows in FIG. 7 ), the displayed image can be scrolled horizontally to view the image at any desired time.

[0056] The subtraction processing unit 312 generates, for example, rectangular regions in pixel regions where the difference in brightness is greater than or equal to a predetermined difference in brightness, and superimposes markings 104a and 104b on the median image 102. Note that the markings 104a and 104b are not limited to rectangular shapes, and the types of markings 104a and 104b are not limited. Thereafter, five minutes later, at "YYYY-MM-DD 16:30:00," the subtraction processing unit 312 extracts pixel regions where the difference in brightness from the normal image included in the median image 103 is greater than or equal to a predetermined difference in brightness, and generates and superimposes marking 105 as a changed region.

[0057] Here, the difference processing unit 312 generates markings 104a, 104b, and 105 based on the changed regions for the median images 101, 102, and 103 on the display screen 100 to be displayed on the output unit 14 or the display unit 541. The image processing unit 311 of the image processing server 30 superimposes the markings 104a, 104b, and 105 on the median images 101, 102, and 103 displayed on the display unit 541 or the like. A toolbar 106 for adjusting the median images 101 to 103 and the screen is displayed on the screen. The user can zoom in or out on an image by tapping a specific part of the toolbar 106 according to the desired operation.

[0058] Furthermore, if the area of ​​the change region, i.e., the sum of the areas of the change region surrounded by the markings 104a and 104b or the area of ​​the change region surrounded by the marking 105, is equal to or greater than a predetermined value, the subtraction processor 312 determines that an abnormality has occurred at the image capturing location. Information about the image capturing locations of the median images 101, 102, and 103 may be included in the image identification information. The subtraction processor 312 continuously generates the markings 104 and 105 and determines whether an abnormality has occurred. If the subtraction processor 312 determines an abnormality a predetermined number of times, e.g., three consecutive times for a total of 15 minutes, it transmits an alarm signal to the plant control device 20. In this case, the median images 102 and 103 are determined to be abnormal for, e.g., three consecutive images over a 15-minute period. Various time periods can be set for determining the duration of an abnormality before transmitting an alarm signal, based on the predetermined number of times and the time interval between the median images 101 to 103.

[0059] 3, upon receiving an alarm signal, the plant control unit 21 of the plant control device 20 outputs information indicating that an abnormality has occurred from the input / output unit 24. Here, the input / output unit 24 can notify the outside of the occurrence of the abnormality by outputting audio, images, or videos. Upon receiving the alarm from the plant control device 20, the user checks the images captured by the camera group 45 using the user terminal 50 or the like, and performs processing to check the site and eliminate the abnormality as necessary.

[0060] As shown in FIG. 7 , the difference processing unit 312 of the image processing server 30 generates time-series data 107 based on the calculated difference amount at predetermined intervals, for example, every five minutes, and outputs the time-series data 107 to the output unit 14 or the display unit 541. The time interval of the time-series data 107 can be arbitrarily set by the user. The difference amount is calculated as the area of ​​a portion where a change occurs between multiple images. In the example shown in FIG. 7 , the time-series data 107 is output as a predetermined graph, but various display methods can be set, and the difference amount can be output as a line graph 107b. The time-series data 107 is associated with median images 101 to 103 based on time. Thus, by tapping any position on the time-series data 107, the median images 101 to 103 corresponding to the time of the tapped portion are displayed below the time-series data 107. Various arrangements can be set for the median images 101 to 103 and the time-series data, and are not limited to the example shown in FIG. 7 .

[0061] In the example of the time-series data 107 shown in FIG. 7, a preset threshold is indicated by a red line (dotted line in FIG. 7). When the difference amount calculated for the median images 101 to 103 exceeds the threshold for determining an "abnormality," the difference processing unit 312 changes the color of the bar graph of the time-series data 107, for example, from green (normal part: lightly hatched part in FIG. 7) to red (abnormal part: darkly hatched part in FIG. 7). When an operator visits the location where the abnormality occurred and takes measures to correct the abnormality ("taking measures"), the graph corresponding to that time is changed to, for example, yellow (diagonal lines in FIG. 7). This indicates that the "taking measures" period indicates that the operator is taking measures at the location where the abnormality occurred. In this case, a user such as an operator can arbitrarily change the status to "taking measures." Furthermore, when the status is set to "taking measures," the abnormality detection process may be stopped.

[0062] Next, in the median images 101 to 103 output as described above, there may be areas where a steady change occurs during normal operation, which is a state where no abnormality is present. In this case, in this embodiment, masking is performed to exclude the abnormality detection area. Figure 8 is a diagram illustrating an example of a masking method in the output unit of the information processing device according to this embodiment.

[0063] As shown in Fig. 8, the user uses the input unit 15 or the input / output unit 54 to input a range where detection is not desired on a display image 210 on the output unit 14 of the information processing device 10 or the display unit 541 of the user terminal 50, which serves as an information processing device. Here, the user sets an area where abnormality detection is not desired, for example, in a rectangular shape, and inputs a mask area 211. Here, the mask area 211 is set as a rectangular area as a predetermined selection area 211a, or an area overlapping the predetermined selection area 211a is specified as a rectangular shape and set as a predetermined selection area 211b. In addition, the user inputs an area that is likely to be detected as a changed area even when the plant is operating normally, such as a door, as a predetermined selection area 211c.

[0064] Thereafter, the input at least one piece of information is transmitted from the information processing device 10 or the user terminal 50 to the image processing server 30 via the communication units 13, 53. Based on the received information, the difference processing unit 312 of the control unit 31 in the image processing server 30 sets a mask area 211 in which detection of a changed area is not performed for the display image 210, which is the corresponding median image. Here, the difference processing unit 312 sets the mask area 211 in the display image 210 based on pixel coordinates, and excludes the area from being subject to extraction of a changed area. The difference processing unit 312 extracts a changed area from an area in which the mask area 211 is not set, and sets the above-mentioned marking. As a result, the marking is set for the area other than the mask area 211.

[0065] Furthermore, the user can selectively delete a predetermined area of ​​the mask area 211, or change the size or shape of predetermined selection areas 211a, 211b, and 211c by selecting the mask area 211 that has already been set and clicking or tapping on the toolbar 212 displayed on the display screen. In this way, the masking process according to this embodiment is executed.

[0066] According to the embodiment described above, in order to prevent a change area from being determined as an abnormality even when it occurs during normal operating conditions, predetermined selection areas 211a to 211c are selected and input on the display screen, and information on the mask area 211 of these selection areas 211a to 211c is transmitted to the image processing server 30, and a masking process is performed to specify an area where abnormality detection by background subtraction is not performed. This makes it possible to set the range of the mask area 211 so that even if a change area occurs, it will not be detected as an abnormality, thereby reducing the occurrence of undesired abnormality detection.

[0067] Furthermore, the output unit 14 of the information processing device 10 and the display unit 541 of the user terminal 50 display multiple median images of multiple images taken at specific positions within the plant 44, and by extracting change areas that are indicators of the occurrence of an abnormality and superimposing the median images 101 to 103 using markings 104, 105, and simultaneously displaying time series data 107 showing the detected difference amount at regular intervals on the same display screen 100, it is possible to clarify in which part an abnormality has been detected, and the abnormal part can be easily compared with the normal part from the median images 101 to 103 obtained by taking images of the equipment.

[0068] (Recording medium) In the above-described embodiment, a program capable of executing the image display method can be recorded on a computer-readable recording medium for use in a computer or other machine or device (hereinafter, referred to as a computer, etc.). By loading and executing the program from the recording medium into a computer, etc., the computer functions as the information processing device 10 or the user terminal 50. Here, a computer-readable recording medium refers to a non-transitory recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc. Examples of such recording media that are removable from a computer, etc. include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, BDs, DATs, magnetic tapes, and memory cards such as flash memory. Furthermore, examples of recording media that are fixed to a computer, etc. include hard disks and ROMs. Furthermore, SSDs can be used as both recording media that are removable from a computer, etc. and recording media that are fixed to a computer, etc.

[0069] Furthermore, the program executed by the driving assistance device according to an embodiment may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0070] (Other embodiments) Furthermore, in the information processing device 10, the plant control device 20, the image processing server 30, the imaging device 40, and the user terminal 50 according to an embodiment, the above-mentioned "unit" can be read as a "circuit" or the like. For example, the communication unit can be read as a communication circuit.

[0071] Further effects and modifications can be easily derived by those skilled in the art. The broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Therefore, various changes are possible without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents. For example, the numerical values ​​and types of information listed in the above embodiment are merely examples, and different numerical values ​​and types of information may be used as necessary. The present invention is not limited by the description and drawings that form part of the disclosure of the above embodiment of the present invention.

[0072] In the embodiment described above, the information processing device 10, the plant control device 20, and the imaging device 40 are configured as separate entities, but they can also be configured as a single information processing device. Furthermore, it is also possible to configure multiple servers among these as a single server and configure at least one other server as a separate entity. For example, the information processing device 10 and the plant control device 20 may be configured as an integrated cloud server, or the information processing device 10, the plant control device 20, and the imaging device 40 may be configured as an integrated server. [Explanation of symbols]

[0073] 1. Anomaly detection system 2 Network 10. Information processing equipment 11,31 Control unit 12,22,32,42,52 Storage section 13, 23, 33, 43, 53 Communications Department 14 Output section 15 Input section 20 Plant control device 21 Plant Control Section 24 Input / output section 30 Image Processing Server 40 Imaging device 41 Camera control unit 44 Plant 45 cameras 50 User Terminals 51 Electronic control unit 54 Input / output section 100 display screens 101,102,103 Median image 104, 104a, 104b, 105 marking 106,212 toolbars 107 Time Series Data 107a Arrow 107b Line graph 111,311 Image processing unit 121,321 image database 210 display images 211 Mask Area 211a, 211b, 211c Selection area 312 Differential Processing Unit 322 Image Processing Program 323 Differential Processing Program 541 Display section

Claims

1. an imaging means configured to be able to capture images of an object at predetermined time intervals; a control unit configured to be able to detect an abnormality occurring in the object based on the captured image data generated by the imaging means, and to be able to perform predetermined image processing on the captured image data; an information processing means configured to acquire captured image data of the object captured by the imaging means, store the captured image data in the storage means, and output the captured image data to the control means; a display unit configured to be able to display information about the detected abnormality together with the captured image data; an input unit configured to allow a user to input information, The control unit a mask area is set by combining a selection area input from the input unit with the captured image data read from the storage unit and displayed on the display unit; The range of the set mask area in the captured image data is set to be outside the scope of detection of abnormalities occurring in the object. Anomaly detection system.

2. The selected area is set to a rectangular shape, The mask area is composed of a plurality of the selection areas. The anomaly detection system according to claim 1 .

3. The selection area is configured to be changeable based on information input from the input unit. The anomaly detection system according to claim 1 .

4. an imaging means configured to be able to capture images of an object at predetermined time intervals; a control unit configured to be able to detect an abnormality occurring in the object based on the captured image data generated by the imaging means, and to be able to perform predetermined image processing on the captured image data; an information processing means configured to acquire captured image data of the object captured by the imaging means, store the captured image data in the storage means, and output the captured image data to the control means; a display unit configured to be able to display information about the detected abnormality together with the captured image data; an input unit configured to allow a user to input information; and an anomaly detection method executed by a control unit in an anomaly detection system having the input unit, The control unit a mask area is set by combining a selection area input from the input unit with the captured image data read from the storage unit and displayed on the display unit; The range of the set mask area in the captured image data is set to be outside the scope of detection of abnormalities occurring in the object. Anomaly detection methods.

5. an information processing unit configured to acquire image data of the object captured by the imaging unit and store the image data in the storage unit, and to output the image data to the control unit; a display unit configured to display information related to the detected abnormality together with the image data; and an input unit configured to allow a user to input information. A mask area is set by combining a selected area input from the input unit with respect to the captured image data read from the storage unit and displayed on the display unit, and the range of the set mask area in the captured image data is set outside the scope of detection of an abnormality occurring in the object. A program that makes it happen.

Images