Information processing systems, information processing methods, and programs

Abstract

This invention provides a control method and program for an information processing system that reduces false notifications of anomalies. [Solution] In an information processing system in which a camera and an information processing device are connected via an image transfer cable such as USB, Ethernet, or Camera Link, the information processing device includes a determination unit that determines whether there is a change that should be notified based on a comparison between a reference image and an image to be analyzed, a notification unit that notifies the determination result that there is a change that should be notified, and an acquisition unit that acquires a predetermined value related to the suppression of notification. The notification unit reduces the erroneous notification of abnormalities by suppressing the notification of the determination result that there is a change that should be notified based on the predetermined value.

Description

Technical Field

[0001] The present invention relates to an information processing system, a control method for an information processing system, and a program, and more particularly to a technique suitable for use in detecting abnormalities.

Background Art

[0002] Conventionally, there has been an information processing system that notifies an abnormality based on a comparison between a reference image and an analysis target image.

[0003] Patent Document 1 discloses a technique for applying a filter to an image including noise and discriminating an abnormality.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, in an information processing system that notifies an abnormality based on a comparison between a previous video frame (reference image) and a target video frame (analysis target image), it is described that an abnormality is discriminated by applying a filter to an image including noise abnormal pixels and performing differential operation processing or the like.

[0006] However, noise is generated due to malfunctions of devices such as cameras, cable deterioration, electromagnetic waves, high voltages, etc., and the appearance of noise is diverse. Therefore, in many cases, noise cannot be completely removed from an image, and in such cases, an abnormality may be erroneously notified.

[0007] Furthermore, in addition to noise, there are also cases where organisms such as birds and insects or garbage unexpectedly appear in the image, resulting in an incorrect notification of an abnormality.

[0008] Therefore, the objective of the present invention is to provide a mechanism for reducing false notifications of abnormalities. [Means for solving the problem]

[0009] An information processing system comprising: determination means for determining whether there is a change that should be notified based on a comparison between a reference image and an image to be analyzed; notification means for notifying the determination result that there is a change that should be notified; and acquisition means for acquiring a predetermined value related to suppressing the notification, wherein the notification means suppresses the notification of the determination result that there is a change that should be notified based on the predetermined value. [Effects of the Invention]

[0010] According to the present invention, a mechanism can be provided to reduce false notifications of abnormalities. [Brief explanation of the drawing]

[0011] [Figure 1] This is a system configuration diagram of information processing system 100. [Figure 2] This is a hardware block diagram of the information processing device 104. [Figure 3] This is an example of a block diagram showing the software configuration. [Figure 4] (a) A flowchart of the configuration process and (b) an example of a flowchart of the anomaly detection parameter setting process for the monitoring task. [Figure 5] This is an example of the initial display of the settings screen. [Figure 6] This is an example of an overall view of the settings screen. [Figure 7] This is an example of the settings screen (input tab). [Figure 8] This is an example of the settings screen (input tab). [Figure 9] This is an example of the settings screen (analysis tab). [Figure 10] This is an example of the settings screen (notification tab). [Figure 11] This is an example of the settings screen (monitoring processes and monitoring tasks). [Figure 12] This is an example of the display of the settings screen (input tab). [Figure 13] This is an example of the display of the settings screen (input tab). [Figure 14] This is an example of the display of the settings screen (notification tab). [Figure 15] This is an example of the display of the settings screen (notification tab). [Figure 16] This is an example of the display of the settings screen (monitoring task). [Figure 17] This is an example of the display of the adjustment screen for analysis settings. [Figure 18] This is an example of the display of the grid settings screen (grid division). [Figure 19] This is an example of the display of the grid settings screen (grid correction). [Figure 20] This is an example of the display of the analysis settings screen. [Figure 21] This is an example of the flowchart of the abnormality detection process. [Figure 22] This is an example of the display of task registration information. [Figure 23] This is an example of the flowchart of the image difference (abnormality) detection process. [Figure 24] This is an example of the flowchart of each histogram calculation process. [Figure 25] This is an example of the display of grid correction. [Figure 26] This is an example of the display of the notification during grid correction. [Figure 27] This is an example of the flowchart of the abnormality notification process. [Figure 28] This is an example of the display of the settings screen (abnormality notification). [Figure 29] This is an example of the display of the abnormality notification dashboard screen. [Figure 30] This is an example of an analysis target image containing noise. [Figure 31] This is an example of the flowchart of the image difference (abnormality) detection process. [Figure 32] This is an example of the flowchart of the initialization process for each grid. [Figure 33]This is an example of a flowchart for anomaly detection processing for each grid. [Figure 34] This is an example of the display of the analysis settings screen in the second embodiment. [Figure 35] This is an example of an abnormality notification in the second embodiment. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0013] Figure 1 is a system configuration diagram showing an example of the configuration of the information processing system 100 of the present invention.

[0014] Figure 1 shows a configuration in which the camera 102 and the information processing device 104 are connected via an image transfer cable (USB, Ethernet, Camera Link, etc.). However, a configuration of an information processing device equipped with a camera is also acceptable, even without this specific setup.

[0015] The information processing device 104 acquires an image obtained by the camera 102 capturing the target object via an image transfer cable, and performs processing related to the detection of abnormalities in the image.

[0016] Furthermore, the information processing system 100 of this embodiment may also have a configuration in which the camera 102 with the above-described configuration does not exist.

[0017] In that case, processing related to the detection of abnormalities in images is performed on images acquired from other terminals on the network or on images stored in the memory (RAM) of the information processing device 104.

[0018] The following describes the hardware configuration of an information processing device applicable to the information processing device 104 shown in Figure 1, using Figure 2 as an example.

[0019] Figure 2 is a block diagram showing a hardware configuration applicable to the information processing device 104 shown in Figure 1.

[0020] In Figure 2, 201 is the CPU, which comprehensively controls each device and controller connected to the system bus 204. The ROM 202 or external memory 212 stores the BIOS (Basic Input / Output System), which is the control program for the CPU 201, the operating system program (hereinafter referred to as the OS), and various programs necessary to implement the functions executed by each PC, as described later.

[0021] 203 is RAM, which functions as the main memory, work area, etc., of the CPU 201. The CPU 201 loads the necessary programs, etc., from ROM 202 or external memory 212 into RAM 203, and then executes the loaded programs to perform various operations.

[0022] 205 is an input controller that controls input from pointing devices such as the keyboard (KB) 210 and a mouse (not shown).

[0023] 206 is a video controller that controls the display on indicators such as display 211.

[0024] 207 is a memory controller that controls access to external memory 212, such as external storage devices (hard disks (HDs)) that store various types of data, flexible disks (FDs), or CompactFlash® memory connected to a PCMCIA card slot via an adapter.

[0025] 208 is a communication interface controller that controls the reception of image data from the external PC 213 via the network (TCP / IP). 209 is an image I / F controller that controls the reception of image data from camera 102 via an image transfer cable (USB, Ethernet, Camera Link, etc.).

[0026] The various programs described later for realizing the present invention are stored in RAM 203 and executed by CPU 201.

[0027] Furthermore, the image data used when executing the above program is stored in ROM202, external memory212, external PC213, and camera102 depending on the application, and is stored in RAM203 via various controllers when the program is executed.

[0028] Figure 3 is an example of a block diagram showing the software configuration of an embodiment of the present invention. The information processing device 104 includes the following functional units.

[0029] The acquisition unit 301 is a functional unit that acquires images captured at the same angle of view.

[0030] The reception unit 302 is a functional unit that receives specifications for the area to be included in the image and the method of displaying that area.

[0031] The determination unit 303 is a functional unit that determines whether a change has occurred in a given region based on changes between corresponding regions of multiple images acquired by the acquisition unit 301.

[0032] The output unit 304 is a functional unit that outputs the determination result from the determination unit 303.

[0033] The output unit 304 is a functional unit that displays the area specified by the reception unit 302 using the display method specified by the reception unit 302, and controls whether or not to output a judgment result based on the area specified by the reception unit 302.

[0034] The area modification unit 305 is a functional unit that modifies the area specified by the reception unit 302 based on the area specified by the reception unit 302 and the determination result of that area.

[0035] The splitting unit 306 is a functional unit that splits the image acquired by the acquisition unit 301.

[0036] The determination unit 303 is a functional unit that determines whether a change has occurred in a given region based on changes between corresponding regions that have been divided by the division means from among multiple images acquired by the acquisition unit 301.

[0037] The division unit 306 is a functional unit that divides an image using two or more division methods.

[0038] The output unit 304 is a functional unit that outputs a determination result based on the region divided by the first division method when the determination unit 303 determines that there has been a change in the region divided by the second division method.

[0039] The determination unit 303 is a functional unit that determines whether a change has occurred in a given region based on the changes between corresponding regions of multiple images acquired by the acquisition means and multiple types of thresholds.

[0040] The output unit 304 is a functional unit that identifies and outputs which of the multiple types of thresholds was used to determine that a change had occurred.

[0041] The reception unit 302 is a functional unit that accepts specifications for the method of identification and output for each type of threshold.

[0042] The reception unit 302 is a functional unit that controls the system to prevent the same output method specification from being accepted for different threshold types.

[0043] The determination unit 303 is a functional unit that determines whether there is a change that should be notified based on a comparison between the reference image and the image to be analyzed. The acquisition unit 301 is a functional unit that notifies the determination result that there has been a change that should be notified.

[0044] The notification unit 307 is a functional unit that acquires a predetermined value related to the suppression of notifications.

[0045] The notification unit 307 is a functional unit that suppresses the notification of a determination result indicating that a change requiring notification has occurred, based on a predetermined value.

[0046] The notification unit 307 is a functional unit that suppresses the notification of a determination result indicating that a change requiring notification has occurred, based on at least one of the number of changes to be notified and a predetermined value.

[0047] The notification unit 307 is a functional unit that suppresses the notification of the determination result that there has been a change that should be notified when the number of changes that should be notified is less than or equal to a predetermined value.

[0048] This concludes the explanation of Figure 3.

[0049] The setup process shown in Figure 4(a) will be explained below. In S401, the information processing device 104 determines whether a configuration file for the operating process exists in the external memory 212. If it is not registered, the process proceeds to S402; if it is registered, the process proceeds to S403.

[0050] In S402, the information processing device 104 displays the settings screen 510 (Figure 5), and when it receives a press of the add monitoring process button 511 from the user, it registers the operation process in RAM 203 and displays the settings screen 520.

[0051] In S403, when the information processing device 104 receives a press of the button 521 for selecting the operation process to be edited on the setting screen 502, it displays the setting screen 610 (Figure 6).

[0052] The setting acceptance process performed on setting screen 610 in S403 will be explained below using Figures 6 to 10. When the information processing device 104 receives input from the user for the name 611 and description 612 of the operating process, it edits the operating process information in the RAM 203 and displays the settings screen 620.

[0053] When the information processing device 104 receives a press from the user on the settings screen 610, it displays the settings screen 710 (Figure 7).

[0054] When the information processing device 104 receives a selection from the user for "Video file loading (FFmpeg)" (721 in Figure 7) from the list of functions 711, it displays the settings screen 720. At this time, the settings screen 720 edits the operation process information of RAM 203 based on the location 722 and option 723 values ​​of the command entered by the user.

[0055] When the information processing device 104 receives a selection from the user for "Video file loading (OpenCV)" (731 in Figure 7) from the list of functions 711, it displays the settings screen 730 and edits the operation process information of RAM 203.

[0056] When the information processing device 104 receives a selection from the user for "Live video loading (video surveillance server)" (811 in Figure 8) from the list of functions 711, it displays the settings screen 810 (Figure 8). "Live video loading (video surveillance server)" is a function that loads live video captured by a network camera from a video surveillance server (not shown) that can be connected via the network.

[0057] Based on the computer name 812, username 813, and password 814 values ​​entered by the user on the settings screen 810, the operating process information of RAM 203 is edited. Also, if the connection confirmation 815 is pressed, a connection test is performed from the information processing device 104 to the video surveillance server via the communication I / F controller 208 using the information in RAM 203.

[0058] When the information processing device 104 receives a click from the user on the settings screen 610 to select the analysis tab 614 (Figure 6), it displays the settings screen 910 (Figure 9).

[0059] When the information processing device 104 receives a selection of "Detection by video comparison" from the function list 911 from the user, it displays the settings screen 920 and edits the operation process information of RAM 203. At this time, if the license is not authenticated, "No license" is displayed in the license information 922.

[0060] When the information processing device 104 receives a press of license read button 923 from the user, if a valid license has been authenticated, it displays the settings screen 930, displays the license information 931, and edits the operation process information of RAM 203.

[0061] When the information processing device 104 receives a notification from the user on the settings screen 610 by clicking the notification tab 615 (Figure 6), it displays the settings screen 1010 (Figure 10).

[0062] When the information processing device 104 receives a selection of a "custom command" (1021 in Figure 10) from the function list 1011 from the user, it displays the settings screen 1020 and edits the operation process information of the RAM 203.

[0063] A "custom command" is a function that executes commands to instruct batches or applications to run when an anomaly is detected. Using this function, it becomes possible to execute batch processing, instruct email sending to mail clients, instruct automated voice calls to be made, and integrate with other applications or external systems.

[0064] When the information processing device 104 receives a selection from the user for "Analysis Result Notification (Video Surveillance Server)" (1031 in Figure 10) from the function list 1011, it displays the settings screen 1030.

[0065] "Analysis Result Notification (Video Surveillance Server)" is a function that notifies the video surveillance server of the analysis results. This function is used when the video surveillance server is configured to notify users of anomalies, or when the video surveillance server analyzes and manages events such as anomalies.

[0066] Based on the values ​​of address 1032 and port number 1033 entered by the user on the settings screen 1030, the operating process information of RAM203 is edited.

[0067] When the information processing device 104 receives a connection confirmation 1034 from the user, it performs a connection test via the communication I / F controller 208 using the information in the RAM 203.

[0068] Returning to the explanation of Figure 4. In S404, when the information processing device 104 receives a press of the OK button 631 on the setting screen 630 (Figure 6), it saves the operation process information registered in RAM 203 to external memory 212 as an operation process setting file.

[0069] In S405, the information processing device 104 determines whether a configuration file for the monitoring task exists in the external memory 212. If it is not registered, the process proceeds to S406; if it is registered, the process proceeds to S407.

[0070] In S406, the information processing device 104 displays the settings screen 630, and when it receives a press from the user of the Add task button 632 for the monitoring process, it registers the monitoring task in RAM 203 and displays the settings screen 1110 (Figure 11).

[0071] In S407, when the information processing device 104 receives a click on the monitoring task 1111 to be edited on the settings screen 1110, it displays the settings screen 1120.

[0072] The following describes the process for receiving settings for monitoring tasks performed in S407, using Figures 11 to 15. When the information processing device 104 receives input from the user on the settings screen 1120, including a name 1121 and a description 1122, and a change in the toggle button 1123 for enabling this task, it edits the operation process information of the RAM 203 and displays the settings screen 1130.

[0073] When the information processing device 104 receives a press of the input tab 1124 from the user on the settings screen 1120, it displays either the settings screen 1200 (Figure 12) or the settings screen 1300 (Figure 13) depending on the settings of the operation process.

[0074] If the input setting for the operation process is "Load video file (FFmpeg)" or "Load video file (OpenCV)", the settings screen 1200 (Figure 12) will be displayed.

[0075] When the information processing device 104 receives a click on a video folder 1201 and a folder selection from the user, input of a frame acquisition interval 1202, and a change in the toggle button for deleting processed videos 1203, it edits the operation process information of RAM 203 and displays the settings screen 1210.

[0076] If the input setting for the operation process is "live video loading," the settings screen 1300 (Figure 13) is displayed. When the information processing device 104 receives a selection of a list item for camera 1301 from the user, it edits the operation process information in RAM 203 and displays the settings screen 1310.

[0077] When the information processing device 104 receives a notification from the user on the settings screen 1120, it displays either the settings screen 1400 or the settings screen 1500, depending on the settings of the operation process.

[0078] If the notification setting for the operation process is "custom command," the settings screen 1400 (Figure 14) is displayed. When the information processing device 104 receives input of the command path 1401 from the user or a change in the toggle button for asynchronous execution 1402, it edits the operation process information in RAM 203 and displays the settings screen 1410.

[0079] If the notification setting for the operation process is "Analysis Result Notification (Video Surveillance Server)", the settings screen 1500 (Figure 15) is displayed. The information processing device 104 accepts the input of name 1501 and camera ID 1502 from the user and displays screen 1510. At this time, if the test event button 1511 is pressed by the user, a test notification process is executed to check whether the notification with the input content works correctly. At this time, if the setting of the detection frame display toggle button 1512 is enabled, test pseudo detection result information is also added to the notification content.

[0080] Returning to the explanation of Figure 4. In S408, when the information processing device 104 receives a press of the OK button 1602 from the user on the setting screen 1600 (Figure 16), it saves the monitoring task information registered in RAM 203 as a monitoring task setting file in external memory 212 and displays the setting screen 1610.

[0081] In S409, when the user clicks the analysis tab 1611 on the settings screen 1610, the settings screen 1700 (Figure 17) is displayed. At this time, when the user clicks the adjustment button 1701, the analysis settings adjustment screen 1710 is displayed.

[0082] Subsequently, the information processing device 104 executes the process shown in Figure 4(b).

[0083] The process for setting anomaly detection parameters for the monitoring task shown in Figure 4(b) will be explained below. In S421, when the information processing device 104 receives a press of the basic settings button 1711 from the user on the analysis settings adjustment screen 1710 (Figure 17), it displays the basic settings screen 1720 for analysis settings and proceeds to S422. If the Basic Settings button is not pressed, proceed to S428.

[0084] In S422, when the information processing device 104 receives a request from the user to change the slider bar to 1801 rows (Figure 18) and 1802 columns, it updates the display of the grid (dashed lines) on the screen, which is divided by 1801 rows and 1802 columns, as shown in screen 1800 (Figure 18), and saves the settings to RAM 203.

[0085] Specifically, on screen 1800, the number of rows is set to 10 (1801) and the number of columns to 10 (1802), so the image to be analyzed is displayed, divided into a 10x10 grid.

[0086] While a larger number of grids allows for more precise detection, it also increases the number of processes involved, such as the image difference (anomaly) detection process described later (Figure 23), potentially increasing the processing time.

[0087] In S423, when the information processing device 104 receives a press 1811 of any area divided by a grid on the user's screen, a press of the full mask setting button 1812, or a press of the full mask release button 1813, it displays the presence or absence of a mask (notification suppression area) as shown on screen 1810 and saves the setting in RAM 203. In other words, this step is an example of a process that accepts the specification of an area included in an image.

[0088] If there are many areas to mask, you can easily set the areas to be masked by first masking all areas using the "Set All Masks" button (1812), and then specifying the areas to be unmasked.

[0089] If the masked area is small, you can easily set the area to be masked by first removing the mask from the entire area using the "Remove All Mask" button (1813), and then specifying the area to be masked.

[0090] Furthermore, if the user changes the mask color inversion 1821 toggle button at this time, the mask drawing color is changed as shown in screen 1820. Specifically, in screen 1810, the mask area was represented by transparent black, but in screen 1820, the mask area is represented by transparent white. In other words, this step is an example of a process that accepts a specification for how to display an area included in an image. This step is also an example of a process that accepts a specification for the display method of the area, and displays the area that has been specified using the accepted display method.

[0091] This allows the user to select the color of the mask area according to the color tone and brightness of the image being analyzed. This is achieved by representing the mask area with transparent black when the image is bright and transparent white when the image is dark. As a result, the user can easily identify and understand the appearance of the mask area.

[0092] In this embodiment, the color of the mask area is set to "transparent black" or "transparent white," but this method is not limited to this. Other colors may be used, the user may specify a color, patterns such as diagonal lines may be used to add visual variation, a frame may be used, or other methods may be used to distinguish the monitored area from the mask area.

[0093] In S424, when the information processing device 104 receives a change from the user to the grid correction (shifting the grid) toggle button 1901 (Figure 19) or 1911, it updates the display of the frame 1902 (with frame) or 1912 (without frame) that indicates the detection range and saves the setting to RAM 203.

[0094] In this embodiment, grid correction is set for the monitoring process, but this method is not limited to this one. Grid correction may also be set directly for the network camera or video monitoring server that is the source of the image input.

[0095] In S425, when the information processing device 104 receives a numerical input 1921 for the number of reference image frames from the user, it saves the setting in the RAM 203.

[0096] In S426, when the information processing device 104 receives a press of the back button 1803 from the user while the basic settings screen 1800 (Figure 18) for analysis settings is displayed, it determines whether there are any changes to the basic settings. If there are changes, it proceeds to S427. If there are no changes, it proceeds to S428.

[0097] In S427, the information processing device 104 reflects the contents set in RAM 203 into the monitoring task configuration file in external memory 212. The information processing device 104 also detects changes in the monitoring task configuration file and performs reference image discarding during the analysis process. (In S2306, it determines that updating the base information is necessary.) In S428, when the information processing device 104 receives a change in the slider bar for each item of the threshold 2001 on the analysis setting screen 2000 (Figure 20), it saves the setting to the RAM 203.

[0098] In this embodiment, the user sets each threshold, but this method is not the only way. The user may select a dataset for each threshold, the user may set the objects to be monitored or the anomalies to be detected to automatically set the threshold parameters, or the information processing device 104 may determine the objects to be monitored or the anomalies to be detected and automatically set the threshold parameters.

[0099] This makes it easy to set anomaly detection thresholds.

[0100] In S429, when the information processing device 104 receives a press of the apply button 2002 from the user, it reflects the contents set in RAM 203 into the monitoring task configuration file in external memory 212. The information processing device 104 also detects changes in the monitoring task configuration file and modifies the parameter values ​​used for detection processing. (This modifies the threshold used in S2310.)

[0101] The anomaly detection process shown in Figure 21 will be explained below. In S2101, the information processing device 104 executes anomaly detection processing if the external memory 212 contains a configuration file for the monitoring task (task registration information 2201 (Figure 22)) and the task activation setting toggle 2202 is set to enabled.

[0102] In S2102, the information processing device 104 monitors whether there is new input information. If there is new input information, the process proceeds to S2103. If there is no new input information, the process returns to S2101 and repeats.

[0103] In S2103, the information processing device 104 acquires input information according to the configured input method (configured on the configuration screen 1200 or configuration screen 1300).

[0104] Specifically, if the input setting for the operation process (settings screen 1200) is "Load video file (FFmpeg)" or "Load video file (OpenCV)", the video files located in the specified video folder are used as input information and moved to the working area on external memory 212.

[0105] Furthermore, if the input setting for the operation process is "Live video loading (video monitoring server)" (settings screen 1300), the process of acquiring video from the specified video acquisition destination is executed, and if a still image is acquired, the process proceeds to S2106.

[0106] In S2104, if the information processing device 104 performed a video file transfer on the external memory 212 in S2103, it proceeds to S2105. If it has performed a video acquisition process and acquired a still image, it proceeds to S2106.

[0107] In S2105, the information processing device 104 reads the video file moved to the external memory 212 in S2103 using the means set in the input settings of the operation process, divides it into a series of still images for each frame, saves them as files on the external memory 212 or stores them as data on the RAM 203, and proceeds to S2106.

[0108] In S2106, the information processing device 104 executes the process shown in Figure 23.

[0109] The first embodiment will now be described. The image difference (anomaly) detection process shown in Figure 23 will be explained.

[0110] In S2301, if the information processing device 104 has been splitting the video file into a series of still images frame by frame (S2105), it repeats the subsequent processing until processing is completed for all the generated still images.

[0111] In S2302, the information processing device 104 reads the still image to be processed and loads it onto the RAM 203. In other words, this step is an example of a process for acquiring images captured at the same angle of view.

[0112] In S2303, the information processing device 104 calculates the hue, saturation, and lightness information from the image data loaded on the RAM 203.

[0113] In S2304, the information processing device 104 performs each histogram calculation process (Figure 24) to calculate a histogram from the image data.

[0114] Here, we will explain the calculation process for each histogram in Figure 24. In S2401, the information processing device 104 proceeds to S2402 if the grid correction set in S424 is enabled among the settings stored in RAM 203. Otherwise, it proceeds to S2403.

[0115] In S2402, the information processing device 104 includes not only the regions demarcated by the grid 2500 (Figure 25) set in S422, but also regions demarcated by grid 2501, which is shifted horizontally and vertically by half the size of one region, as the target regions for histogram calculation. In other words, this step is an example of a process for dividing the acquired image.

[0116] Specifically, as shown in 2510 and 2520, if an anomaly occurs near the 3x4 grid set in S422, the anomaly spans multiple areas. As a result, the threshold is not exceeded in each area, and all areas are judged as normal, potentially leading to low anomaly detection accuracy.

[0117] As in S2402, by determining anomalies not only in the area demarcated by the normal grid 2500 set in S422, but also in the area demarcated by the corrected grid 2501 (shifted grid), it becomes easier to improve the accuracy of anomaly detection even when an anomaly occurs near the normal grid 2500 set in S422.

[0118] In this embodiment, the grid is shifted horizontally and vertically by half the size of one region, but this method is not limited to this. Two new grids may be created by shifting them horizontally and vertically by "1 / 3" and "2 / 3" of the size of one region, or a hexagonal grid may be created instead of a grid (square). In other words, this step is an example of a process in which an image is divided using two or more division methods. Specifically, this is an example in which a region divided by the first division method partially overlaps with at least one of the regions divided by the second division method.

[0119] This makes it easier to improve the accuracy of detecting anomalies, even if an anomaly occurs near grid 2500.

[0120] In S2403, the information processing device 104 calculates a histogram of luminance information from the luminance information acquired in S2303 for each processing target area set in S422 and S2402.

[0121] In S2404, the information processing device 104 calculates a histogram of hue information from the hue and saturation information acquired in S2303 for each processing target area set in S422 and S2402.

[0122] In S2405, the information processing device 104 calculates a histogram of saturation information from the saturation information acquired in S2303 for each processing target area set in S422 and S2402.

[0123] In S2406, the information processing device 104 calculates edge gradient information from the lightness information acquired in S2303 for each processing area set in S422 and S2402, and uses the edge gradient information to calculate an edge gradient intensity histogram and an edge gradient angle histogram.

[0124] In other words, steps S2304 and Figure 24 show an example of a process that compares a reference image and an image to be analyzed based on the image's features.

[0125] Furthermore, S2304 and Figure 24 show an example of a process that compares a reference image and an image to be analyzed based on information relating to at least one of the following: brightness, hue, saturation, edge gradient intensity, and edge gradient angle.

[0126] Let's return to the explanation of Figure 23.

[0127] In S2305, the information processing device 104 calculates the similarity for each processing area set in S422 and S2402 using a histogram of reference information and a histogram intersection, which is one of the histogram comparison methods.

[0128] In this case, the reference information is the histogram calculated in S2308, described later. If reference histogram data is unavailable, the similarity is set to 100%.

[0129] In this embodiment, a histogram intersection was used as the histogram comparison method, but the method is not limited to this, and other methods such as the Bhattacharya coefficient may be used.

[0130] Furthermore, while this embodiment involves detecting anomalies by comparing histograms, it is not limited to this method. Other methods, such as template matching or feature point detection, may also be used to detect anomalies.

[0131] In S2306, if the number of data already set in the reference image is less than the "number of reference image frames" set in S425, the information processing device 104 determines that the reference image needs to be updated and proceeds to S2307. If data that satisfies the "number of reference image frames" has already been set in the reference image, the process proceeds to S2309.

[0132] Furthermore, the information processing device 104 discards the reference image in the following cases and initializes the number of set data items to 0.

[0133] • When the basic settings are changed during user operation of S427 • When the user clicks the "Recreate Reference Image" button (2003) This allows users to easily instruct the system to recreate the reference image.

[0134] In S2307, the information processing device 104 determines whether the data is suitable as a reference image based on the similarity calculated in S2305. If there are no areas with low similarity, the process proceeds to S2308. If there are many areas with low similarity, the device determines that the data is not suitable as a reference image and terminates the process.

[0135] In S2308, the information processing device 104 adds the histogram calculated in S2304 to the histogram of the reference information currently held, recalculates the average value, updates the histogram of the reference information, and terminates the process.

[0136] In S2309, the information processing device 104 compares the similarity of each processing target area set in S422, calculated in S2305, with the threshold set in S428, and records the areas that fall below the threshold as abnormal areas on the RAM 203. In other words, this step is an example of a process that determines whether there has been a change in a given area based on changes between corresponding areas of multiple acquired images. In other words, this step is an example of a process that determines whether there is a change that should be notified based on a comparison between a reference image and an image to be analyzed.

[0137] Specifically, in threshold 2001 (Figure 20), if the brightness threshold is set to 22%, the region to be processed will be judged as abnormal if the brightness similarity of the region is less than 22%.

[0138] In this embodiment, similarity < threshold is used, but this is not the only option; similarity ≤ threshold may also be used.

[0139] Furthermore, while this embodiment uses similarity, it is not limited to similarity; change amount may also be used. In that case, the region will be judged as abnormal if change amount > threshold or change amount ≥ threshold.

[0140] In other words, this step is to show an example of what constitutes a change that should be notified: a change where the similarity between the reference image and the image being analyzed is less than a predetermined threshold, or the similarity is less than or equal to a predetermined threshold, or the amount of change exceeds a predetermined threshold, or the amount of change is greater than or equal to a predetermined threshold.

[0141] Furthermore, this step compares the reference image and the image to be analyzed in corresponding regions and provides an example of when the similarity between the regions is less than a predetermined threshold, or the similarity between the regions is less than or equal to a predetermined threshold, or the amount of change between the regions exceeds a predetermined threshold, or the amount of change between the regions is greater than or equal to a predetermined threshold.

[0142] In S2310, the information processing device 104 compares the similarity of each region (regions separated by a correction grid) that was calculated in S2305 and set as the region to be processed in S2402 with the threshold set in S428, and detects regions that fall below the threshold.

[0143] If an abnormality is detected at this point, proceed to S2311. If no abnormality is detected, proceed to S2312.

[0144] In S2311, the information processing device 104 determines that the area 2600 (Figure 26) where an anomaly was detected is different from the area to be notified to the user as set in S422. Therefore, it considers all areas 2610 (four areas in the case of Figure 26) that overlap with the detected area 2600 as areas where an anomaly was detected (2620) and records them as anomaly locations on the RAM 203.

[0145] As a result, the area that receives notification of an anomaly is the area demarcated by the normal grid 2500 set in S422, regardless of whether grid correction is performed or not. In other words, this step is an example of a process that outputs a determination result based on the area demarcated by the first division method when it is determined that there has been a change in the area demarcated by the second division method.

[0146] In this way, anomalies can always be notified to users and external applications based on areas divided by a fixed grid. In other words, because the notification method is consistent regardless of whether grid correction is applied or not, it is easy for users to understand the notifications, and for external applications, it has the advantage of not requiring complex methods for receiving anomaly notifications, as the receiving logic and output method for anomalies can be kept consistent.

[0147] In S2312, the information processing device 104 compares the abnormal area recorded on the RAM with the mask area set in S423. If there is an abnormal area other than the mask area, the process proceeds to S2313. If there is no abnormal area, or if all of the abnormal areas are mask areas, the process terminates. In other words, this step is an example of a process that controls whether or not to output a judgment result based on the area specified.

[0148] In this embodiment, the histogram calculation process (S2304) is performed even in the masked region, but this method is not limited to this, and the histogram calculation process (S2304) may not be performed in the masked region. By doing so, the number of times the histogram calculation process is performed is reduced when there is a masked region, and the overall processing time can be shortened.

[0149] On the other hand, when performing histogram calculation processing even in masked areas, as in this embodiment, there are advantages such as being able to record anomalies in the masked areas and suggest changes to the masked areas to the user, and being able to stably meet non-functional requirements by always maintaining a constant processing speed.

[0150] In S2313, the information processing device 104 performs an anomaly notification process (Figure 27) and notifies the user and external applications of information about the anomaly area.

[0151] The abnormality notification process shown in Figure 27 will be explained below. In S2701, the information processing device 104 determines whether there is an anomaly to be notified based on judgment criteria such as the continuity of anomaly detection. If it determines that there is an anomaly to be notified, it proceeds to S2702. If it determines that there is no anomaly to be notified, it proceeds to S2703.

[0152] Specifically, a criterion is needed to determine whether to notify a system of an anomaly if it is detected in a single image, or if to notify a system of an anomaly if it is detected consecutively in images spanning several seconds.

[0153] For example, the information processing device 104 may accept the user's pre-set criteria for how many consecutive frames of anomaly detection will trigger a notification, or it may determine what the image being analyzed is of (factory, home appliance, plant, etc.) or whether it is indoors or outdoors, and then determine the criteria for notifying an anomaly based on the object and environment.

[0154] In S2702, the information processing device 104 records information about the region where the notification target abnormality has occurred on the RAM 203.

[0155] In S2703, the information processing device 104 deletes the notification-targeted abnormal information recorded on the RAM 203.

[0156] In S2710, the information processing device 104 passes the still image information read in S2302 to the processing unit that draws the settings screen.

[0157] In S2711, the information processing device 104 passes the abnormal information recorded in S2702 to the processing unit that draws the settings screen.

[0158] In S2712, the information processing device 104 determines whether the settings screen is being displayed based on user input. If the settings screen is being displayed, the process proceeds to S2713. If the settings screen is not being displayed, the process terminates.

[0159] In S2713, the information processing device 104 checks the settings 2801 for the detection area drawing color on RAM 203. If there is a specification regarding the drawing method, the process proceeds to S2714. If there is no specification, the process proceeds to S2715.

[0160] In S2714, the information processing device 104 changes the highlighting during the drawing process in S2715, described later, based on the setting 2801 (Figure 28). In other words, this step is an example of a process that identifies and outputs which of several types of thresholds was used to determine that a change had occurred.

[0161] Setting 2801 specifies how to highlight items according to the detection factors for anomaly detection (items that fall below the threshold in S2309). In other words, this step is an example of a process that accepts specification of how to identify and output each type of threshold.

[0162] Here, to ensure that the highlighting for each threshold is distinguishable, control is implemented to prevent the same highlighting settings from overlapping (e.g., displaying an error message if the same highlighting is selected for different thresholds, preventing the selection of a previously selected highlighting). In other words, this step demonstrates an example of a process that prevents the specification of the same output method for different types of thresholds.

[0163] Specifically, the highlighting method can be set for each type of threshold, such as (1) red when the brightness threshold is exceeded, (2) yellow when the hue threshold is exceeded, (3) green when the saturation threshold is exceeded, (4) blue when the edge gradient strength threshold is exceeded, and (5) purple when the edge gradient angle threshold is exceeded.

[0164] In this embodiment, highlighting is done using color, but this method is not limited to this. Other methods may be used, such as using patterns like diagonal lines, setting the highlighting method to surround items with dotted or thick lines, or setting different notification methods such as voice or email. It is also possible to set notification methods using methods other than highlighting.

[0165] In S2715, the information processing device 104 overlays highlighting on the areas containing the still image information acquired in S2710 and the abnormal information acquired in S2711, and then displays the drawing image in the drawing memory on the RAM 203.

[0166] In S2716, the information processing device 104 draws the drawing image created in S2715 onto the setting screen 2810 (Figure 28). In other words, this step is an example of a process that outputs a result of determining whether or not there has been a change in the area.

[0167] If the drawing method is changed in S2714, the method of highlighting the abnormal region is changed and displayed as shown in abnormal region 2811(1)(2)(4).

[0168] This makes it easy for users to understand what threshold was used to determine that something was abnormal.

[0169] Alternatively, the highlighting of the abnormal area and the display of the masked area may be shown on the same screen (not illustrated).

[0170] Furthermore, although S2312 states that abnormal areas will not be displayed in the masked area, it is also possible to highlight the abnormal detection area (not shown) or highlight it in a different way (not shown) even in the masked area, based on user instructions. Doing so would give the user a reason to decide to remove the mask even in an area designated as a masked area, depending on the nature of the abnormal detection. In other words, it becomes possible to easily configure the abnormal detection desired by the user.

[0171] Furthermore, the information processing device 104 may have a function to notify the user on screens 1810 and 1820 of what kind of abnormality has occurred in the mask area, even without receiving instructions from the user.

[0172] Furthermore, the system may have functions to suggest to the user that they change the mask settings based on the number and nature of anomaly detections, or for the information processing device 104 to automatically change the mask settings. Doing so would enable more accurate anomaly detection. In other words, this step is an example of a process that changes the specified area based on the specified area and the judgment result of that area.

[0173] In S2720, the information processing device 104 notifies the output destination set in S407 (e.g., video monitoring server, mailer, software that can instruct to make calls using automated voice calls, etc.) of the abnormal information recorded in S2702.

[0174] In other words, this step is an example of a process that notifies the result of a determination that a change requiring notification has occurred.

[0175] Figure 29 shows an example of the anomaly notification dashboard screen, which displays a list of anomalies detected so far in a table format. Here, the detection time, monitoring task name, and coordinates of the detected area are displayed.

[0176] As a result, users will be able to easily find out what threshold was used to determine that something was abnormal. Furthermore, it will be possible to improve the accuracy of detecting anomalies near the grid.

[0177] Furthermore, it becomes easier to determine what criteria were used to detect the anomaly.

[0178] This concludes the description of the first embodiment.

[0179] Next, a second embodiment that reduces false notifications of abnormalities will be described. In the second embodiment, the flowchart in Figure 23 of the first embodiment is replaced with Figure 31. Figure 30 shows an example of an image analyzed that contains noise.

[0180] Here, 000.jpg is the reference image, and 001-006.jpg are the images to be analyzed. 005.jpg is an image that is distorted by noise and contains abnormal pixels.

[0181] The information processing device 104 determines that images 001 to 004.jpg are normal because they do not differ from the reference image and no abnormal areas are detected. (S2309 in Figure 23) On the other hand, 005.jpg contains noise and differs significantly from the reference image, so it is determined that there is an abnormal region in 005.jpg (S2309 in Figure 23), and as a result, an abnormality notification process (S2313 in Figure 23, Figure 27) is performed.

[0182] In other words, in the first embodiment, noise can cause an anomaly to be falsely reported.

[0183] In this embodiment, the cause of the false notification of an anomaly is stated as "noise," but it is not limited to "noise." It may also be the appearance of other organisms such as birds or insects, or debris. In other words, if organisms such as birds or insects, or debris, appear unexpectedly in the image, it will result in a false notification of an anomaly, just like noise.

[0184] Figure 31 is an example of a flowchart that rewrites the flowchart in Figure 23 of the first embodiment for use in the second embodiment. The same processes as in Figure 23 are omitted from the explanation. In S3101, the information processing device 104 performs the initialization process for each grid (Figure 32).

[0185] In S3102, the information processing device 104 performs an anomaly detection process for each grid (Figure 33). This concludes the explanation of Figure 31.

[0186] Figure 32 shows an example of a flowchart for the initialization process for each grid.

[0187] In S3201, the information processing device 104 initializes the number of normal detections and the number of abnormal detections held by each region separated by the grid 2500 and the correction grid 2501 to 0.

[0188] The number of normal detections is the number of consecutive times the system was determined to be normal, and the number of abnormal detections is the number of consecutive times the system was determined to be abnormal.

[0189] For example, if an anomaly is detected when the number of detected anomalies is 0, the number of detected anomalies will become 1.

[0190] If an anomaly is detected when the anomaly detection count is 1, the anomaly detection count becomes 2.

[0191] When the number of detected anomalies is greater than or equal to 1, and the system is determined to be normal, the number of detected anomalies resets to 0.

[0192] In S3202, the information processing device 104 sets the noise cancellation count for each region demarcated by the grid 2500 and the correction grid 2501, and stores it in the RAM 203.

[0193] The noise cancellation count is changed by modifying the slider bar for noise cancellation count 3401 on the analysis setting screen 3400 (Figure 34) in the second embodiment (S428 in Figure 4), saving the setting in RAM 203 (S429 in Figure 4), and obtaining the value of noise cancellation count 3401 from RAM 203 and using it in S3202.

[0194] The noise cancellation threshold is the threshold for whether to suppress notifications or stop suppressing notifications (i.e., send notifications) when normal detection and abnormal detection occur consecutively.

[0195] In this embodiment, the slider bar is changed on the analysis settings screen 3400 (Figure 34), but this method of acceptance is not limited to this. The value of the noise cancellation count may be accepted in a text box, or the value of the noise cancellation count may be entered in a settings file instead of on the settings screen.

[0196] In other words, this is an example of a process for obtaining a predetermined value related to the suppression of notifications.

[0197] Furthermore, although this embodiment accepts the "number of noise cancellations" from the user, it is not limited to this method. The "number of noise cancellations" may also be determined by accepting settings for the monitoring environment, the type of object to be monitored, the selection of setting templates, etc., from the user, or by the information processing device 104 automatically determining the environment to be monitored and the type of object to be monitored.

[0198] Specifically, the noise cancellation level could be set to 1 if the monitoring environment is indoors, and to 5 if it is outdoors (as there is a higher chance of birds, insects, or other living things unexpectedly appearing in the image). The "noise cancellation level" could be determined based on the settings of the monitoring environment or automatic detection of the monitoring environment.

[0199] Furthermore, if the monitored object is something that rarely changes, such as a building or furniture, the noise cancellation count can be set to 0. If it is something that changes gradually, such as a plant or crystal, but occasionally changes instantaneously, the noise cancellation count can be set to 3 (suppressing anomaly notifications due to momentary noise, shaking, or blurring of the monitored object, while still issuing an anomaly notification if three consecutive anomaly detections occur, treating it as a series of changes). The "noise cancellation count" can be determined based on the settings of the monitored object or automatic identification of the monitored object.

[0200] The details will be explained using Figure 35. Figure 35 shows an example of an abnormality notification in the second embodiment. In the first embodiment, there is no setting for the number of noise cancellations, so if, as in 3511, the image is distorted due to noise, etc., even though nothing is actually happening to the monitored target, and this is judged as an anomaly, an anomaly notification will be sent immediately.

[0201] In other words, because an anomaly is notified every time noise occurs in the image, if noise is caused by equipment failure such as a camera, cable deterioration, electromagnetic waves, high voltage, etc., an anomaly will be immediately and incorrectly notified.

[0202] Furthermore, in addition to noise, there are cases where living things such as birds or insects, or even trash, unexpectedly appear in the image, leading to incorrect notifications of anomalies.

[0203] On the other hand, in the second embodiment, when the noise cancellation count is set to "1", the information processing device 104 does not notify of an anomaly after the first anomaly detection (3521), and only notifies of an anomaly after two consecutive anomaly detections (3522).

[0204] Similarly, if the noise cancellation count is set to "2", the information processing device 104 will not notify of an anomaly after two consecutive anomaly detections, and will only notify of an anomaly after three consecutive anomaly detections.

[0205] Furthermore, if the noise cancellation count is set to "0", the information processing device 104 recognizes that an anomaly has been detected even once. In other words, by setting it to "0", notifications similar to those in the first embodiment can be set in the second embodiment as well.

[0206] In this embodiment, although the term "noise cancellation count" is used, it is not limited to "noise," and therefore the name is not limited to "noise cancellation count." It may also be called "number of (judgment result notification suppression) (indicating a change that should be notified)" or "notification threshold."

[0207] Alternatively, instead of using "number of noise cancellations," the system could use "noise cancellation time" and notify the system of an anomaly if "duration of anomaly detection > noise cancellation time" or "duration of anomaly detection ≥ noise cancellation time."

[0208] Furthermore, by not accepting the setting for "noise cancellation count" or obtaining the value of "noise cancellation count," it is possible to suppress false notifications caused by momentary noise or other occurrences by processing the nth anomaly detection so that no anomaly notification is sent.

[0209] In other words, this is an example of a process that suppresses notification of a determination result indicating that a change requiring notification has occurred, based on at least one of the number of changes that should be notified and a predetermined value.

[0210] In this embodiment, anomaly notifications are suppressed based on "the number of consecutive images analyzed in which a change requiring notification has been determined." However, this is not limited to "the number of consecutive images analyzed." It may also be "the number of (non-consecutive) images analyzed in which a change requiring notification has been determined," "the number of locations in the images analyzed in which a change requiring notification has been determined," or "the number of regions in the images analyzed in which a change requiring notification has been determined."

[0211] Furthermore, in this embodiment, anomaly notifications are suppressed based on "the number of consecutive images to be analyzed in which it is determined that there is a change that should be notified in a predetermined area of ​​the image to be analyzed." However, this is not limited to "the number of consecutive images to be analyzed," and may be "the number of (non-consecutive) images to be analyzed in which it is determined that there is a change that should be notified in a predetermined area of ​​the image to be analyzed," "the number of locations in a predetermined area of ​​the image to be analyzed in which it is determined that there is a change that should be notified in the image to be analyzed," or the like.

[0212] As a result, it becomes possible to control the system as shown in Figures 3521 and 3522, preventing the system from mistakenly notifying of abnormalities caused by noise, and thus reducing false notifications of abnormalities. This concludes the explanation of Figures 32, 34, and 35.

[0213] Figure 33 shows an example of a flowchart for the anomaly detection process for each grid.

[0214] In S3301, the information processing device 104 determines whether each region separated by the grid 2500 and the correction grid 2501 is an abnormal region or a normal region based on the determination results in S2309 (Figure 23) and S2311 (Figure 23). If it is an abnormal region, the device proceeds to S3302; if it is a normal region, the device proceeds to S3308.

[0215] In S3302, the information processing device 104 clears the number of normal detections held by the area that was determined to be abnormal in S3301 to 0.

[0216] In S3303, the information processing device 104 determines whether the area that was determined to be abnormal in S3301 has been detected as abnormal or not. If no abnormality has been detected, the process proceeds to S3304. If an abnormality has been detected, the process shown in Figure 33 is terminated.

[0217] In S3304, the information processing device 104 adds 1 to the number of abnormality detections held by the area that was determined to have no abnormality detected in S3303.

[0218] In S3305, the information processing device 104 compares the number of anomaly detections added in S3304 with the noise cancellation setting in S3202. If it determines that "number of anomaly detections > number of noise cancellations", it proceeds to S3306. If it determines that "number of anomaly detections ≤ number of noise cancellations", it terminates the process shown in Figure 33.

[0219] In other words, if it is determined that "number of anomaly detections > number of noise cancellations", an anomaly notification will be sent, and if it is determined that "number of anomaly detections ≤ number of noise cancellations", the anomaly notification will be suppressed.

[0220] In S3305, the determination was made as to whether "number of anomaly detections > number of noise cancellations," but this method is not limited to this one. It is also possible to determine whether "number of anomaly detections ≥ number of noise cancellations," or whether "duration of anomaly detection > noise cancellation duration," or whether "duration of anomaly detection ≥ noise cancellation duration."

[0221] In other words, this step is an example of a process that suppresses notification of a determination result indicating that a change requiring notification has occurred, based on a predetermined value.

[0222] In other words, this step is an example of a process that suppresses notification of a determination result indicating that a change requiring notification has occurred when the number of changes requiring notification is less than or equal to a predetermined value.

[0223] In S3306, the information processing device 104 clears the number of anomaly detections to 0.

[0224] In S3307, the information processing device 104 sets the region where the number of anomaly detections was determined to be greater than the number of noise cancellations in S3305 as "anomaly detected" and terminates the process shown in Figure 33.

[0225] In S3308, the information processing device 104 clears the number of abnormality detections held by the area that was determined to be normal in S3301 to 0.

[0226] In S3309, the information processing device 104 determines whether the area that was determined to be abnormal in S3301 has been detected as abnormal or not. If an abnormality has been detected, the process proceeds to S3310; otherwise, the process shown in Figure 33 is terminated.

[0227] In S3310, the information processing device 104 adds 1 to the number of normal detections held by the area that was determined to have detected an anomaly in S3309.

[0228] In S3311, the information processing device 104 compares the number of normal detections added in S3310 with the noise cancellation setting in S3202. If it determines that "number of normal detections > number of noise cancellations", it proceeds to S3312. If it determines that "number of normal detections ≤ number of noise cancellations", it terminates the process shown in Figure 33.

[0229] In other words, if it is determined that "number of normal detections ≤ number of noise cancellations", the abnormality notification will continue, and if it is determined that "number of normal detections > number of noise cancellations", the abnormality notification that was previously being sent will be stopped.

[0230] In S3311, it was determined whether "number of normal detections > number of noise cancellations" was true, but this method is not the only way to determine this. It is also possible to determine whether "number of normal detections ≥ number of noise cancellations" is true, or whether "duration of normal detection > noise cancellation duration" or "duration of normal detection ≥ noise cancellation duration" is true.

[0231] In S3312, the information processing device 104 clears the number of normal detections to 0.

[0232] In S3313, the information processing device 104 cancels the abnormal detection status for the region where the number of normal detections > the number of noise cancellations determined in S3311, and terminates the process shown in Figure 33.

[0233] This means that instead of immediately stopping the abnormality notification once normalcy is detected after an abnormality notification, the abnormality notification will continue until the number of normal detections exceeds the limit.

[0234] This means that, for example, when an anomaly is indicated by a flashing light or warning sound, the anomaly notification will not end instantly, but will continue for a certain period of time, making it easier for the monitor to recognize the anomaly.

[0235] This concludes the explanation of Figure 33. As a result, it becomes possible to control the system as shown in Figures 3521 and 3522, thereby reducing false notifications of abnormalities. Furthermore, since the anomaly detection target is not limited to still images, but can also include video files and live streaming (S2103), it becomes possible to reduce false notifications of anomalies even in videos or live streaming that contain noise, etc.

[0236] Furthermore, since the process shown in Figure 33 is a simple process, unlike the complex processes of filter application and difference calculation processing described in Patent Document 1, it can reduce false notifications of anomalies without placing a heavy load on the information processing system.

[0237] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium containing a program that realizes the functions of the embodiments described above to a system or device, and by having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0238] In this case, the program read from the recording medium itself realizes the novel function of the present invention, and the recording medium on which that program is recorded constitutes the present invention.

[0239] For recording media used to supply programs, examples include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, and the like.

[0240] Furthermore, it goes without saying that the functions of the aforementioned embodiments are realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of that program, thereby realizing the functions of the aforementioned embodiments.

[0241] Furthermore, it goes without saying that this also includes cases where, after a program read from a recording medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer, the CPU or other components of the function expansion board or function expansion unit perform some or all of the actual processing based on the instructions of the program code, and the functions of the aforementioned embodiments are realized through that processing.

[0242] Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device. It goes without saying that the present invention can also be applied when the results are achieved by supplying a program to a system or device. In this case, by reading a recording medium containing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

[0243] The above program may consist of object code, program code executed by an interpreter, script data supplied to the OS (operating system), and other forms.

[0244] Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that all configurations combining the above-described embodiments and their modified forms are also included in the present invention. [Explanation of Symbols]

[0245] 100 Information Processing Systems 102 Cameras 104 Information Processing Device

Claims

1. A determination means that determines whether there is a change that should be notified based on a comparison between a reference image and the image to be analyzed, A notification means for notifying the result of the determination that there has been a change that warrants notification, means for obtaining a predetermined value related to the suppression of the aforementioned notification Equipped with, The notification means is, Based on the predetermined value, the notification of the determination result that there has been a change that should be notified is suppressed. An information processing system characterized by the following.

2. The notification means is, The notification of the determination that a change requiring notification has occurred is suppressed based on at least one of the number of changes requiring notification and the predetermined value. The information processing system according to claim 1, characterized by the following:

3. The notification means is, If the number of changes that should be notified is less than or equal to the predetermined value, the notification of the determination that a change that should be notified has occurred will be suppressed. The information processing system according to claim 1, characterized by the following:

4. The notification means is, After notifying the determination result that there has been a change that warrants notification, the notification of the determination result that there has been a change that warrants notification shall be stopped based on at least one of the predetermined value and the number of people determined not to have made a change that warrants notification. The information processing system according to claim 1, characterized by the following:

5. The changes that need to be notified are: The reference image and the image to be analyzed are compared, and the similarity is less than a predetermined threshold, or the similarity is less than or equal to a predetermined threshold, or the amount of change exceeds a predetermined threshold, or the amount of change is greater than or equal to a predetermined threshold. The information processing system according to claim 1, characterized by the following:

6. The changes that need to be notified are: The reference image and the image to be analyzed are compared in corresponding regions, and the similarity between the regions is less than a predetermined threshold, or the similarity between the regions is less than or equal to a predetermined threshold, or the amount of change between the regions exceeds a predetermined threshold, or the amount of change between the regions is greater than or equal to a predetermined threshold. The information processing system according to claim 1, characterized by the following:

7. The comparison between the aforementioned reference image and the image to be analyzed is as follows: The reference image and the image to be analyzed are compared based on the image's features. The information processing system according to claim 1, characterized by the following:

8. The comparison between the aforementioned reference image and the image to be analyzed is as follows: The reference image and the image to be analyzed are compared based on information relating to at least one of the following: brightness, hue, saturation, edge gradient intensity, and edge gradient angle. The information processing system according to claim 1, characterized by the following:

9. At least one of the reference image and the image to be analyzed is an image extracted from a video. The information processing system according to claim 1, characterized by the following:

10. The number of changes to be notified is, The number of the analyzed images for which it was determined that there was a change that should be notified, The number of consecutive images analyzed in which it was determined that there was a change that warranted notification, The number of locations in the analyzed image where it was determined that there was a change that warranted notification, or This is the number of regions in the aforementioned analyzed image that were determined to have a change that warrants notification. The information processing system according to claim 2 or 3, characterized by the above.

11. The number of changes to be notified is, in a predetermined region of the image to be analyzed, The number of the analyzed images for which it was determined that there was a change that should be notified, The number of consecutive images analyzed in which a change requiring notification has been determined, or This is the number of locations in the aforementioned analyzed image where it was determined that there was a change that warranted notification. The information processing system according to claim 2 or 3, characterized by the above.

12. The predetermined threshold is, The threshold must relate to at least one of the following: luminance, hue, saturation, edge gradient intensity, and edge gradient angle. The information processing system according to claim 5 or 6, characterized by the above.

13. A determination step that determines whether there are any changes that should be notified based on a comparison between the reference image and the image to be analyzed, A notification step to notify the result of the determination that there has been a change that warrants notification, An acquisition step to obtain a predetermined value related to the suppression of the aforementioned notification. Equipped with, The aforementioned notification step is, Based on the predetermined value, the notification of the determination result that there has been a change that should be notified is suppressed. A control method for an information processing system characterized by the following.

14. A program for causing at least one computer to function as one of the means of an information processing system described in any one of claims 1 to 9.

Images