Information processing systems, information processing methods, and programs

Abstract

This provides a mechanism that allows for the appropriate setting of thresholds in anomaly detection. [Solution] An information processing system characterized by comprising: setting display means for displaying settings related to the determination of whether an image has changed; image display means for displaying the image and the region in which the image has been determined to have changed in an identifiable manner; and graph display means for displaying information related to the region of the image as a graph.

Description

Technical Field

[0001] The present invention relates to an information processing system, a control method for an information processing system, and a program, and particularly to a technique suitable for use in detecting abnormalities.

Background Art

[0002] Conventionally, there is a method of detecting an abnormality of an imaging object based on a change in an image by processing a plurality of images obtained by imaging with a camera or the like.

[0003] Patent Document 1 discloses a technique for detecting a local difference region between images using feature amounts of the images and displaying the difference region in an identifiable manner.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, it is described that a graph of the degree of difference between videos is displayed along with the time axis of the video. However, there is a problem that the reference value of the degree of difference cannot be adjusted, and it is desired to enable the user to easily adjust the reference value while checking the difference region.

[0006] Therefore, an object of the present invention is to provide a mechanism capable of suitably setting a threshold value in abnormality detection.

Means for Solving the Problems

[0007] The present invention is characterized by comprising: setting display means for displaying settings related to the determination of whether a change has occurred in an image; image display means for displaying the image and the region in which a change has been determined to have occurred in the image in an identifiable manner; and graph display means for displaying information related to the region of the image as a graph. [Effects of the Invention]

[0008] According to the present invention, a mechanism can be provided that allows for the appropriate setting of a threshold value in anomaly detection. [Brief explanation of the drawing]

[0009] [Figure 1] This is a system configuration diagram of information processing system 100. [Figure 2] This is a hardware block diagram of the information processing device 104. [Figure 3] This is an example of a block diagram showing the software configuration. [Figure 4] (a) A flowchart of the configuration process and (b) an example of a flowchart of the anomaly detection parameter setting process for the monitoring task. [Figure 5] This is an example of the initial display of the settings screen. [Figure 6] This is an example of an overall view of the settings screen. [Figure 7] This is an example of the settings screen (input tab). [Figure 8] This is an example of the settings screen (input tab). [Figure 9] This is an example of the settings screen (analysis tab). [Figure 10] This is an example of the settings screen (notification tab). [Figure 11] This is an example of the settings screen (monitoring processes and monitoring tasks). [Figure 12] This is an example of the settings screen (input tab). [Figure 13] This is an example of the settings screen (input tab). [Figure 14] This is an example of the settings screen (notification tab). [Figure 15] This is an example of the display of the settings screen (notification tab). [Figure 16] This is an example of the display of the settings screen (monitoring task). [Figure 17] This is an example of the display of the adjustment screen for analysis settings. [Figure 18] This is an example of the display of the grid settings screen (grid division). [Figure 19] This is an example of the display of the grid settings screen (grid correction). [Figure 20] This is an example of the display of the analysis settings screen. [Figure 21] This is an example of the flowchart of the abnormality detection process. [Figure 22] This is an example of the display of task registration information. [Figure 23] This is an example of the flowchart of the image difference (abnormality) detection process. [Figure 24] This is an example of the flowchart of each histogram calculation process. [Figure 25] This is an example of the display of grid correction. [Figure 26] This is an example of the display of notifications during grid correction. [Figure 27] This is an example of the flowchart of the abnormality notification process. [Figure 28] This is an example of the display of the settings screen (abnormality notification). [Figure 29] This is an example of the display of the abnormality notification dashboard screen. [Figure 30] This is an example of the flowchart of the image difference (abnormality) confirmation process. [Figure 31] This is an example of the flowchart of the graph display process of the selected grid. [Figure 32] This is an example of the display of the adjustment screen for analysis settings and each histogram.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] Figure 1 is a system configuration diagram showing an example of the configuration of the information processing system 100 of the present invention.

[0012] Figure 1 shows a configuration in which the camera 102 and the information processing device 104 are connected via an image transfer cable (USB, Ethernet, CameraLink, etc.). However, a configuration of an information processing device equipped with a camera is also acceptable, even without this specific setup.

[0013] The information processing device 104 acquires an image obtained by the camera 102 capturing the target object via an image transfer cable, and performs processing related to the detection of abnormalities in the image.

[0014] Furthermore, the information processing system 100 of this embodiment may also have a configuration in which the camera 102 with the above-described configuration does not exist.

[0015] In that case, processing related to the detection of abnormalities in images is performed on images acquired from other terminals on the network or on images stored in the memory (RAM) of the information processing device 104.

[0016] The following describes the hardware configuration of an information processing device applicable to the information processing device 104 shown in Figure 1, using Figure 2 as an example.

[0017] Figure 2 is a block diagram showing a hardware configuration applicable to the information processing device 104 shown in Figure 1.

[0018] In Figure 2, 201 is the CPU, which comprehensively controls each device and controller connected to the system bus 204. The ROM 202 or external memory 212 stores the BIOS (Basic Input / Output System), which is the control program for the CPU 201, the operating system program (hereinafter referred to as the OS), and various programs necessary to implement the functions executed by each PC, as described later.

[0019] 203 is RAM, which functions as the main memory, work area, etc., of the CPU 201. The CPU 201 loads the necessary programs, etc., from ROM 202 or external memory 212 into RAM 203, and then executes the loaded programs to perform various operations.

[0020] 205 is an input controller that controls input from pointing devices such as the keyboard (KB) 210 and a mouse (not shown).

[0021] 206 is a video controller that controls the display on indicators such as display 211.

[0022] 207 is the memory controller, which stores various data on external storage devices (hard disks (HDs)), flexible disks (FDs), or PCMCIA card slots. It controls access to external memory 212, such as CompactFlash® memory, which is connected via an adapter.

[0023] 208 is a communication interface controller that controls the reception of image data from the external PC 213 via the network (TCP / IP). 209 is an image I / F controller that controls the reception of image data from camera 102 via an image transfer cable (USB, Ethernet, Camera Link, etc.).

[0024] The various programs described later for realizing the present invention are stored in RAM 203 and executed by CPU 201.

[0025] Furthermore, the image data used when executing the above program is stored in ROM202, external memory212, external PC213, and camera102 depending on the application, and is stored in RAM203 via various controllers when the program is executed. Figure 3 is an example of a block diagram showing the software configuration of an embodiment of the present invention.

[0026] The information processing device 104 includes the following functional units.

[0027] The acquisition unit 301 is a functional unit that acquires images captured at the same angle of view.

[0028] The reception unit 302 is a functional unit that receives specifications for the area to be included in the image and the method of displaying that area.

[0029] The determination unit 303 is a functional unit that determines whether a change has occurred in a given region based on changes between corresponding regions of multiple images acquired by the acquisition unit 301.

[0030] The output unit 304 is a functional unit that outputs the determination result from the determination unit 303.

[0031] The output unit 304 is a functional unit that displays the area specified by the reception unit 302 using the display method specified by the reception unit 302, and controls whether or not to output a judgment result based on the area specified by the reception unit 302.

[0032] The area change unit 305 is a functional unit that changes the area specified by the reception unit 302 based on the area specified by the reception unit 302 and the determination result of that area.

[0033] The splitting unit 306 is a functional unit that splits the image acquired by the acquisition unit 301.

[0034] The determination unit 303 is a functional unit that determines whether a change has occurred in a given region based on changes between corresponding regions that have been divided by the division means from among multiple images acquired by the acquisition unit 301.

[0035] The division unit 306 is a functional unit that divides an image using two or more division methods.

[0036] The output unit 304 is a functional unit that outputs a determination result based on the region divided by the first division method when the determination unit 303 determines that there has been a change in the region divided by the second division method.

[0037] The determination unit 303 is a functional unit that determines whether a change has occurred in a given region based on the changes between corresponding regions of multiple images acquired by the acquisition means and multiple types of thresholds.

[0038] The output unit 304 is a functional unit that identifies and outputs which of the multiple types of thresholds was used to determine that a change had occurred.

[0039] The reception unit 302 is a functional unit that accepts specifications for the method of identification and output for each type of threshold.

[0040] The reception unit 302 is a functional unit that controls the system to prevent the same output method specification from being accepted for different threshold types.

[0041] This concludes the explanation of Figure 3. The setup process shown in Figure 4(a) will be explained below.

[0042] In S401, the information processing device 104 determines whether a configuration file for the operating process exists in the external memory 212. If it is not registered, the process proceeds to S402; if it is registered, the process proceeds to S403.

[0043] In S402, the information processing device 104 displays the settings screen 510 (Figure 5), and when it receives a press of the add monitoring process button 511 from the user, it registers the operation process in RAM 203 and displays the settings screen 520.

[0044] In S403, when the information processing device 104 receives a press of the button 521 for selecting the operation process to be edited on the setting screen 502, it displays the setting screen 610 (Figure 6). The setting acceptance process performed on setting screen 610 in S403 will be explained below using Figures 6 to 10.

[0045] When the information processing device 104 receives input from the user for the name 611 and description 612 of the operating process, it edits the operating process information in the RAM 203 and displays the settings screen 620.

[0046] When the information processing device 104 receives a press from the user on the settings screen 610, it displays the settings screen 710 (Figure 7).

[0047] When the information processing device 104 receives a selection from the user for "Video file loading (FFmpeg)" (721 in Figure 7) from the list of functions 711, it displays the settings screen 720. At this time, the settings screen 720 edits the operation process information of RAM 203 based on the location 722 and option 723 values ​​of the command entered by the user.

[0048] When the information processing device 104 receives a selection from the user for "Video file loading (OpenCV)" (731 in Figure 7) from the list of functions 711, it displays the settings screen 730 and edits the operation process information of RAM 203.

[0049] When the information processing device 104 receives a selection from the user for "Live video loading (video surveillance server)" (811 in Figure 8) from the list of functions 711, it displays the settings screen 810 (Figure 8). "Live video loading (video surveillance server)" is a function that loads live video captured by a network camera from a video surveillance server (not shown) that can be connected via the network.

[0050] Based on the computer name 812, username 813, and password 814 values ​​entered by the user on the settings screen 810, the operating process information of RAM 203 is edited. Also, if the connection confirmation 815 is pressed, a connection test is performed from the information processing device 104 to the video surveillance server via the communication I / F controller 208 using the information in RAM 203.

[0051] When the information processing device 104 receives a click from the user on the settings screen 610 to select the analysis tab 614 (Figure 6), it displays the settings screen 910 (Figure 9).

[0052] When the information processing device 104 receives a selection of "Detection by video comparison" from the function list 911 from the user, it displays the settings screen 920 and edits the operation process information of RAM 203. At this time, if the license is not authenticated, "No license" is displayed in the license information 922.

[0053] When the information processing device 104 receives a press of the license read button 923 from the user, if a valid license has been authenticated, it displays the settings screen 930, displays the license information 931, and edits the operation process information of the RAM 203.

[0054] When the information processing device 104 receives a notification from the user on the settings screen 610 by clicking the notification tab 615 (Figure 6), it displays the settings screen 1010 (Figure 10).

[0055] When the information processing device 104 receives a selection of a "custom command" (1021 in Figure 10) from the function list 1011 from the user, it displays the settings screen 1020 and edits the operation process information of the RAM 203.

[0056] A "custom command" is a function that executes commands to instruct batches or applications to run when an anomaly is detected. Using this function, it becomes possible to execute batch processing, instruct email sending to mail clients, instruct automated voice calls to be made, and integrate with other applications or external systems.

[0057] When the information processing device 104 receives a selection from the user for "Analysis Result Notification (Video Surveillance Server)" (1031 in Figure 10) from the function list 1011, it displays the settings screen 1030.

[0058] "Analysis Result Notification (Video Surveillance Server)" is a function that notifies the video surveillance server of the analysis results. This function is used when the video surveillance server is configured to notify users of anomalies, or when the video surveillance server analyzes and manages events such as anomalies.

[0059] Based on the values ​​of address 1032 and port number 1033 entered by the user on the settings screen 1030, the operating process information of RAM203 is edited.

[0060] When the information processing device 104 receives a connection confirmation 1034 from the user, it performs a connection test via the communication I / F controller 208 using the information in the RAM 203. Returning to the explanation of Figure 4.

[0061] In S404, when the information processing device 104 receives a press of the OK button 631 on the setting screen 630 (Figure 6), it saves the operation process information registered in RAM 203 to external memory 212 as an operation process setting file.

[0062] In S405, the information processing device 104 determines whether a configuration file for the monitoring task exists in the external memory 212. If it is not registered, the process proceeds to S406; if it is registered, the process proceeds to S407.

[0063] In S406, the information processing device 104 displays the settings screen 630, and when it receives a press from the user of the Add task button 632 for the monitoring process, it registers the monitoring task in RAM 203 and displays the settings screen 1110 (Figure 11).

[0064] In S407, when the information processing device 104 receives a click on the monitoring task 1111 to be edited on the settings screen 1110, it displays the settings screen 1120. The following describes the process for receiving settings for monitoring tasks performed in S407, using Figures 11 to 15.

[0065] When the information processing device 104 receives input from the user on the settings screen 1120, including a name 1121 and a description 1122, and a change in the toggle button 1123 for enabling this task, it edits the operation process information of the RAM 203 and displays the settings screen 1130.

[0066] When the information processing device 104 receives a press of the input tab 1124 from the user on the settings screen 1120, it displays either the settings screen 1200 (Figure 12) or the settings screen 1300 (Figure 13) depending on the settings of the operation process.

[0067] If the input setting for the operation process is "Read video file (FFmpeg)" or "Read video file (OpenCV)", the settings screen 1200 (Figure 12) will be displayed.

[0068] When the information processing device 104 receives a click on a video folder 1201 and a folder selection from the user, input of a frame acquisition interval 1202, and a change in the toggle button for deleting processed videos 1203, it edits the operation process information of RAM 203 and displays the settings screen 1210.

[0069] If the input setting for the operation process is "live video loading," the settings screen 1300 (Figure 13) is displayed. When the information processing device 104 receives a selection of a list item for camera 1301 from the user, it edits the operation process information in RAM 203 and displays the settings screen 1310.

[0070] When the information processing device 104 receives a notification from the user on the settings screen 1120 by clicking the notification tab 1125, it displays either the settings screen 1400 or the settings screen 1500, depending on the settings of the operation process.

[0071] If the notification setting for the operation process is "custom command," the settings screen 1400 (Figure 14) is displayed. When the information processing device 104 receives input of the command path 1401 from the user or a change in the toggle button for asynchronous execution 1402, it edits the operation process information in RAM 203 and displays the settings screen 1410.

[0072] If the notification setting for the operation process is "Analysis Result Notification (Video Surveillance Server)", the settings screen 1500 (Figure 15) is displayed. The information processing device 104 accepts the input of name 1501 and camera ID 1502 from the user and displays screen 1510. At this time, if the test event button 1511 is pressed by the user, a test notification process is executed to check whether the notification with the input content works correctly. At this time, if the setting of the detection frame display toggle button 1512 is enabled, test pseudo detection result information is also added to the notification content. Returning to the explanation of Figure 4.

[0073] In S408, when the information processing device 104 receives a press of the OK button 1602 from the user on the setting screen 1600 (Figure 16), it saves the monitoring task information registered in RAM 203 as a monitoring task setting file in external memory 212 and displays the setting screen 1610.

[0074] In S409, when the user clicks the analysis tab 1611 on the settings screen 1610, the settings screen 1700 (Figure 17) is displayed. At this time, when the user clicks the adjustment button 1701, the analysis settings adjustment screen 1710 is displayed.

[0075] Subsequently, the information processing device 104 executes the process shown in Figure 4(b). The process for setting anomaly detection parameters for the monitoring task shown in Figure 4(b) will be explained below.

[0076] In S421, when the information processing device 104 receives a press of the basic settings button 1711 from the user on the analysis settings adjustment screen 1710 (Figure 17), it displays the basic settings screen 1720 for analysis settings and proceeds to S422. If the Basic Settings button is not pressed, proceed to S428.

[0077] In S422, when the information processing device 104 receives a request from the user to change the slider bar to 1801 rows (Figure 18) and 1802 columns, it updates the display of the grid (dashed lines) on the screen, which is divided by 1801 rows and 1802 columns, as shown in screen 1800 (Figure 18), and saves the settings to RAM 203.

[0078] Specifically, on screen 1800, the number of rows is set to 10 (1801) and the number of columns to 10 (1802), so the image to be analyzed is displayed, divided into a 10x10 grid.

[0079] While a larger number of grids allows for more precise detection, it also increases the number of processes involved, such as the image difference (anomaly) detection process described later (Figure 23), potentially increasing the processing time.

[0080] In S423, when the information processing device 104 receives a press 1811 of any area divided by a grid on the user's screen, a press of the full mask setting button 1812, or a press of the full mask release button 1813, it displays the presence or absence of a mask (notification suppression area) as shown on screen 1810 and saves the setting in RAM 203. In other words, this step is an example of a process that accepts the specification of an area included in an image.

[0081] If there are many areas to mask, you can easily set the areas to be masked by first masking all areas using the "Set All Masks" button (1812), and then specifying the areas to be unmasked.

[0082] If the masked area is small, you can easily set the area to be masked by first removing the mask from the entire area using the "Remove All Mask" button (1813), and then specifying the area to be masked.

[0083] Furthermore, if the user changes the mask color inversion 1821 toggle button at this time, the mask drawing color is changed as shown in screen 1820. Specifically, in screen 1810, the mask area was represented by transparent black, but in screen 1820, the mask area is represented by transparent white. In other words, this step is an example of a process that accepts a specification for how to display an area included in an image. This step is also an example of a process that accepts a specification for the display method of the area, and displays the area that has been specified using the accepted display method.

[0084] This allows the user to select the color of the mask area according to the color tone and brightness of the image being analyzed. This is achieved by representing the mask area with transparent black when the image is bright and transparent white when the image is dark. As a result, the user can easily identify and understand the appearance of the mask area.

[0085] In this embodiment, the color of the mask area is set to "transparent black" or "transparent white," but this method is not limited to this. Other colors may be used, the user may specify a color, patterns such as diagonal lines may be used to add visual variation, a frame may be used, or other methods may be used to distinguish the monitored area from the mask area.

[0086] In S424, when the information processing device 104 receives a change from the user to the grid correction (shifting the grid) toggle button 1901 (Figure 19) or 1911, it updates the display of the frame 1902 (with frame) or 1912 (without frame) that indicates the detection range and saves the setting to RAM 203.

[0087] In this embodiment, grid correction is set for the monitoring process, but this method is not limited to this one. Grid correction may also be set directly for the network camera or video monitoring server that is the source of the image input.

[0088] In S425, when the information processing device 104 receives a numerical input 1921 for the number of reference image frames from the user, it saves the setting in the RAM 203.

[0089] In S426, when the information processing device 104 receives a press of the back button 1803 from the user while the basic settings screen 1800 (Figure 18) for analysis settings is displayed, it determines whether there are any changes to the basic settings. If there are changes, it proceeds to S427. If there are no changes, it proceeds to S428.

[0090] In S427, the information processing device 104 reflects the contents set in RAM 203 in the monitoring task configuration file in external memory 212. The information processing device 104 also detects changes in the monitoring task configuration file and performs reference image discarding during the analysis process. (In S2306, it determines that updating the base information is necessary.) In S428, when the information processing device 104 receives a change in the slider bar for each item of the threshold 2001 on the analysis setting screen 2000 (Figure 20), it saves the setting to the RAM 203.

[0091] In this embodiment, the user sets each threshold, but this method is not the only way. The user may select a dataset for each threshold, the user may set the objects to be monitored or the anomalies to be detected to automatically set the threshold parameters, or the information processing device 104 may determine the objects to be monitored or the anomalies to be detected and automatically set the threshold parameters.

[0092] This makes it easy to set anomaly detection thresholds.

[0093] In S429, when the information processing device 104 receives a press of the apply button 2002 from the user, it reflects the contents set in RAM 203 into the monitoring task configuration file in external memory 212. The information processing device 104 also detects changes in the monitoring task configuration file and modifies the parameter values ​​used for detection processing. (This modifies the threshold used in S2310.) The anomaly detection process shown in Figure 21 will be explained below.

[0094] In S2101, the information processing device 104 executes anomaly detection processing if the external memory 212 contains a configuration file for the monitoring task (task registration information 2201 (Figure 22)) and the task activation setting toggle 2202 is set to enabled.

[0095] In S2102, the information processing device 104 monitors whether there is new input information. If there is new input information, the process proceeds to S2103. If there is no new input information, the process returns to S2101 and repeats.

[0096] In S2103, the information processing device 104 acquires input information according to the configured input method (configured on the configuration screen 1200 or configuration screen 1300).

[0097] Specifically, if the input setting for the operation process (settings screen 1200) is "Read video file (FFmpeg)" or "Read video file (OpenCV)", the video files located in the specified video folder are used as input information and moved to the working area on external memory 212.

[0098] Furthermore, if the input setting for the operation process is "Live video loading (video monitoring server)" (settings screen 1300), the process of acquiring video from the specified video acquisition destination is executed, and if a still image is acquired, the process proceeds to S2106.

[0099] In S2104, if the information processing device 104 performed a video file transfer on the external memory 212 in S2103, it proceeds to S2105. If it has performed a video acquisition process and acquired a still image, it proceeds to S2106.

[0100] In S2105, the information processing device 104 reads the video file moved to the external memory 212 in S2103 using the means set in the input settings of the operation process, divides it into a series of still images for each frame, saves them as files on the external memory 212 or stores them as data on the RAM 203, and proceeds to S2106.

[0101] In S2106, the information processing device 104 executes the process shown in Figure 23. The image difference (anomaly) detection process shown in Figure 23 will be explained.

[0102] In S2301, if the information processing device 104 has been splitting the video file into a series of still images frame by frame (S2105), it repeats the subsequent processing until processing is completed for all the generated still images.

[0103] In S2302, the information processing device 104 reads the still image to be processed and loads it onto the RAM 203. In other words, this step is an example of a process for acquiring images captured at the same angle of view.

[0104] In S2303, the information processing device 104 calculates the hue, saturation, and lightness information from the image data loaded on the RAM 203.

[0105] In S2304, the information processing device 104 performs each histogram calculation process (Figure 24) to calculate a histogram from the image data. Here, we will explain the calculation process for each histogram in Figure 24.

[0106] In S2401, the information processing device 104 proceeds to S2402 if the grid correction set in S424 is enabled among the settings stored in RAM 203. Otherwise, it proceeds to S2403.

[0107] In S2402, the information processing device 104 includes not only the regions demarcated by the grid 2500 (Figure 25) set in S422, but also regions demarcated by grid 2501, which is shifted horizontally and vertically by half the size of one region, as the target regions for histogram calculation. In other words, this step is an example of a process for dividing the acquired image.

[0108] Specifically, as shown in 2510 and 2520, if an anomaly occurs near the 3x4 grid set in S422, the anomaly spans multiple areas. As a result, the threshold is not exceeded in each area, and all areas are judged as normal, potentially leading to low anomaly detection accuracy.

[0109] As in S2402, by determining anomalies not only in the area demarcated by the normal grid 2500 set in S422, but also in the area demarcated by the corrected grid 2501 (shifted grid), it becomes easier to improve the accuracy of anomaly detection even when an anomaly occurs near the normal grid 2500 set in S422.

[0110] In this embodiment, the grid is shifted horizontally and vertically by half the size of one region, but this method is not limited to this. Two new grids may be created by shifting them horizontally and vertically by "1 / 3" and "2 / 3" of the size of one region, or a hexagonal grid may be created instead of a grid (square). In other words, this step is an example of a process in which an image is divided using two or more division methods. Specifically, this is an example in which a region divided by the first division method partially overlaps with at least one of the regions divided by the second division method.

[0111] This makes it easier to improve the accuracy of detecting anomalies, even if an anomaly occurs near grid 2500.

[0112] In S2403, the information processing device 104 calculates a histogram of luminance information from the luminance information acquired in S2303 for each processing target area set in S422 and S2402.

[0113] In S2404, the information processing device 104 calculates a histogram of hue information from the hue and saturation information acquired in S2303 for each processing target area set in S422 and S2402.

[0114] In S2405, the information processing device 104 calculates a histogram of saturation information from the saturation information acquired in S2303 for each processing target area set in S422 and S2402.

[0115] In S2406, the information processing device 104 calculates edge gradient information from the lightness information acquired in S2303 for each processing area set in S422 and S2402, and uses the edge gradient information to calculate an edge gradient intensity histogram and an edge gradient angle histogram. Let's return to the explanation of Figure 23.

[0116] In S2305, the information processing device 104 calculates the similarity for each processing area set in S422 and S2402 using a histogram of reference information and a histogram intersection, which is one of the histogram comparison methods.

[0117] In this case, the reference information is the histogram calculated in S2308, described later. If reference histogram data is unavailable, the similarity is set to 100%.

[0118] In this embodiment, a histogram intersection was used as the histogram comparison method, but the method is not limited to this, and other methods such as the Bhattacharya coefficient may be used.

[0119] Furthermore, while this embodiment involves detecting anomalies by comparing histograms, it is not limited to this method. Other methods, such as template matching or feature point detection, may also be used to detect anomalies.

[0120] In S2306, if the number of data already set in the reference image is less than the "number of reference image frames" set in S425, the information processing device 104 determines that the reference image needs to be updated and proceeds to S2307. If data that satisfies the "number of reference image frames" has already been set in the reference image, the process proceeds to S2309.

[0121] Furthermore, the information processing device 104 discards the reference image in the following cases and initializes the number of set data items to 0.

[0122] • When the basic settings are changed during user operation of S427 • When the user clicks the "Recreate Reference Image" button (2003) This allows users to easily instruct the system to recreate the reference image.

[0123] In S2307, the information processing device 104 determines whether the data is suitable as a reference image based on the similarity calculated in S2305. If there are no areas with low similarity, the process proceeds to S2308. If there are many areas with low similarity, the device determines that the data is not suitable as a reference image and terminates the process.

[0124] In S2308, the information processing device 104 adds the histogram calculated in S2304 to the histogram of the reference information currently held, recalculates the average value, updates the histogram of the reference information, and terminates the process.

[0125] In S2309, the information processing device 104 compares the similarity of each processing target area set in S422, calculated in S2305, with the threshold set in S428, and records the areas that fall below the threshold as abnormal areas on the RAM 203. In other words, this step is an example of a process that determines whether a change has occurred in a given area based on changes between corresponding areas of multiple acquired images.

[0126] Specifically, in threshold 2001 (Figure 20), if the brightness threshold is set to 22%, the region to be processed will be judged as abnormal if the brightness similarity of the region is less than 22%.

[0127] In S2310, the information processing device 104 compares the similarity of each region (regions separated by a correction grid) that was calculated in S2305 and set as the region to be processed in S2402 with the threshold set in S428, and detects regions that fall below the threshold.

[0128] If an abnormality is detected at this point, proceed to S2311. If no abnormality is detected, proceed to S2312.

[0129] In S2311, the information processing device 104 determines that the area 2600 (Figure 26) where an anomaly was detected is different from the area to be notified to the user as set in S422. Therefore, it considers all areas 2610 (four areas in the case of Figure 26) that overlap with the detected area 2600 as areas where an anomaly was detected (2620) and records them as anomaly locations on the RAM 203.

[0130] As a result, the area that receives notification of an anomaly is the area demarcated by the normal grid 2500 set in S422, regardless of whether grid correction is performed or not. In other words, this step is an example of a process that outputs a determination result based on the area demarcated by the first division method when it is determined that there has been a change in the area demarcated by the second division method.

[0131] In this way, anomalies can always be notified to users and external applications based on areas divided by a fixed grid. In other words, because the notification method is consistent regardless of whether grid correction is applied or not, it is easy for users to understand the notifications, and for external applications, it has the advantage of not requiring complex methods for receiving anomaly notifications, as the receiving logic and output method for anomalies can be kept consistent.

[0132] In S2312, the information processing device 104 compares the abnormal area recorded on the RAM with the mask area set in S423. If there is an abnormal area other than the mask area, the process proceeds to S2313. If there is no abnormal area, or if all of the abnormal areas are mask areas, the process terminates. In other words, this step is an example of a process that controls whether or not to output a judgment result based on the area specified.

[0133] In this embodiment, the histogram calculation process (S2304) is performed even in the masked region, but this method is not limited to this, and the histogram calculation process (S2304) may not be performed in the masked region. By doing so, the number of times the histogram calculation process is performed is reduced when there is a masked region, and the overall processing time can be shortened.

[0134] On the other hand, when performing histogram calculation processing even in masked areas, as in this embodiment, there are advantages such as being able to record anomalies in the masked areas and suggest changes to the masked areas to the user, and being able to stably meet non-functional requirements by always maintaining a constant processing speed.

[0135] In S2313, the information processing device 104 performs an anomaly notification process (Figure 27) and notifies the user and external applications of information about the anomaly area. The abnormality notification process shown in Figure 27 will be explained below.

[0136] In S2701, the information processing device 104 determines whether there is an anomaly to be notified based on judgment criteria such as the continuity of anomaly detection. If it is determined that there is an anomaly to be notified, it proceeds to S2702. If it is determined that there is no anomaly to be notified, it proceeds to S2703.

[0137] Specifically, a criterion is needed to determine whether to notify a system of an anomaly if it is detected in a single image, or if to notify a system of an anomaly if it is detected consecutively in images spanning several seconds.

[0138] For example, the information processing device 104 may accept the user's pre-set criteria for how many consecutive frames of anomaly detection will trigger a notification, or it may determine what the image being analyzed is of (factory, home appliance, plant, etc.) or whether it is indoors or outdoors, and then determine the criteria for notifying an anomaly based on the object and environment.

[0139] In S2702, the information processing device 104 records information about the region where the notification target abnormality has occurred on the RAM 203.

[0140] In S2703, the information processing device 104 deletes the notification-targeted abnormal information recorded on the RAM 203.

[0141] In S2710, the information processing device 104 passes the still image information read in S2302 to the processing unit that draws the settings screen.

[0142] In S2711, the information processing device 104 passes the abnormal information recorded in S2702 to the processing unit that draws the settings screen.

[0143] In S2712, the information processing device 104 determines whether the settings screen is being displayed based on user input. If the settings screen is being displayed, the process proceeds to S2713. If the settings screen is not being displayed, the process terminates.

[0144] In S2713, the information processing device 104 checks the settings 2801 for the detection area drawing color on RAM 203. If there is a specification regarding the drawing method, the process proceeds to S2714. If there is no specification, the process proceeds to S2715.

[0145] In S2714, the information processing device 104 changes the highlighting during the drawing process in S2715, described later, based on the setting 2801 (Figure 28). In other words, this step is an example of a process that identifies and outputs which of several types of thresholds was used to determine that a change had occurred.

[0146] Setting 2801 specifies how to highlight items according to the detection factors for anomaly detection (items that fall below the threshold in S2309). In other words, this step is an example of a process that accepts the specification of how to identify and output each type of threshold.

[0147] Here, to ensure that the highlighting for each threshold is distinguishable, control is implemented to prevent the same highlighting settings from overlapping (e.g., displaying an error message if the same highlighting is selected for different thresholds, preventing the selection of a previously selected highlighting). In other words, this step demonstrates an example of a process that prevents the specification of the same output method for different types of thresholds.

[0148] Specifically, the highlighting method can be set for each type of threshold, such as (1) red when the brightness threshold is exceeded, (2) yellow when the hue threshold is exceeded, (3) green when the saturation threshold is exceeded, (4) blue when the edge gradient strength threshold is exceeded, and (5) purple when the edge gradient angle threshold is exceeded.

[0149] In this embodiment, highlighting is done using color, but this method is not limited to this. Other methods may be used, such as using patterns like diagonal lines, setting the highlighting method to surround items with dotted or thick lines, or setting different notification methods such as voice or email. It is also possible to set notification methods using methods other than highlighting.

[0150] In S2715, the information processing device 104 overlays highlighting on the areas containing the still image information acquired in S2710 and the abnormal information acquired in S2711, and then displays the drawing image in the drawing memory on the RAM 203.

[0151] In S2716, the information processing device 104 draws the drawing image created in S2715 onto the setting screen 2810 (Figure 28). In other words, this step is an example of a process that outputs a result of determining whether or not there has been a change in the area.

[0152] If the drawing method is changed in S2714, the method of highlighting the abnormal region is changed and displayed as shown in abnormal region 2811(1)(2)(4).

[0153] This makes it easy for users to understand what threshold was used to determine that something was abnormal.

[0154] Alternatively, the highlighting of the abnormal area and the display of the masked area may be shown on the same screen (not illustrated).

[0155] Furthermore, although S2312 states that abnormal areas will not be displayed in the masked area, it is also possible to highlight the abnormal detection area (not shown) or highlight it in a different way (not shown) even in the masked area, based on user instructions. Doing so would give the user a reason to decide to remove the mask even in an area designated as a masked area, depending on the nature of the abnormal detection. In other words, it becomes possible to easily configure the abnormal detection desired by the user.

[0156] Furthermore, the information processing device 104 may have a function to notify the user on screens 1810 and 1820 of what kind of abnormality has occurred in the mask area, even without receiving instructions from the user.

[0157] Furthermore, the system may have functions to suggest to the user that they change the mask settings based on the number and nature of anomaly detections, or for the information processing device 104 to automatically change the mask settings. Doing so would enable more accurate anomaly detection. In other words, this step is an example of a process that changes the specified area based on the specified area and the judgment result of that area.

[0158] In S2720, the information processing device 104 notifies the output destination set in S407 (e.g., video monitoring server, mailer, software that can instruct to make calls using automated voice communication) of the abnormal information recorded in S2702.

[0159] Figure 29 shows an example of the anomaly notification dashboard screen, which displays a list of anomalies detected so far in a table format. Here, the detection time, monitoring task name, and coordinates of the detected area are displayed.

[0160] Figure 30 is an example of a flowchart for image difference (anomaly) verification processing. In S3001, the information processing device 104 accepts the setting of an anomaly judgment threshold for each comparison element in order to determine anomalies based on image differences. In other words, this step shows an example of processing that accepts the setting of similarity thresholds for multiple images.

[0161] In S3002, the information processing device 104 reads a reference image and then reads a still image to be compared, and loads it onto the RAM 203 in order to display the location of the abnormality.

[0162] The still image used for comparison can be any captured image, such as the most recently captured image or an image in which an anomaly was detected (a change from the reference image). In addition, if a video file has been split into a series of still images frame by frame (S2105), as in S2301, the generated still images can also be used.

[0163] In step S3003, the information processing device 104 obtains information on the comparison elements, namely Hue, Saturation, and Lightness, from the image data loaded onto the RAM 203.

[0164] In S3004, the information processing device 104 performs the aforementioned histogram calculation processes (Figure 24) to calculate a histogram from the image data.

[0165] In S3005, the information processing device 104 calculates the similarity to the histogram of reference information for each processing area set in S422 and S2402 using histogram intersection, which is one of the histogram comparison methods.

[0166] In S3006, the information processing device 104 compares the similarity of each processing target area set in S422, calculated in S3005, with the threshold set in S3001, and records the areas that fall below the threshold as abnormal areas on the RAM 203. In other words, this step is an example of a process that determines that there has been a change in an area based on the similarity of information relating to multiple images.

[0167] In S3007, the information processing device 104 displays the processing result from S3006. If the grid to be displayed is set to "normal grid + correction grid", it proceeds to S3010. If it is set to "normal grid only", it proceeds to S3008. If it is set to "correction grid only", it proceeds to S3009. Here, "correction grid" is a setting that can be enabled in S2401, and it is a grid that is shifted horizontally and vertically by half the size of one area.

[0168] In other words, this step illustrates an example of a process in which the area that accepts the specification is at least one of a first region obtained by dividing the image into multiple rectangles and a second region obtained by dividing the image using a different method than the first region. Note that the display of the normal grid and the correction grid will be described later in Figure 32, section 3202.

[0169] In S3008, the information processing device 104 compares the similarity of each of the 2500 normal grid regions (Figure 25) within the regions set as the processing target region in S2402, which was calculated in S3005, with the threshold set in S3001, detects regions that fall below the threshold, and designates them as anomaly detection regions.

[0170] In S3009, the information processing device 104 compares the similarity of each correction grid region 2501 (Figure 25) among the regions set as processing targets in S2402, which were calculated in S3005, with the threshold set in S3001, detects regions that fall below the threshold, and designates them as anomaly detection regions.

[0171] In S3010, the information processing device 104 compares the similarity of each of the 2500 normal grid regions (Figure 25) within the regions set as the processing target region in S2402, which was calculated in S3005, with the threshold set in S3001, detects regions that fall below the threshold, and designates them as anomaly detection regions.

[0172] In S3011, the information processing device 104 compares the similarity of each correction grid region 2501 (regions separated by the correction grid) within the region set as the processing area in S2402, which was calculated in S3005, with the threshold set in S3001, and determines whether an anomaly has been detected by detecting regions that fall below the threshold. If an anomaly is detected, the process proceeds to S3012; otherwise, it proceeds to S3013.

[0173] In S3012, the information processing device 104 considers all areas overlapping with the detection area where an anomaly was detected in S3011 (in the case of Figure 26, the four areas 2610 that overlap with the area 2600 where the anomaly was detected) to be an anomaly (2620), and records them as anomaly locations on the RAM 203. In other words, the areas of the normal grid that span the correction grid are designated as anomaly detection areas. At this time, the correction grid where the anomaly was detected and the normal grids that overlap them are displayed with different colors. In this embodiment, the system is configured to consider all areas of the normal grid that span the correction grid as an anomaly area, but it is not limited to this, and it is also possible to set it so that only some of the normal grids that overlap the correction grid are designated as anomaly detection areas.

[0174] The display of the grid will be explained using Figures 3205(a) to (c). In this embodiment, when a grid is selected, it is displayed surrounded by a thick border as in 3205(a), and when it is not selected, it is displayed without a thick border as in 3205(b). By surrounding the periphery of the selected grid with a thick line, the grid selection status is clearly displayed. When selecting a correction grid, it is possible to select the correction grid in the same way as a normal grid by selecting "Correction Grid Only" on the grid settings screen (3202).

[0175] Next, we will explain how to display the area where an anomaly is detected, depending on the display grid selected in S3007. If "Correction grid only" or "Normal grid only" is selected, as described above in S3008 and S3009, the area where an anomaly was detected will be displayed in an identifiable manner within the area demarcated by the selected grid.

[0176] In 3205(a) and (b), the "normal grid + correction grid" is selected, and the area of ​​the normal grid (in S3012, the normal grid that spans the correction grid where an anomaly was detected is also considered to have detected an anomaly, so the normal grid may or may not have detected an anomaly) and the area of ​​the correction grid where an anomaly was detected are displayed in different colors for easy identification.

[0177] 3205(c) is the display when "Normal Grid + Correction Grid" is selected, no abnormalities are detected in the correction grid (judgment result is No in S3011), and abnormalities are detected only in the normal grid. Since no abnormalities are detected in the correction grid, only the normal grid is displayed, filled with a single color.

[0178] As mentioned above, when "Normal Grid + Correction Grid" is selected, the correction grid and the normal grid are displayed in different colors, allowing the user to see at a glance whether an anomaly has been detected by the correction grid.

[0179] In S3013, the information processing device 104 determines whether the display image setting is set to display only the current image (for example, the most recently captured still image), to display only the reference image, or to display the reference image and the current image superimposed. If the display image setting is set to "current image", proceed to S3014; if it is set to "reference image", proceed to S3015. If it is set to "reference image + current image", proceed to S3016. In this embodiment, the configuration displays the "reference image" and the "current image", but it may also be configured to acquire and set an image that is considered to have changed from the reference image. That is, this step shows an example of processing in which the image for which the region specification is accepted by the first receiving means is at least one of the reference image, the image that has been determined to have changed, and the current image from among multiple images. Furthermore, this step shows an example of processing in which the image is a superimposed image of the reference image, the image that has been determined to have changed, or the current image from among multiple images, or at least two of the reference image, the image that has been determined to have changed, and the current image from among multiple images.

[0180] In S3014, the information processing device 104 overlays highlighting on the area containing the still image information acquired in S3002 (in this embodiment, the current image or the image determined to have changed) and the area containing the abnormal information acquired in S3008, S3009, S3010, or S3012, and then expands the drawing image in the drawing memory on the RAM 203. In other words, this step is an example of a process that displays the area determined to have changed in an identifiable manner on the image based on the determination.

[0181] In S3015, the information processing device 104 overlays highlighting on one of the still image pieces of reference information (a reference image in this embodiment) from the reference information updated in S2308, and on the area containing the abnormal information acquired in S3008, S3009, S3010, or S3012, and then expands the drawing image into the drawing memory on the RAM 203.

[0182] In S3016, the information processing device 104 combines one of the still image pieces of reference information to be updated in S2308 with the still image information acquired in S3002 (in this embodiment, the current image or an image that has been determined to have changed), and expands the combined still image information into a drawing image in the drawing memory on the RAM 203.

[0183] In S3017, the information processing device 104 overlays highlighting on the area containing the still image information synthesized in S3016 and the abnormal information acquired in S3008, S3009, S3010, or S3012, and then expands the drawing image into the drawing memory on the RAM 203.

[0184] In S3018, the information processing device 104 displays the drawing images that were loaded into RAM 203 in S3014, S3015, and S3017 on the screen. In other words, it displays the abnormality detection area, which is displayed based on the grid type selected in S3007, on the image selected in S3013. This is a step that shows an example of a process in which the selection of the image division method is accepted, and in accordance with the accepted selection, the area determined to have changed is displayed on the image in an identifiable manner based on the determination of the determination means.

[0185] In S3019, the information processing device 104 accepts the grid selection and displays the calculated histogram as a graph (Figure 31).

[0186] In S3020, the information processing device 104 accepts a termination operation for the image difference (abnormality) confirmation process. If a termination operation (for example, pressing the × button in the upper right corner of 3200 in Figure 32) is accepted, the process in Figure 30 is terminated. If no termination operation is accepted, the process returns to S3001.

[0187] Figure 31 is an example of a flowchart for the graph display process of the selected grid. In S3101, the information processing device 104 checks whether a normal grid or a correction grid (regions 2500 and 2501 (Figure 25)) is selected, and if not, it terminates the process shown in Figure 30. If selected, it proceeds to S3102. In other words, this step is an example of a process that accepts the specification of a region included in the captured image.

[0188] In S3102, the information processing device 104 obtains the position of the selected grid (regions 2500 and 2501 (Figure 25)).

[0189] In S3103, the information processing device 104 graphs and displays the histogram of the reference information updated in S2308 and the histogram calculated in S3004, according to the position of the selected grid acquired in S3102. In other words, this step is an example of a process that displays information relating to multiple images in a specified area as a graph that can identify which of the multiple images the information relates to. In this embodiment, the multiple images are set to two images, but it may also be configured to display information for three or more images. The display of each histogram will be described later in Figure 32.

[0190] Figure 32 shows an example of the analysis settings adjustment screen and the display of each histogram. The analysis settings adjustment screen (3200) displays a reference image in the center of the screen showing the area where an anomaly was detected, the current image, the image determined to have changed (a still image used for comparison with the reference image acquired in S3002), or an image with these superimposed. On the right side of the screen, a histogram calculated for each comparison element is displayed. Furthermore, on the left side of the screen, an area for accepting anomaly judgment threshold settings for each comparison element is displayed. In other words, this screen shows an example of a process that displays settings related to the judgment of whether an image has changed, a process that displays the image and the area determined to have changed in an identifiable manner, and a process that displays information related to the area of ​​the image as a graph.

[0191] In this embodiment, the screen configuration is as described above, but it is not limited to this. The central part of the screen, the histogram section for each comparison element, and the section for setting the anomaly detection threshold may be displayed in different layouts, or each may be displayed on a separate screen.

[0192] In the anomaly detection threshold setting area (3201), the threshold for each element can be changed by operating the slider. When the toggle button for the AND detection item is turned ON, it can be set so that an anomaly is detected only when all of the set thresholds are below the specified threshold. In other words, it can be set to make it less likely to detect an anomaly compared to when AND detection is not enabled. When the threshold is changed by operating the slider, the area that is detected as an anomaly changes accordingly, so the anomaly detection area displayed on the image in the center of the screen changes. That is, this screen shows an example of a process in which the setting display means displays the threshold settings related to the image, and the image display means displays the area that has been determined to have changed based on the change in the threshold set by the setting display means in an identifiable manner.

[0193] In this embodiment, the threshold can be changed by operating a slider, but it is also possible to display the similarity value and scale of the selected grid on the slider, for example. That is, the setting display means is a screen that shows an example of a process that displays the similarity of information related to the image region near the display of the threshold related to the judgment. This allows the user to set the threshold while comparing the threshold and the similarity on the slider, making it easy to adjust the threshold. As a result, the threshold for anomaly detection can be suitably set.

[0194] As mentioned above in the explanation of the analysis settings screen 2000 (Figure 20), in this embodiment, the user sets each threshold. However, this method is not the only way, and the user may select a dataset for each threshold, or the user may set the objects to be monitored or the anomalies to be detected to automatically set the threshold parameters, or the information processing device 104 may determine the objects to be monitored or the anomalies to be detected and automatically set the threshold parameters.

[0195] The grid settings screen (3202) allows you to select whether to display only the normal grid, only the correction grid, or both the normal and correction grids on the image (S3007 in Figure 30). If the normal grid is selected, dotted lines indicating the boundaries of the normal grid are displayed. If the correction grid is selected, dotted lines indicating the boundaries of the correction grid are displayed. If you select to display both the normal and correction grids, dotted lines indicating the boundaries of the normal grid are displayed. Note that it is also possible to configure the system to display the boundaries of both the normal and correction grids, or to display the boundaries of the normal grid and correction grids with lines of different shapes or colors so that they can be distinguished. In other words, this is a step that shows an example of a process in which the system accepts the selection of at least one of a first division method that divides an image into multiple rectangles and a second division method that divides the image in a way different from the first division method, and displays the boundaries of the regions divided by the accepted division method on the image for which the region specification is accepted by the first receiving means.

[0196] On the image display settings screen (3203), you can select whether to display only the current image, only the reference image, or overlay the current image and the reference image for the image displayed in the center (S3013 in Figure 30). This allows the user to easily compare the current image and the reference image and check the results of anomaly detection.

[0197] Using examples 3204(a) and 3204(b), histograms calculated for each comparison element will be explained. The numbers displayed on the histogram represent the similarity between the reference image of the selected grid and the current image. In other words, this is an example of a process in which the graph display means displays the similarity of information related to the image region. In this embodiment, the similarity numbers displayed on the histogram are displayed in red to stand out if the selected grid is detected as abnormal. It is sufficient for the user to be able to identify whether the selected grid has been detected as abnormal; other methods such as bold text, borders, or pop-ups may also be used for highlighting.

[0198] The calculated histogram shows the distribution of each comparison element between the reference image and the current image. In this embodiment, the histogram of the reference image is displayed in green, the histogram of the current image in yellow, and the overlapping portion of the histograms of the reference image and the current image in blue. In other words, this screen shows an example of a process that displays graph elements of information related to each of multiple images and graph elements where information from multiple images overlap in a distinguishable manner. In this embodiment, the information of each image is displayed in a distinguishable manner by using different colors, but it is sufficient if the user can identify which image the information relates to, and other methods such as distinguishing by patterns or including a legend may also be used.

[0199] As described above, by displaying information for each comparison element between the reference image and the current image in a way that is identifiable by color, pattern, etc., users can easily compare the information of the reference image and the current image and make it easier to decide whether to loosen or tighten the threshold. For example, if, after checking the histogram, it can be seen that the brightness of the selected grid is higher in the current image compared to the reference image, and after checking the image and determining that the reason for the higher brightness is not abnormal (e.g., it is exposed to direct sunlight), it becomes possible to loosen the threshold. In this way, the threshold for anomaly detection can be set appropriately.

[0200] An example histogram, 3204(a), is used to explain the histogram of edge gradients and how to calculate it. In the embodiment shown in Figure 32, the edge gradient intensity and edge gradient angle items in Figure 20 are displayed as a single histogram. First, the brightness difference between points diagonally opposite each other is calculated, centered on an arbitrary point in the grid (for example, 1 pixel). A virtual coordinate is set by plotting the brightness difference on an XY Cartesian coordinate system, and the angle (arc) of the virtual coordinate, obtained by an inverse trigonometric function, and the intensity, which is the length of the straight line between the virtual coordinate and the origin, are calculated. The calculated angle and intensity are averaged, and the range is divided into a certain number of segments (8 in this embodiment) based on the angle, and an intensity histogram is created within that classified range.

[0201] In other words, it shows the edge gradient strength for each of the eight virtual edge gradient angles, allowing the user to see at a glance how similar the reference image and the current image are in terms of intensity for each edge gradient angle.

[0202] The histograms for hue, saturation, and brightness will be explained using an example histogram, 3204(b). For the hue histogram, the RGB values ​​(numerical indicators of the intensity of the primary colors red, green, and blue) of any point in the grid are converted into a hue circle, and a histogram is created with a 360-degree horizontal axis. In other words, the hue histogram shows the number of points in the grid for each color in the 360-degree hue circle. This allows the user to check the distribution of hues in the reference image and the current image and set thresholds. However, in this embodiment, if the brightness is below a certain level and color cannot be distinguished, it may be displayed as "not measured" (hue similarity 100%) as shown in the hue diagram of example histogram 3204(a).

[0203] For the saturation histogram, the saturation is calculated from the RGB values ​​of any point in the grid, and the histogram is created based on this value. The horizontal axis of the histogram represents saturation, and the vertical axis represents the number of any given point in the grid.

[0204] For the luminance histogram, the luminance is calculated from the RGB values ​​of any point in the grid, and a histogram is created based on these values. The horizontal axis of the histogram represents luminance, and the vertical axis represents the number of any point in the grid. In other words, this is a step that demonstrates an example of processing where the information related to the image includes at least one of the following: edge gradient, hue, saturation, and luminance.

[0205] As a result, users can visually understand the threshold used to determine an anomaly, or how similar the image is to a reference image. This allows for the appropriate setting of thresholds in anomaly detection.

[0206] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium containing a program that realizes the functions of the embodiments described above to a system or device, and by having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0207] In this case, the program read from the recording medium itself realizes the novel function of the present invention, and the recording medium on which that program is recorded constitutes the present invention.

[0208] For recording media used to supply programs, examples include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, and the like.

[0209] Furthermore, it goes without saying that the functions of the aforementioned embodiments are realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of that program, thereby realizing the functions of the aforementioned embodiments.

[0210] Furthermore, it goes without saying that this also includes cases where, after a program read from a recording medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer, the CPU or other components of the function expansion board or function expansion unit perform some or all of the actual processing based on the instructions of the program code, and the functions of the aforementioned embodiments are realized through that processing.

[0211] Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device. It goes without saying that the present invention can also be applied when the results are achieved by supplying a program to a system or device. In this case, by reading a recording medium containing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

[0212] The above program may consist of object code, program code executed by an interpreter, script data supplied to the OS (operating system), and other forms.

[0213] Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that all configurations combining the above-described embodiments and their modified forms are also included in the present invention. [Explanation of symbols]

[0214] 100 Information Processing Systems 102 Cameras 104 Information Processing Device

Claims

1. A setting display means that displays settings related to the determination of whether or not there has been a change in the image, Image display means that displays the aforementioned image and the region in which a change has been determined to have occurred in the aforementioned image in an identifiable manner, A graph display means that displays information relating to the region of the aforementioned image as a graph, An information processing system characterized by comprising the following features.

2. The setting display means displays the threshold setting for the image, and the image display means displays the region where a change has been determined to have occurred based on the change in the threshold set by the setting display means in an identifiable manner. The information processing apparatus according to claim 1, characterized by the following:

3. The region is at least one of a first region obtained by dividing the image into multiple rectangles and a second region obtained by dividing the image using a different method than the first region. The information processing system according to claim 1, characterized by the following:

4. The information relating to the region of the aforementioned image includes at least one of edge gradient, hue, saturation, and brightness. The information processing system according to claim 1, characterized by the following:

5. The graph display means displays graph elements of information relating to each of the multiple images and graph elements where the information of the multiple images overlaps in a distinguishable manner. The information processing system according to claim 1, characterized by the following:

6. The graph display means displays the similarity of information relating to the region of the image. The information processing system according to claim 1, characterized by the following:

7. The setting display means displays the similarity of information relating to the region of the image near the display of the threshold related to the determination. The information processing system according to claim 1, characterized by the following:

8. The graph display means further includes a receiving means for receiving a specification of a region included in an image displayed by the image display means, and the graph display means displays the information relating to the region of a plurality of images corresponding to the region received by the receiving means as a graph that can identify which of the plurality of images the information relates to. The information processing system according to claim 1, characterized by the following:

9. The aforementioned image is at least one of the images in which a change was determined and the current image. The information processing system according to claim 8, characterized by the following:

10. The setting display means of the information processing system includes a setting display step that displays the setting related to the determination that there has been a change in the image, An image display step in which an image display means of an information processing system displays the image and the region in the image where it has been determined that a change has occurred in the image in an identifiable manner, The graph display means of the information processing system includes a graph display step in which it displays information relating to the region of the image as a graph, A control method for an information processing system, characterized by comprising the following:

11. A program for causing at least one computer to function as one of the means of an information processing system described in any one of claims 1 to 9.

Images