Video processing system, video processing method, and video processing program

Abstract

To provide a video processing system, a method, and a program for improving analysis accuracy during actual operation of a behavior analysis system.SOLUTION: A video processing system includes: setting means for setting a category of an object detected from video captured by a camera; first input means for accepting input of the name of the category of the object included in the video that is different from the set category by operator's operation; second input means for accepting input of a symbol frame that specifies an area corresponding to the different category by the operator's operation on the video; third input means for accepting input that associates the different category with a partial section of the video by operator's operation on a display showing the relationship between the category of the object included in the video and the section in the video; and data storage means for storing the name of the different category in association with the position of the symbol in the video, in order to have an image analysis module learn the data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for analyzing video from a surveillance camera. [Background technology]

[0002] In the above technical field, Patent Document 1 discloses a technology that uses real-time learning to eliminate the need for prior knowledge and prior learning of a behavior recognition system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2008 / 098188 publication Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology described in the above document, behavior recognition is machine-learned, and certain behaviors are characterized as normal or abnormal based on past observations of similar objects. This means that the system operator does not actively intervene or support the system, and classifier learning cannot be performed during operation. In other words, the analysis accuracy of the behavior analysis system cannot be improved during actual operation.

[0005] An object of the present invention is to provide a technique for solving the above-mentioned problems. [Means for solving the problem]

[0006] In order to achieve the above object, a video processing system according to the present invention comprises: A setting means for setting a category of an object detected from an image captured by a camera; a first input means for receiving, by an operator's operation, an input of a name of a category of an object included in the video that is different from a preset category; a second input means for receiving an input of a frame of a figure that specifies an area corresponding to the different category by an operation of an operator on the image; a third input means for receiving an input for associating the different category with a part of the section of the video by an operator's operation on a display showing a relationship between a category of an object included in the video and a section of the video; a data storage means for storing names of the different categories and positions of the figures in the video in association with each other so that the names can be learned by an image analysis module; The video processing system includes:

[0007] In order to achieve the above object, a video processing method according to the present invention comprises: Set the category of objects detected from the camera footage, receiving, through an operator's operation, an input of a name of a category of an object included in the video that is different from a preset category; accepting input of a frame of a figure designating an area corresponding to the different categories by an operation of an operator on the image; receiving an input for associating the different categories with a portion of a section of the video by an operator's operation on a display showing a relationship between a category of an object included in the video and a section of the video; storing names of the different categories in association with positions of the figures in the video for learning by an image analysis module; A video processing method.

[0008] In order to achieve the above object, a video processing program according to the present invention comprises: A process of setting a category for objects detected from the video captured by the camera; A process of receiving, through an operator's operation, an input of a name of a category of an object included in the video that is different from a preset category; receiving input of a graphic frame that specifies an area corresponding to the different categories by an operation of an operator on the image; receiving an input for associating the different categories with a portion of the video by an operator's operation on a display showing a relationship between a category of an object included in the video and a section in the video; storing names of the different categories in association with positions of the figures in the video for learning by an image analysis module; It is a program that causes a computer to execute the above. Effect of the Invention

[0009] According to the present invention, it is possible to effectively and efficiently improve the accuracy of video analysis during actual operation of a surveillance system. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a video processing device according to a first embodiment of the present invention. [Figure 2A] 1 is a block diagram showing a configuration of a video monitoring system according to a prerequisite technique of the present invention; [Figure 2B] 1 is a block diagram showing a configuration of a video monitoring system according to a prerequisite technique of the present invention; [Diagram 3] 4 is a flowchart showing a process flow of the video monitoring system according to the base technology of the present invention. [Figure 4] 1 is a flowchart showing the flow of a learning process of a video surveillance system according to a prerequisite technique of the present invention. [Diagram 5] FIG. 11 is a block diagram showing a configuration of a video monitoring system according to a second embodiment of the present invention. [Figure 6A] FIG. 11 is a diagram showing the configuration of a category table used in the video surveillance system according to the second embodiment of the present invention. [Figure 6B] FIG. 11 is a diagram showing the configuration of a category table used in the video surveillance system according to the second embodiment of the present invention. [Figure 6C] 13 shows the contents of category information sent from a video monitoring operation terminal group to a learning video extraction unit in a video monitoring system according to a second embodiment of the present invention. [Figure 7A] 13A and 13B are diagrams showing examples of a display image of a video monitoring system according to a second embodiment of the present invention. [Figure 7B] FIG. 11 is a diagram showing an example of an occurrence event table for each camera stored in the video surveillance system according to the second embodiment of the present invention. [Figure 8] 13A and 13B are diagrams showing examples of a display image of a video monitoring system according to a second embodiment of the present invention. [Figure 9A] 10 is a flowchart showing the flow of a learning process of a video monitoring system according to a second embodiment of the present invention. [Figure 9B] 10 is a flowchart showing the flow of a learning process of a video monitoring system according to a second embodiment of the present invention. [Figure 10] FIG. 11 is a block diagram showing a configuration of a video monitoring system according to a third embodiment of the present invention. [Figure 11] FIG. 13 is a block diagram showing the contents of an incentive table of the video surveillance system according to the third embodiment of the present invention. [Figure 12] 13 is a flowchart showing the flow of a learning process of a video monitoring system according to a third embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing an example of a display image of a video monitoring system according to a fourth embodiment of the present invention. [Figure 14] 13 is a flowchart showing the flow of a learning process of a video monitoring system according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiment are merely examples, and are not intended to limit the technical scope of the present invention.

[0012] [First embodiment] A video processing device 101 according to a first embodiment of the present invention will be described with reference to Fig. 1. As shown in Fig. 1, the video processing device 101 includes a video data storage unit 121, a video analysis unit 111, a display control unit 123, and a learning data storage unit 140.

[0013] The video data storage unit 121 stores video data captured by the surveillance camera 102. The video analysis unit 111 analyzes the video data stored in the video data storage unit 121 to detect events belonging to specific category information, and outputs the detection result. The display control unit 123 also displays a category information setting screen for setting category information of events included in the video together with the video of the video data stored in the video data storage unit 121. The learning data storage unit 140 stores the category information set in response to the operation of the operator 180 on the category information setting screen and the video data in which the category information is set, as learning data. Then, the video analysis unit 111 performs learning processing using the learning data stored in the learning data storage unit 140.

[0014] According to this embodiment, it is possible to effectively and efficiently improve the accuracy of video analysis during actual operation of a surveillance system.

[0015] [Second embodiment] The second embodiment of the present invention relates to a technology that collects learning videos of detection targets of a video detection engine by category, and uses the collected learning videos to create new modules and improve the accuracy of existing modules. Note that in the following explanation, the term "video" is used as a concept that does not only mean moving images, but also includes still images.

[0016] (Prerequisite technology) First, a prerequisite technique for a video surveillance system according to a second embodiment of the present invention will be described with reference to Figures 2A to 4. Figures 2A and 2B are diagrams for explaining a video surveillance system 200 as a prerequisite technique for this embodiment.

[0017] As shown in FIG. 2A, the video monitoring system 200 includes a data center 201 and a group of monitoring cameras 202. The data center 201 includes a video analysis platform 210 and a video monitoring platform 220, and further includes a group of multiple video monitoring operation terminals 232. This figure shows a video monitoring room 230 equipped with a group of multiple video monitoring operation terminals 232. In the video monitoring room 230, an operator 240 checks the monitored video while watching the two-screen monitors equipped on each terminal of the group of video monitoring operation terminals 232. Here, an example is shown in which a 16-screen split display is performed on the left screen and a single-screen enlarged display is performed on the right screen. However, the present invention is not limited to this, and any display may be used. For example, the left and right may be reversed, and the number of divisions on each screen may be any number. A plurality of large monitors 231 are provided on the front wall of the video monitoring room 230, and problematic video or still images are displayed. In such a video monitoring room 230, for example, 200 operators 240 take turns monitoring the footage from 16 cameras each, and continue to monitor the footage from all 2000 cameras 24 hours a day, 360 days a year. While watching the footage from the 16 surveillance cameras assigned to them, the operators 240 find events that should be detected, such as reckless cars, dangerous objects such as guns and knives, theft, snatching, running away from home, assault, murder, drug dealing, and problematic behaviors such as trespassing, and similar objects and behaviors (e.g., suspicious people and crowd movements), and report them to the supervisor. The supervisor rechecks the footage, contacts the police or hospitals as necessary, and cooperates in rescuing victims and arresting criminals.

[0018] The video surveillance platform 220 is called a VMS (Video Management System), and stores video data acquired from the surveillance cameras 202 and distributes it to the video surveillance operation terminals 232. As a result, the video surveillance operation terminals 232 display the video data in real time according to a predetermined allocation rule. In response to a request from an operator 240 who operates the video surveillance operation terminals 232, the video surveillance platform 220 selects one of the surveillance cameras 202 and sends a PTZ (pan-tilt-zoom) operation instruction.

[0019] The video analysis platform 210 performs an analysis process on the video data stored in the video monitoring platform 220, and if video data that meets the conditions is found, it transmits category information specifying the target video data to the video monitoring platform 220. The video monitoring platform 220 generates an alert screen according to the category information received from the video analysis platform 210, and notifies a predetermined terminal of the video monitoring operation terminal group 232. In some cases, the problematic video is forcibly enlarged and displayed on the large monitor 231.

[0020] Fig. 2B is a block diagram showing a detailed configuration of the video surveillance system 200. As shown in Fig. 2B, the video surveillance platform 220 collects video data 250 from the surveillance cameras 202, adds the collection time, camera position, camera ID, and other information, and stores the data in a video storage 221. In addition, the camera selection operation unit 222 receives camera designation and PTZ (pan-tilt-zoom) operation instructions from the operator 240 via the video surveillance operation terminals 232, and operates the designated surveillance camera.

[0021] The video surveillance platform 220 includes a display control unit 223 that displays an alert on the video surveillance operation terminal group 232, and a video reading processing unit 224 that plays / edits past video stored in the video storage 221 in response to instructions from the video surveillance operation terminal group 232.

[0022] The video analysis platform 210 includes predefined video analysis modules 211. Each video analysis module is configured with an algorithm and / or parameters for detecting different types of problematic video. These predefined video analysis modules 211 use predefined algorithms and parameters to detect video including predefined events, and transmit predefined category information for the detected video data to the video surveillance platform 220.

[0023] 3 is a flowchart for explaining the flow of processing in the data center 201. In step S301, the video monitoring platform 220 receives the video data 250 from the group of monitoring cameras 202.

[0024] Next, in step S 302 , the video monitoring platform 220 stores the received video data 250 in the video storage 221 , and transmits it to the video monitoring operation terminal group 232 and the video analysis platform 210 .

[0025] Next, in step S303, the camera selection operation unit 222 receives camera selection information and camera operation information from the operator 240 via the video monitoring operation terminal group 232, and transmits an operation command to the selected monitoring camera.

[0026] On the other hand, in step S304, the video analysis platform 210 performs an analysis process of the video data received from the video monitoring platform 220 using the default video analysis module 211.

[0027] Then, in step S305, if the default video analysis module 211 detects a video that satisfies the predetermined conditions, the process proceeds to step S307, and transmits the category information to the video surveillance platform 220. Even if the default video analysis module 211 does not detect a video that satisfies the conditions, the process proceeds to step S310, and transmits the category information "no category information" to the video surveillance platform 220.

[0028] Furthermore, in step S308, an alert screen is generated in the display control unit 223 of the video monitoring platform 220 and transmitted to the video monitoring operation terminal group 232 together with the video from the target surveillance camera.

[0029] In step S309, an operation by the operator 240 on the alert screen (an operation to report to a supervisor or the police) is accepted.

[0030] FIG. 4 is a flowchart for explaining the flow of processing when generating the default video analysis module 211. The video analysis module generation processing here is performed before the data center 201 is constructed on-site. First, in step S401, a large amount of video including an event to be detected is extracted by human eyes from a large amount of past video. Alternatively, in step S402, an event similar to the event to be detected is intentionally generated in an environment similar to the actual operation environment, filmed, and sample video is extracted. In step S403, additional information is manually added to each of the large amount of extracted / collected video data to create a learning video. Furthermore, in step S404, a researcher / engineer selects an optimal algorithm for the target object, event, and action, and generates the default video analysis module 211 by having the learning video data learn the algorithm.

[0031] (Issues with prerequisite technology) When creating a default video analysis module using the above prerequisite technologies, it required a huge amount of man-hours and time to collect and assign correct answers. For example, 2,000 images were required for face recognition, and 1 million images were required for specific event detection using deep learning, making the introduction of these technologies a high hurdle. In other words, the creation of a video analysis module (classifier) ​​from learning video was done manually, and during this process, the operation of the video analysis module was also verified separately, and the environment was prepared separately, which required man-hours and time.

[0032] In response to this, in recent years, as the types of crimes and accidents have become more diverse, there has been an increasing demand from customers for additional processing of detectable events. In addition, with a default video analysis module that only applies learning videos collected in an environment different from the surveillance environment, the detection accuracy of problematic videos can be significantly reduced depending on the actual video surveillance environment. Furthermore, it takes a lot of work and a long time to adapt to the actual surveillance environment.

[0033] (Configuration of this embodiment) FIG. 5 is a block diagram showing a configuration of a video surveillance system 500 as an example of a surveillance information processing system according to this embodiment. The same components as those in the base technology are given the same reference numerals, and the description thereof is omitted. Unlike the base technology shown in FIG. 2B, a data center 501 as an example of a video processing device according to this embodiment includes a learning database 540 and a video surveillance operation terminal 570. The learning database 540 accumulates learning video data 560 to which category information 561 selected by an operator 240 is added. The video surveillance operation terminal 570 is a terminal for a supervisor 580 to specify category information. The video analysis platform 510 includes a new video analysis module 511 that is newly created, a category information adding unit 515, and a new video analysis module generating unit 516. On the other hand, the video surveillance platform 520 includes a new learning video extraction unit 525.

[0034] Category information is information that indicates the classification of objects and actions to be detected in a video. Examples of category information include "gun," "knife," "fight," "runaway," "double riding on a motorcycle," and "drug deal." Video analysis platform 510 includes category information tables 517 and 518, in which various types of category information and their attributes are associated and stored.

[0035] 6A and 6B are diagrams showing the contents of the default category information table 517 and the new category information table 518. The category information tables 517 and 518 store category information types, shapes, trajectory information, size thresholds, etc. in association with category information names. By referring to these, the video analysis modules 211 and 511 can determine the category of an event included in the video data.

[0036] FIG. 6C shows the contents of category information 531 sent from video monitoring operation terminal group 232 to learning video extraction unit 525. As shown in FIG. 6C, category information 531 includes registered "category", "camera ID", "video shooting time", "event area", "operator information", and "category type". Here, "category" refers to the name of the object to be detected and the classification of the action. "Camera ID" refers to an identifier for identifying a surveillance camera. "Video shooting time" refers to information indicating the date and time when the video data to which the category information should be added was shot. It may also refer to a specific period (start to end of collection). "Event area" refers to information indicating the "target shape" in the video, "video background difference", and "position of the background difference in the entire video". Event areas are not limited to rectangular areas, but various types such as masked video, background difference video, and polygonal video are prepared. "Operator information" refers to information on the operator who added the category information, and includes operator ID and name. Category type refers to the type of object / event to be detected. For example, the category information "gun" is a category type that accumulates the "shape" of the target gun as learning video data. The category "runaway" is a category information type that accumulates learning video data with the trajectory from the start point to the end point of the background difference of a specified area as the "action." Furthermore, the category information "drug trafficking" is a category information type that accumulates learning video data with the trajectory of the background difference in the entire video as the "action."

[0037] Returning to Fig. 5, when the category of an event to be newly detected can be increased by adjusting or adding parameters in the already existing video analysis modules 211, 511, the category adding unit 515 adjusts or adds parameters to those video analysis modules 211, 511. On the other hand, when it is determined that the category of an event to be newly detected cannot be increased by adjusting or adding parameters to the already existing video analysis modules 211, 511, a new video analysis module generating unit 516 is used to generate a new video analysis module 511. In addition, learning video data 560 accumulated in learning database 540 is allocated to the video analysis modules 211, 511 selected based on category information 561, and each video analysis module is made to perform learning processing.

[0038] The display control unit 523 generates a category information selection screen 701 as shown in FIG. 7A and sends it to the video monitoring operation terminal group 232. The category information selection screen 701 includes "Other" 711 as a choice of category information in addition to category information prepared in advance (for example, "no helmet", "two people riding a motorcycle", "speeding", "gun", "knife", "drug deal", "fight", etc.). Here, in order not to lower the motivation of the operator 240 to select category information, it is preferable to display only a part of category information candidates (for example, five) predicted in advance by referring to the camera event table 702 shown in FIG. 7B on the category information selection screen 701. The camera event table 702 accumulates category information of events that are likely to be included in the video captured by the camera for each camera ID 721. That is, the category 722 of the event that occurred and its occurrence rate 723 are stored in order from the highest occurrence rate. The category information related to the selected category information is accumulated in the learning database 540 together with the video data from which the category information was selected.

[0039] The learning video extraction unit 525 separately accumulates the video data for which "Other" 711 has been selected in the learning database 540 so that the supervisor 580 can input specific category information via the video monitoring operation terminal 570 as appropriate. When "Other" is selected, a category information input request is sent to the video monitoring operation terminal 570 for the supervisor 580 together with video identification information indicating the video data at that time. When the operator 240 performs a category information selection operation during video monitoring via the video monitoring operation terminal group 232, a set 531 of video identification information and category is accumulated in the learning database 540 via the video monitoring platform 520. Meanwhile, the display control unit 523 transmits a new category information generation screen to the video monitoring operation terminal 570 to prompt the supervisor 580 to perform new category information generation processing.

[0040] Fig. 8 is a diagram showing a specific example of such a new category information setting screen 801. In this new category information setting screen 801, for example, in addition to a category information name input field 811 and a category information type selection field 813, an area designation graphic object 812 for designating an area to focus on in order to detect a video included in the category information is prepared. In addition to the new category information setting screen 801 shown in Fig. 8, information for identifying the operator who selected the "Other" category information, the location and time at which the video was acquired, etc. may be displayed. It may be determined by video analysis which category information the video categorized as "Other" is similar to, and this may be presented to the supervisor 580.

[0041] The new video analysis module generation unit 516 selects an existing algorithm that fits the new category information, and creates a new video analysis module using a neural network or the like according to that algorithm. It then learns the module using the accumulated learning video data. For the learning and application process, it is possible to select either batch processing or on-the-fly processing according to the category information.

[0042] The category information addition unit 515 performs batch processing or real-time processing, and registers information regarding the added category information in a category information table 518, and the existing video analysis module 211 or the new video analysis module 511 specifies the category information to be referenced in the category information tables 517, 518.

[0043] The default video analysis module 211 and the new video analysis module 511 perform learning processing using the respective learning video data based on the specified category information. This improves the video analysis accuracy of the default video analysis module 211, and the new video analysis module 511 is completed as a new video analysis module.

[0044] If there is no existing algorithm that fits the new category, the new video analysis module generation unit 516 may automatically generate a new algorithm (for example, recognizing a person behind as a person even if multiple people pass in front of them).

[0045] (Processing flow) 9A and 9B are flowcharts for explaining the flow of processing by the video monitoring system 500. Steps S301 to S309 are similar to the processing of the base technology explained in Fig. 3, so the explanation will be omitted here and only steps S900 to S911 after step S309 will be explained.

[0046] In step S309, the learning video extraction unit 525 examines the video, and when an alert is generated indicating that the video contains an event that should be detected, the process proceeds to step S900, and the display control unit 523 displays a category information selection screen 701 as shown in FIG. 7A on the video monitoring operation terminal group 232.

[0047] In step S901, the operator 240 selects category information. If the category information selected here is specific category information, the process proceeds to step S902, where the learning video extraction unit 525 generates category information and assigns it to the video data to generate learning video data.

[0048] Next, in step S903, the learning video extraction unit 525 accumulates the generated learning video data in the learning database 540, and further in step S904, the predetermined video analysis module 211 performs learning processing.

[0049] On the other hand, if in step S901 the operator 240 selects the "Other" category information, the process proceeds to step S905, where the learning video extraction unit 525 generates category information with the category information name "Other" and the category information type "NULL", and assigns it to the learning video data.

[0050] Next, in step S906, the learning video extraction unit 525 stores the learning video data with the category information "Other" added in the learning database 540, and at the same time, the display control unit 523 sends the learning video data and new category information setting screen 801 to the video monitoring operation terminal 570 of the supervisor 580.

[0051] In step S907, the category adding unit 515 receives an instruction from the supervisor 580 to set new category information and associates it with the accumulated learning video data.

[0052] In step S908, category information adding section 515 determines whether or not there is a default video analysis module 211 that fits the set new category information. If there is a default video analysis module 211 that fits the set new category information, the process proceeds to step S909, where it is set as new category information for the target default video analysis module 211, and further in step S904, learning is performed using the learning video data with the new category information added.

[0053] On the other hand, if it is determined in step S908 that there is no predefined video analysis module 211 that fits the newly set category information, the process proceeds to step S911, where a new video analysis module 511 is generated and trained on the training video.

[0054] FIG. 9B is a flowchart showing a detailed flow of the new video analysis module generation process in step S911. In step S921, an algorithm database (not shown) is referenced. In step S923, an algorithm (e.g., an algorithm for extracting a feature vector, an algorithm for extracting a cluster consisting of a set of blobs (small image regions) and the boundary of the cluster) according to the category information type specified by the supervisor is selected, and the framework of the video analysis program module is generated. Next, in step S925, a video region to be analyzed by the video analysis module is set using a judgment region specified by the supervisor. Furthermore, in step S927, a plurality of learning video data are used to determine thresholds for the shape and size of an object to be detected, and the direction and distance of a movement to be detected. At this time, a feature vector of an object to be detected and its movement in the learning video data may be extracted, and the feature amount may be set as the threshold. When a cluster consisting of a set of blobs and the boundary of the cluster are extracted as a feature amount, the feature amount may be set as the threshold.

[0055] As described above, according to this embodiment, an operator can easily accumulate learning videos during actual operation and simultaneously associate categories with them. This allows for semi-automatic collection of learning videos and association with categories, and reduces the man-hours and time required for localization to an environment and generation of a new video analysis module.

[0056] In addition, since the system can learn from the images of the operating environment, it is possible to build a more accurate image analysis module. This technology can be applied to the security field, such as video surveillance and guarding. It can also be applied to customer-oriented analysis in stores and public areas using video.

[0057] [Third embodiment] Next, a video monitoring system according to a third embodiment of the present invention will be described. The video monitoring system according to this embodiment differs from the second embodiment in that it takes into consideration incentives for operators. Other configurations and operations are the same as those of the second embodiment, so the same configurations are given the same reference numerals and detailed descriptions are omitted here.

[0058] 10 is a block diagram showing the configuration of a video monitoring system 1000 according to this embodiment. Unlike the second embodiment, a video monitoring platform 1020 included in the video monitoring system 1000 includes an incentive table 1026. The incentive table 1026 collects statistics on the number of learning videos stored in the learning database 540 for each operator and provides incentives to improve collection efficiency.

[0059] An example of the incentive table 1026 is shown in FIG. 11. The incentive table 1026 stores and manages the number of learning videos 1102, the number of new categories 1103, and points 1104, linked to an operator ID 1101. The number of learning videos 1102 indicates the number of learning video data for which the operator selected and assigned a category. The number of new categories 1103 indicates the number of categories for which the operator selected "other" as a category and which were ultimately generated as new categories by the supervisor. Since the number of learning videos and the number of new categories can be evaluated as the contribution value of the operator's monitoring work, points 1104 are calculated according to these values, and are also stored and updated while linked to the operator ID. The importance of the video discovered by the operator and assigned a category may be taken into consideration and weighted when calculating the points 1104.

[0060] By determining the hourly wage or salary of the operator according to the value of this point 1104, it is possible to motivate the operator in the monitoring work. In the incentive table 1026, a value indicating the accuracy of category assignment may be used as a value for evaluating the operator, other than points. For example, test video data assigned with the correct category may be shown to a plurality of operators, and the detection speed, detection accuracy, and category correct probability of each operator may be verified, and the operator evaluation value may be calculated using these.

[0061] 12 is a flowchart explaining the flow of processing in the video monitoring system 1000. When a default video analysis module is made to learn or a new video analysis module is generated by category selection or category generation, the process proceeds to step S1211, where points are awarded. That is, for example, an incentive (points) according to the category of the event discovered by the operator is associated with the operator ID who performed the categorization and saved.

[0062] According to this embodiment, it is possible to stimulate the operator's motivation for monitoring.

[0063] [Fourth embodiment] As an example of this method, we explain a system that enables operators to add and modify category information (Fig. 2B), which is the prerequisite technology for the monitoring information system.

[0064] The display control unit 523 generates a category information selection screen 1301 as shown in FIG. 13 and sends it to the video monitoring operation terminal group 232. The category information selection screen 1301 includes a video display screen 1303, a category information bar 1302 indicating the occurrence status of an alert, video control parts 1304 (for example, "play", "stop", "pause", "rewind", "fast forward", etc.), a progress bar 1307 indicating the video playback status, and a category information setting button 1305. The category information setting button 1305 includes "other" 1306 as a category information option in addition to the category information prepared in advance. The operator 240 checks the video using the video control parts 1304. The operator 240 modifies and adds category information of the video data displayed on the display screen 1303 using the category information setting button 705. The category information related to the category information set by the operator 240 is stored in the learning database 540 together with the video data.

[0065] Fig. 14 is a flowchart for explaining the flow of processing by video monitoring system 500. Steps S301 to S309 are similar to the processing of the base technology explained in Fig. 3, so explanations will be omitted here, and steps S900 to S910 after step S309 will be explained. Also, steps S900 and steps S902 to S910 are similar to the processing explained in Fig. 9A, so explanations will be omitted here, and step S1401 will be explained. In step S309, the learning video extraction unit 525 examines the video, and when an alert is generated indicating that the video contains an event that should be detected, the process proceeds to step S900, and the display control unit 523 displays a category information selection screen 1301 as shown in FIG. 13 on the video monitoring operation terminal group 232.

[0066] On the category information selection screen 1301, the video and its category information are displayed on a display unit 1303 and a category information bar 1302. In addition, by using a video control component 1304, the category information and the video can be checked by playing, rewinding, and fast-forwarding. The category information bar 1302 displays, in color, the category information generated for the video displayed on the category information selection screen 1301 and the correct category information corrected or added by the operator 240. For example, a section where an alert of "no helmet" has occurred is displayed in blue, a section where no alert has occurred or a section 1310 where there is no category information is displayed in black, and a section where an alert of "two people riding on a motorcycle" has occurred is displayed in red. In addition, the correct category information corrected or added by the operator 240 using a category information setting button 1305 is also displayed in the same manner.

[0067] In step S1401, the operator 240 checks the video in which an alert has occurred and its category information using the category information selection screen 1301, and modifies or adds category information for any section that requires such modification or addition.

[0068] When modifying or adding category information, a category information setting button 1305 corresponding to a specific category information is pressed while playing back a portion of the video. For example, consider modifying a section where a "no helmet" alert has occurred to an alert for "two-person motorcycle." Here, the section that belongs to the category information of "no helmet" is displayed in blue in the category information bar 1302. Using the video control parts 1304 on the category information selection screen 1301, play back the video on the display unit 1303, and when the corresponding video data is displayed on the display unit 1303, press the "two-person motorcycle" button of the category information setting buttons 1305 to modify the category to "two-person motorcycle." At this time, the color of the category information bar 1302 corresponding to the section for which category information has been set by the operator 240 changes from "blue" to "red."

[0069] It is also possible to add multiple types of category information to the same section using video control components 1304. In this case, the category information bar 1302 for a section to which multiple types of category information have been added is displayed in multiple layers of colors. For example, if "no helmet" and "two people riding on a motorcycle" are added, blue and red are displayed in layers, as in the category information bar 1309.

[0070] Furthermore, if you wish to delete category information, simply use the video control component 1304 to play back the video of the section for which you wish to delete category information, and press the "no category information" button 1308. This is the same as correcting the category information to "no category information." The category information bar 1310 for the section corrected to "no category information" is displayed in black, just like a section in which no alert has occurred.

[0071] Here, when modifying or adding category information for a certain continuous section of video, while playing the video of that section, it suffices to keep pressing a specific button of category information setting buttons 1305. Furthermore, if the section for which you want to modify or add category information is long, you may make category information setting button 1305 a toggle type so that you press the button only at the beginning and end of the section for which you want to modify or add category information.

[0072] In step S1401, if the operator has corrected or added the correct category information, the video analysis module 211 can learn the alert that should be output.

[0073] If the corrected / added category information is specific category information, the process proceeds to step S902, where learning video extraction section 525 adds the category information to the video data, thereby generating learning video data.

[0074] By having an operator check the category information from the video and the surveillance information system and set the correct category information, a surveillance information system that can detect new objects and actions can be constructed. Also, by having an operator correct the category information, the accuracy of the surveillance information system can be improved.

[0075] [Other embodiments] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In addition, systems or devices that combine the separate features included in each embodiment in any way are also included in the scope of the present invention.

[0076] The present invention may be applied to a system consisting of multiple devices, or to a single device. Furthermore, the present invention is also applicable to a case where an information processing program for realizing the functions of the embodiments is supplied directly or remotely to a system or device. Therefore, the scope of the present invention also includes a program installed on a computer to realize the functions of the present invention by a computer, a medium storing the program, and a WWW (World Wide Web) server for downloading the program. In particular, the scope of the present invention includes at least a non-transitory computer readable medium storing a program for causing a computer to execute the processing steps included in the above-mentioned embodiments.

[0077] This application claims priority based on Japanese Patent Application No. 2013-136953, filed on June 28, 2013, the disclosure of which is incorporated herein in its entirety.

Claims

1. A setting means for setting a category of an object detected from an image captured by a camera; a first input means for receiving, by an operator's operation, an input of a name of a category of an object included in the video image that is different from a preset category; a second input means for receiving an input of a frame of a figure designating an area corresponding to the different category by an operation of an operator on the image; a third input means for receiving an input for associating the different categories with a part of a section of the video by an operator's operation on a display showing a relationship between a category of an object included in the video and a section of the video; a data storage means for storing names of the different categories and positions of the figures in the video in association with each other so that the names can be learned by an image analysis module; A video processing system comprising:

2. 2. The image processing system according to claim 1, wherein the names of the different categories and the positions of the graphics in the image are stored in association with identification information of the operator.

3. 3. The image processing system according to claim 1, wherein an object existing in the area corresponding to the different category is different from an object whose category is set by the setting means.

4. Further, a detection means is provided for performing detection using a result of learning the names of the different categories and the positions of the figures in the video. The image processing system according to any one of claims 1 to 3.

5. The image processing system according to claim 1 , wherein the graphic form is a rectangle.

6. The image processing system according to claim 1 , wherein the graphic figure is a polygon.

7. Set the category of objects detected from the camera footage, receiving, through an operator's operation, an input of a name of a category of an object included in the video that is different from a preset category; accepting input of a frame of a figure designating an area corresponding to the different categories by an operation of an operator on the image; receiving an input for associating the different categories with a portion of a section of the video by an operator's operation on a display showing a relationship between a category of an object included in the video and a section of the video; storing names of the different categories in association with positions of the figures in the video for learning by an image analysis module; Image processing method.

8. A process of setting a category for objects detected from the video captured by the camera; A process of receiving, through an operator's operation, an input of a name of a category of an object included in the video that is different from a preset category; receiving input of a graphic frame that specifies an area corresponding to the different categories by an operation of an operator on the image; receiving an input for associating the different categories with a portion of the video by an operator's operation on a display showing a relationship between a category of an object included in the video and a section in the video; storing names of the different categories in association with positions of the figures in the video for learning by an image analysis module; A program that causes a computer to execute the following.

Images