Video management device, video management method, and video management program

Abstract

To effectively confirm a video image photographed by a monitoring camera.SOLUTION: A video image management device comprises: a video image reception part that records a monitoring video image received from a monitoring camera into a video image storage part, and transmits it to a reproduction control part in real time; a person detection part that detects a person from the monitoring video image recorded in the video image storage part, and records a person detection segment as a time segment of the monitoring video image in which the person is detected to additional information corresponding to the monitoring video image; and a marking processing part that receives a marking from a user against the monitoring video image displayed in real time, and records a marking time as a time of the monitoring video image to be marked to the additional information. The reproduction control part continuously reproduces the person detection segment in the monitoring video image on the basis of the additional information, or continuously reproduces the marking segment as the time segment including the marking time in the monitoring video image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a video management device, a video management method, and a video management program. [Background technology]

[0002] There is known a technique for detecting a person from a video image captured by a surveillance camera using AI (Artificial Intelligence) (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-187124 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a possibility that AI-based person detection may miss some detections. On the other hand, because the video footage captured by surveillance cameras is very long, it is not realistic to check all the recorded footage.

[0005] An object of the present disclosure is to provide a technology for efficiently checking video captured by a surveillance camera. [Means for solving the problem]

[0006] A video management device according to one aspect of the present disclosure comprises a video receiving unit that records surveillance video received from a surveillance camera in a video storage unit and transmits the video to a playback control unit in real time; a person detection unit that detects a person from the surveillance video recorded in the video storage unit and records a person detection section, which is a time section of the surveillance video in which the person is detected, in additional information corresponding to the surveillance video; and a marking processing unit that accepts markings from a user on the surveillance video displayed in real time and records a marking time, which is the time of the marked surveillance video, in the additional information, and the playback control unit continuously plays back the person detection section in the surveillance video based on the additional information, or continuously plays back a marking section, which is a time section in the surveillance video that includes the marking time.

[0007] A video management device according to one aspect of the present disclosure comprises a video receiving unit that records surveillance video received from a surveillance camera in a video storage unit and transmits the video to a playback control unit in real time; a person detection unit that detects a person from the surveillance video recorded in the video storage unit and records a person detection section, which is a time section of the surveillance video in which the person is detected, in additional information corresponding to the surveillance video; and a marking processing unit that accepts markings from a user on the surveillance video displayed in real time and records a marking time, which is the time of the marked surveillance video, in the additional information, and the playback control unit continuously plays back the person detection section and the marking section, which is a time section including the marking time, in the surveillance video based on the additional information.

[0008] A video management method according to one aspect of the present disclosure includes a processor recording surveillance video received from a surveillance camera, detecting a person from the recorded surveillance video, recording a person detection section, which is a time section of the surveillance video in which the person was detected, in additional information corresponding to the surveillance video, displaying the surveillance video received from the surveillance camera in real time, accepting markings from a user on the displayed surveillance video, recording a marking time, which is the time of the marked surveillance video, in the additional information, and continuously playing back the person detection section in the surveillance video based on the additional information, or continuously playing back a marking section, which is a time section in the surveillance video that includes the marking time.

[0009] A video management program according to one aspect of the present disclosure causes a computer to record surveillance video received from a surveillance camera, detect a person from the recorded surveillance video, record a person detection section, which is a time section of the surveillance video in which the person was detected, in additional information corresponding to the surveillance video, display the surveillance video received from the surveillance camera in real time, accept markings from a user on the displayed surveillance video, record a marking time, which is the time of the marked surveillance video, in the additional information, and, based on the additional information, continuously play back the person detection section in the surveillance video, or continuously play back a marking section, which is a time section in the surveillance video that includes the marking time.

[0010] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to efficiently check images captured by a surveillance camera. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a video management system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an example of a video playback screen according to the present embodiment; [Figure 3A] FIG. 1 is a first diagram showing variations of a playback section bar according to the present embodiment; [Figure 3B] FIG. 2 is a second diagram showing variations of the playback section bar according to the present embodiment; [Figure 4] FIG. 10 is a diagram showing an example of a playback setting screen according to the embodiment; [Figure 5] FIG. 10 is a diagram showing an example of a detection frame setting screen according to the present embodiment; [Figure 6] 1 is a flowchart showing an example of a process performed by a video monitoring device when performing marking according to an embodiment of the present invention. [Figure 7] 1 is a flowchart showing an example of a process for efficiently checking a surveillance video by segment playback according to the present embodiment. [Figure 8] FIG. 1 is a block diagram illustrating an example of the hardware configuration of a video management device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0014] (Embodiment 1) <System configuration> FIG. 1 is a block diagram showing an example of the configuration of a video management system 1 according to this embodiment.

[0015] 1, the video management system 1 includes a surveillance camera 2 and a video management device 10. The surveillance camera 2 is connected to the video management device 10 through a predetermined communication network (not shown). A plurality of surveillance cameras 2 may be connected to the video management device 10. Examples of communication networks include a wired LAN (e.g., Ethernet), a wireless LAN (e.g., WiFi), a mobile communication network (e.g., LTE, 4G, 5G), the Internet, and a VPN (Virtual Private Network).

[0016] The surveillance camera 2 is a camera installed at a monitoring location, and captures images of the monitoring location to generate a surveillance video 52 (see FIG. 2). The surveillance video 52 may be video data. Alternatively, the surveillance video 52 may be data of multiple still images captured continuously. The surveillance camera 2 transmits the generated surveillance video 52 to the video management device 10 via a communication network.

[0017] The video management device 10 includes a video receiving unit 11, a person detecting unit 12, an extended detecting unit 13, a marking processing unit 14, a playback control unit 15, a setting processing unit 16, a video storage unit 21, and an additional information storage unit 22.

[0018] The video receiving unit 11, the person detecting unit 12, the extended detecting unit 13, the marking processing unit 14, the playback control unit 15, and the setting processing unit 16 may be realized by a processor 1001 (see FIG. 8) provided in the video management device 10 working in cooperation with a memory 1002 (see FIG. 8) to execute a predetermined computer program. The video storage unit 21 and the additional information storage unit 22 may be realized by the memory 1002 and / or the storage 1003 (see FIG. 8) provided in the video management device 10.

[0019] The video storage unit 21 stores one or more surveillance videos 52. The additional information storage unit 22 stores additional information corresponding to the surveillance videos 52 stored in the video storage unit 21. The additional information will be described later.

[0020] The video receiving unit 11 receives the surveillance video 52 from the surveillance camera 2 and stores (e.g., records) the received surveillance video 52 in the video storage unit 21. The video receiving unit 11 may transmit the received surveillance video 52 to the playback control unit 15 in real time.

[0021] The person detection unit 12 performs a person detection process on the surveillance video 52 to detect a person from the surveillance video 52. For example, the person detection unit 12 performs the person detection process using a known AI for person detection to detect a person from the surveillance video 52. The AI ​​for person detection may be a deep neural network (e.g., a convolutional neural network (CNN)) that has been trained for person detection. When the person detection unit 12 detects a person from a frame image constituting the surveillance video 52, the person detection unit 12 records a person detection position 53 (see FIG. 2 ) indicating the position where the person in the frame image was detected and a person detection section 62 (see FIG. 3A ) indicating the section of the time period during which the person continued to be detected in additional information corresponding to the surveillance video 52, which is stored in the additional information storage unit 22.

[0022] The extended detection unit 13 detects a detection target object other than a person. For example, the extended detection unit 13 performs an object detection process using a known object detection AI to detect an object from the surveillance video 52. The object detection AI may be a deep neural network (e.g., a CNN) that has been trained to detect the object. The detection target object may be set by the user. For example, if a "car" is set as the detection target object, the extended detection unit 13 detects the car from the surveillance video 52. When the extended detection unit 13 detects a car from a frame image constituting the surveillance video 52, the extended detection unit 13 records an extended detection position (not shown) indicating the position where the car was detected in the frame image and an extended detection section 64 (see FIG. 3B) indicating the time period during which the car continued to be detected in the additional information corresponding to the surveillance video 52, which is stored in the additional information storage unit 22.

[0023] The marking processing unit 14 performs processing related to marking of the surveillance video 52 by a user. For example, while watching the surveillance video 52 being played back, the user marks the entire frame image of interest or a position of interest in the frame image. For example, when the user sees a frame image of the surveillance video 52 in which a suspicious person appears, the user marks the entire frame image or the position of the suspicious person appearing in the frame image. The marking processing unit 14 records a marking position 54 (see FIG. 2 ) indicating the position in the frame image where the marking was made and a marking time (not shown) indicating the time of the frame image where the marking was made in the additional information corresponding to the surveillance video 52 stored in the additional information storage unit 22. Note that when the user marks the entire frame image, the marking processing unit 14 may mark the marking position 54 as the entire frame image. In the following, a section of a predetermined time period including the marking time will be referred to as a marking section 63 (see FIG. 3A ). The length of the predetermined time period may be set arbitrarily.

[0024] For example, the user may set a marking position 54 for a suspicious person among a plurality of person detection positions 53 detected by the person detection unit 12. This allows the user who later reviews the surveillance video 52 to efficiently review the surveillance video 52 for the time period in which the suspicious person appears by checking the marked section 63 of the surveillance video 52, rather than checking all of the person detection sections 62 detected by the person detection unit 12.

[0025] For example, the user may set a marking position 54 for a person who was not detected (i.e., missed detection) by the person detection unit 12. This allows the user, who later reviews the surveillance video 52, to efficiently review the surveillance video 52 for the time period in which people appear, by checking the person detection section 62 and the marking section 63, without missing any people that the person detection unit 12 was unable to detect.

[0026] The playback control unit 15 controls playback of the surveillance video 52 stored in the video storage unit 21 or the surveillance video 52 transmitted in real time from the video receiving unit 11. In doing so, the playback control unit 15 controls playback using additional information corresponding to the surveillance video 52 stored in the additional information storage unit 22. For example, when playing back the surveillance video 52, the playback control unit 15 performs section playback to continuously play back the person detection section 62 and / or the marking section 63 recorded in the additional information corresponding to the surveillance video 52. This allows the user to efficiently check the surveillance video 52 during the time period in which people appear. Note that the user who performs the marking and the user who checks the surveillance video 52 by playing back the section may be different people or may be the same person.

[0027] The setting processing unit 16 processes settings related to detection and playback. For example, the setting processing unit 16 displays a playback setting screen 80 (see FIG. 4) for making settings related to section playback, and sets values ​​input to the playback setting screen 80 in the playback control unit 15. Details of the playback setting screen 80 will be described later (see FIG. 4). For example, the setting processing unit 16 displays a detection frame setting screen 90 (see FIG. 5) for setting a detection frame 92 (see FIG. 5) that indicates the range of person detection processing for frame images of the surveillance video 52, and sets the detection frame 92 input to the detection frame setting screen 90 in the person detection unit 12. Details of the detection frame setting screen 90 will be described later (see FIG. 5).

[0028] <Video playback screen> 2 is a diagram showing an example of a video playback screen 50 according to this embodiment. The video playback screen 50 is generated by the playback control unit 15 and displayed on the display device 1005 (see FIG. 8).

[0029] The video playback screen 50 includes a video display area 51, an operation button area 55, a seek bar 56, a playback section bar 60, and a clipping image area 70.

[0030] In the video display area 51, a surveillance video 52 being played back is displayed.

[0031] The operation button area 55 has buttons arranged therein for controlling the playback of the surveillance video 52. For example, the operation button area 55 has arranged therein buttons such as a play button, a stop button, a pause button, a rewind button, a fast-forward button, a slow-motion playback button, a frame-by-frame playback button, a button for skipping a predetermined time, a button for skipping to the next playback section, and a button for skipping to the previous playback section.

[0032] The seek bar 56 is a bar that indicates the time position during playback of the surveillance video 52 displayed in the video display area 51. The user can play back the surveillance video 52 from any time by moving a slider 57 of the seek bar 56 to any time position.

[0033] As shown in FIG. 2, the playback section bar 60 has the same length as the seek bar 56 and is arranged alongside the seek bar 56. The playback section bar 60 displays a playback section 61 (e.g., a person detection section 62 or a marking section 63) in a different manner (e.g., a different color or pattern) from other sections. This allows the user to see at a glance the playback section bar 60 and determine where on the seek bar 56 the playback section 61 is located. Therefore, the user can easily play back the playback section 61 by moving the slider 57 of the seek bar 56 to the playback section 61 on the playback section bar 60. Variations of the playback section bar 60 will be described later (see FIGS. 3A and 3B).

[0034] In the clipping image area 70, frame images of the surveillance video 52 to which person detection positions 53 or marking positions 54 have been added by the additional information are displayed side by side. Hereinafter, the frame images displayed in the clipping image area 70 are referred to as clipping images 71. In the clipping image area 70, a first icon 72 may be displayed alongside a clipping image 71 to which person detection positions 53 have been added, and a second icon 73 may be displayed alongside a clipping image 71 to which marking positions 54 have been added. The first icon 72 and the second icon 73 may have different appearances (e.g., different colors, patterns, shapes, etc.). This allows the user to distinguish at a glance between the clipping image 71 to which the person detection positions 53 have been added and the clipping image 71 to which the marking positions 54 have been added. In addition, when the user selects one of the multiple clipping images 71 displayed in the clipping image area 70, the playback control unit 15 may play back the surveillance video 52 from the time of the selected clipping image 71. This allows the user to easily check the surveillance video 52 around the time of the clipped image 71.

[0035] <Playback section bar variations> Fig. 3A is a first diagram showing variations of the playback interval bar 60 according to this embodiment, and Fig. 3B is a second diagram showing variations of the playback interval bar 60 according to this embodiment.

[0036] 3A, the person detection section 62 and the marking section 63 may be displayed in different ways (for example, different colors or patterns) on the playback section bar 60. This allows the user to see at a glance the playback section bar 60 and recognize where on the seek bar 56 the person detection section 62 and the marking section 63 are located.

[0037] As shown in FIG. 3A, the playback section bar 60 may display a non-detection section 65 in a different manner (for example, a different color or pattern) from other sections. The non-detection section 65 is a time section in which the person detection section 12 and the extended detection section 13 do not perform detection processing. The non-detection section 65 may be set by the user. This allows the user to check at a glance the playback section bar 60 where the non-detection section 65 is located on the seek bar 56. Furthermore, setting the non-detection section 65 can shorten the time required for the person detection process by the person detection section 12.

[0038] 3B, the extended detection section 64 may be displayed in a manner different from other sections (for example, in a different color or pattern) on the playback section bar 60. This allows the user to see at a glance the playback section bar 60 and determine where on the seek bar 56 the extended detection section 64 is located.

[0039] <Playback settings screen> 4 is a diagram showing an example of a playback setting screen 80 according to this embodiment. The playback setting screen 80 is generated by the setting processing unit 16 and displayed on the display device 1005 (see FIG. 8).

[0040] The playback setting screen 80 includes a skip time input area 81 , a previous playback time input area 82 , and a next playback time input area 83 .

[0041] The skip time input area 81 is used to input the time by which the playback of the surveillance video 52 is skipped when the skip button for the predetermined time is pressed. For example, if "10 seconds" is input in the skip time input area 81 and the skip button for the predetermined time is pressed while the surveillance video 52 is being played back, the playback control unit 15 will skip the playback of the surveillance video 52 for 10 seconds.

[0042] In the previous playback time input area 82, a time indicating how far back in time before the playback section 61 playback should start when the playback section 61 is played back is input.

[0043] The later playback time input area 83 is used to input a time indicating how much later in the playback section 61 the playback will end when the playback of the playback section 61 is to be played back.

[0044] For example, if "5 seconds" is input in the previous playback time input area 82 and "3 seconds" is input in the next playback time input area 83, the playback control unit 15 will start playback 5 seconds before the start time of the playback section 61 and continue playback until 3 seconds after the end time of the playback section 61. This allows the user to easily check what happened before and after the playback section 61 in which a person of interest appears.

[0045] <Detection frame setting screen> 5 is a diagram showing an example of a detection frame setting screen 90 according to this embodiment. The detection frame setting screen 90 is generated by the setting processing unit 16 and displayed on the display device 1005 (see FIG. 8).

[0046] Detection frame setting screen 90 includes a detection frame setting area 91. The user places a combination of multiple rectangles in detection frame setting area 91. Setting processing unit 16 sets the outermost frame of the figure formed by combining the multiple rectangles in detection frame setting area 91 as detection frame 92. This allows the user to easily set detection frame 92 with a complex shape.

[0047] When performing detection processing on a frame image, the person detection unit 12 and the extended detection unit 13 target detection within the range of the detection frame 92, and do not target detection outside the range of the detection frame 92. This makes it possible to suppress unnecessary detection and shorten the time required for detection processing.

[0048] <Marking> Fig. 6 is a flowchart showing an example of the process performed by the video management device 10 when performing marking according to the present embodiment. Note that the process shown in Fig. 3 is an example, and the video management device 10 may perform a process different from the process shown in Fig. 3 depending on the use case.

[0049] The person detection unit 12 executes a person detection process on the surveillance video 52, and records the person detection position 53 and the person detection section 62 in the additional information (S101). Note that, if a detection frame 92 is set, the person detection unit 12 targets the area within the detection frame 92 as the detection target.

[0050] The user inputs whether or not to perform segment playback (S102).

[0051] If an input is made to the effect that section playback is not to be performed (S102: NO), the playback control unit 15 determines the entire time period of the monitoring video 52 as the playback section 61 (S103). Then, the process proceeds to step S110.

[0052] If an input is made to perform section playback (S102: YES), the playback control unit 15 determines the playback section 61 based on the person detection section 62 recorded in the additional information (S104). For example, if a first time is input in the previous playback time input area 82 and a second time is input in the next playback time input area 83, the playback control unit 15 determines the playback section 61 as a section from a time that is the first time before the start time of the person detection section 62 to a time that is the second time after the end time of the person detection section 62. In the following description, the surveillance video 52 of the playback section 61 may be referred to as the section video.

[0053] The playback control unit 15 displays the video playback screen 50 (S105). The person detection section 62 recorded in the additional information in step S101 is displayed in the playback section bar 60 on the video playback screen 50. Then, the process proceeds to step S110.

[0054] The playback control unit 15 plays back the section video continuously (S110). For example, when performing section playback and there are two playback sections 61 spaced apart, the playback control unit 15 plays back the section video of the first playback section 61, and after completing playback of the first section video, skips to the second playback section 61 and plays back the second section video.

[0055] The user marks the video section being played back (S111). For example, if the user detects a suspicious person in the video section, the user marks the position of the person. The user may mark the video section while performing operations such as pausing, slow playback, frame-by-frame playback, rewinding, or fast-forwarding.

[0056] The marking processing unit 14 records the marking position 54 and the marking time of the marking performed in step S111 in the additional information (S112).

[0057] The playback control unit 15 ends this process when playback of the last section video is completed or when the user performs an operation to end marking.

[0058] Through the above processing, the video management device 10 can generate additional information that records the person detection position 53 and person detection section 62 where the person detection unit 12 automatically detected a person from the surveillance video 52, as well as the marking position 54 and marking time that the user manually inputs while watching the surveillance video 52.

[0059] <Video confirmation> 7 is a flowchart showing an example of processing for efficiently checking monitoring video 52 by section playback according to the present embodiment. Note that the processing shown in FIG. 4 is an example, and video management device 10 may perform processing different from this processing depending on the use case.

[0060] The playback control unit 15 reads out the surveillance video 52 from the video storage unit 21, and reads out the additional information corresponding to the surveillance video 52 from the additional information storage unit 22 (S201).

[0061] The playback control unit 15 determines the playback section 61 based on the person detection section 62 and / or the marking time recorded in the additional information (S202). For example, the playback control unit 15 determines the playback section 61 corresponding to the person detection section 62, as described in Fig. 6. Furthermore, for example, when a first time is input in the previous playback time input area 82 and a second time is input in the next playback time input area 83, the playback control unit 15 determines the section from the first time before the marking time to the second time after the marking time as the playback section 61 (marking section 63).

[0062] The playback control unit 15 displays the video playback screen 50 (S203).

[0063] The playback control unit 15 plays back the section video continuously (S204). That is, the playback control unit 15 plays back the monitoring video 52 in the person detection section 62 or the marking section 63 continuously.

[0064] This allows the user to efficiently check, from the recorded surveillance video 52, the video segments in which people appear and the video segments marked by the user.

[0065] Next, several use cases using the video management device 10 according to the present embodiment will be described.

[0066] <Use Case 1> First, the video receiving unit 11 records the surveillance video 52 received from the surveillance camera 2 in the video storage unit 21. The person detection unit 12 performs a person detection process on the surveillance video 52, and records the detected person detection position 53 and person detection section 62 in the additional information corresponding to the surveillance video 52.

[0067] Next, the first user performs marking while playing back the surveillance video 52 recorded in the video storage unit 21. The marking processing unit 14 records the marking position 54 and the marking time in the additional information corresponding to the surveillance video 52.

[0068] Next, it is assumed that some kind of problem (for example, an incident or accident caused by a person) occurs at the location where the surveillance camera 2 is installed.

[0069] In this case, the second user continuously plays back the section video of the person detection section 62 and the marking section 63 based on the additional information on the video playback screen 50. This allows the second user to efficiently check the situation around the time when the problem occurred.

[0070] <Use Case 2> First, the video receiving unit 11 records the surveillance video 52 received from the surveillance camera 2 in the video storage unit 21. The person detection unit 12 performs a person detection process on the surveillance video 52, and records the detected person detection position 53 and person detection section 62 in the additional information corresponding to the surveillance video 52.

[0071] Next, it is assumed that some kind of problem (for example, an incident or accident caused by a person) occurs at the location where the surveillance camera 2 is installed.

[0072] In this case, the first user continuously plays back the section video of the person detection section 62 on the video playback screen 50 based on the additional information, and finds the time around when the problem occurred. Then, the first user plays back the video around the time when the problem occurred, and marks the position or frame image of concern. The marking processing unit 14 records this marked marking position 54 and marking time in the additional information corresponding to the surveillance video 52.

[0073] The second user continuously plays back the section video of the marked section 63 based on the additional information on the video playback screen 50. This allows the second user to efficiently check the video marked by the first user.

[0074] <Use Case 3> First, the video receiving unit 11 records the surveillance video 52 received from the surveillance camera 2 in the video storage unit 21, and transmits the received surveillance video 52 in real time to the playback control unit 15. The person detection unit 12 performs a person detection process on the surveillance video 52 recorded in the video storage unit 21, and records the detected person detection position 53 and person detection section 62 in the additional information corresponding to the surveillance video 52.

[0075] On the other hand, the playback control unit 15 displays the monitoring video 52 transmitted from the video receiving unit 11 in real time.

[0076] A first user (e.g., a security guard at a facility management company) marks a person of interest while watching the surveillance video 52 displayed in real time. The marking processing unit 14 records the marked marking position 54 and the marking time in the additional information corresponding to the surveillance video 52.

[0077] A second user (e.g., the owner of the facility) continuously plays back the section video of the person detection section 62 and / or the marking section 63 based on the additional information on the video playback screen 50. This allows the second user to efficiently check the video around the time when the person of interest was captured.

[0078] <Hardware configuration> FIG. 8 is a block diagram showing an example of the hardware configuration of the video management device 10 according to the present disclosure.

[0079] The above-described video management device 10 may be configured to include a computer 1000 shown in Fig. 8. Furthermore, the processing performed by the above-described video management device 10 may be realized by the computer 1000 shown in Fig. 8 executing a computer program.

[0080] The computer 1000 includes a processor 1001, a memory 1002, a storage 1003, an input device 1004, a display device 1005, a communication device 1006, a graphics processing unit (GPU) 1007, an interface device 1008, and an internal bus 1009. The processor 1001, the memory 1002, the storage 1003, the input device 1004, the display device 1005, the communication device 1006, the graphics processing unit (GPU) 1007, and the interface device 1008 are connected to the internal bus 1009 and can send and receive data bidirectionally via the internal bus 1009.

[0081] However, it is not essential that the video management device 10 include all of the components shown in FIG. 8, and it is sufficient that the video management device 10 includes at least the processor 1001 and the memory 1002.

[0082] The processor 1001 is a device that executes a computer program stored in the memory 1002 and realizes the above-described processing. Therefore, the above-described processing can be interpreted as processing mainly performed by the processor 1001. Examples of the processor 1001 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a controller, an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field-Programmable Gate Array).

[0083] The memory 1002 is a device that stores computer programs and data used by the computer 1000. The memory 1002 may include a read-only memory (ROM) and a random access memory (RAM). The memory 1002 may be configured as either a volatile storage medium or a non-volatile storage medium.

[0084] The storage 1003 is configured as a non-volatile storage medium, and is a device that stores computer programs and data handled by the computer 1000. Examples of the storage 1003 include a hard disk drive (HDD), a solid state drive (SSD), and a flash memory.

[0085] The input device 1004 is a device that accepts input operations from a user. Examples of the input device 1004 include a keyboard, a mouse, a touchpad, and a microphone.

[0086] The display device 1005 is a device that displays a screen. Examples of the display device 1005 include a display and a monitor.

[0087] The communication device 1006 is a device that transmits and receives data to and from other devices via a communication network. The communication device 1006 may include a transmitter that transmits data and a receiver that receives data. The communication device 1006 may support either wired communication or wireless communication.

[0088] The GPU 1007 is a device that processes screen rendering at high speed. The GPU 1007 may be used for AI (Artificial Intelligence) processing (for example, learning and processing of a deep neural network).

[0089] The interface device 1008 is a device to which an external recording medium such as a DVD drive, an external HDD, or a USB (Universal Serial Bus) memory can be connected, and reads and writes data from and to the connected external recording medium.

[0090] (Summary of this embodiment) The present embodiment discloses the following techniques.

[0091] <Technology 1> The video management device 10 comprises a video receiving unit 11 that receives surveillance video 52 captured by the surveillance camera 2, a person detection unit 12 that detects a person from the surveillance video 52 and records a person detection section 62, which is a time section of the surveillance video 52 in which the person was detected, in additional information corresponding to the surveillance video 52, a marking processing unit 14 that accepts markings on the surveillance video 52 from a user and records the marking time, which is the time of the marked surveillance video 52, in the additional information, and a playback control unit 15 that, based on the additional information, continuously plays back the person detection section 62 in the surveillance video 52 or continuously plays back a marking section 63, which is a time section in the surveillance video 52 that includes the marking time. This allows the user to efficiently check the surveillance video 52 by watching the person detection section 62 or the marking section 63 that are continuously played back.

[0092] <Technology 2> The video management device 10 comprises a video receiving unit 11 that receives surveillance video 52 captured by the surveillance camera 2, a person detection unit 12 that detects a person from the surveillance video 52 and records a person detection section 62, which is a time section of the surveillance video 52 in which the person was detected, in additional information corresponding to the surveillance video 52, a marking processing unit 14 that accepts markings from a user on the surveillance video 52 and records the marking time, which is the time of the marked surveillance video 52, in the additional information, and a playback control unit 15 that continuously plays back the person detection section 62 in the surveillance video 52 and a marking section 63, which is a time section including the marking time, based on the additional information. This allows the user to efficiently check the surveillance video 52 by viewing the person detection section 62 and the marking section 63 that are continuously played back.

[0093] <Technology 3> In the video management device 10 described in Technology 1 or 2, the playback control unit 15 displays a playback section bar 60 indicating at least one of a person detection section 62 and a marking section 63 alongside a seek bar 56 indicating the playback position of the surveillance video 52. This allows the user to look at the playback section bar 60 and recognize at what position on the seek bar 56 at least one of the person detection section 62 and the marking section 63 exists. Therefore, the user can easily check the video of the desired person detection section 62 or marking section 63 by moving the slider 57 of the seek bar 56.

[0094] <Technology 4> In the video management device 10 described in Technique 3, the person detection section 62 and the marking section 63 in the playback section bar 60 are displayed in different modes. This allows the user to see the playback section bar 60 and recognize the positions of the person detection section 62 and the marking section 63 .

[0095] <Technology 5> The video management device 10 described in any one of techniques 1 to 4 further includes a setting processing unit 16 that sets a detection frame 92 that determines the detection range of the person detection unit 12 based on a combination of multiple shapes input by the user. This allows the user to easily set a complex detection frame 92.

[0096] <Technology 6> The video management method receives surveillance video 52 captured by a surveillance camera 2, detects a person from the surveillance video 52, records a person detection section 62, which is a time section of the surveillance video 52 in which the person was detected, in additional information corresponding to the surveillance video 52, accepts marking from a user on the surveillance video 52, records the marking time, which is the time of the marked surveillance video 52, in the additional information, and continuously plays back the person detection section 62 and / or a marking section 63, which is a time section including the marking time, in the surveillance video 52 based on the additional information. This allows the user to efficiently check the surveillance video 52 by viewing the person detection section 62 and / or the marking section 63 that are continuously played back.

[0097] <Technology 7> The video management program causes the computer 1000 to receive surveillance video 52 captured by the surveillance camera 2, detect a person from the surveillance video 52, record a person detection section 62, which is the time section of the surveillance video 52 in which the person was detected, in additional information corresponding to the surveillance video 52, accept markings from the user on the surveillance video 52, record the marking time, which is the time of the marked surveillance video 52, in the additional information, and, based on the additional information, continuously play back the person detection section 62 and / or the marking section 63, which is the time section including the marking time, in the surveillance video 52. This allows the user to efficiently check the surveillance video 52 by viewing the person detection section 62 and / or the marking section 63 that are continuously played back.

[0098] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]

[0099] The technology of the present disclosure is useful for efficiently checking images captured by a camera. [Explanation of symbols]

[0100] 1. Video management system 2. Surveillance cameras 10 Video management device 11 Video receiving unit 12 Person detection unit 13 Extended detection section 14 Marking processing section 15 Playback control section 16 Setting processing section 21 Video storage unit 22 Additional information storage section 50 Video playback screen 51 Video display area 52 Surveillance footage 53 Person detection position 54 Marking position 55 Operation button area 56 Seek bar 57 Slider 60 Playback section bar 61 Playback section 62 Person detection section 63 Marking Section 64 Extended detection section 65 Non-detection section 70 Clipping Image Area 71 Clipping Images 72 First Icon 73 Second Icon 80 Playback settings screen 81 Skip time input area 82 Previous playback time input area 83 Post playback time input area 90 Detection frame setting screen 91 Detection frame setting area 92 detection frame

Claims

1. a video receiving unit that receives surveillance video from a surveillance camera; a person detection unit that automatically detects a person from the surveillance video and records a person detection section, which is a time section of the surveillance video in which the person was detected, and a person detection position, which is a position at which the person was detected, as additional information corresponding to the surveillance video; a marking processing unit that receives a marking operation from a user on the surveillance video and records a marking time, which is a time when the surveillance video was marked, and a marking position as the additional information; a playback control unit that, when playing back the surveillance video, continuously plays back the person detection section and a marking section, which is a time section including the marking time, based on the additional information, Video management device.

2. a video receiving unit that receives surveillance video from a surveillance camera; an extended detection unit that automatically detects a detection target object other than a person from the surveillance video, and records an extended detection section, which is a time section of the surveillance video in which the object was detected, and an extended detection position, which is a position where the object was detected, in additional information corresponding to the surveillance video; a marking processing unit that receives a marking operation from a user on the surveillance video and records a marking time, which is a time when the surveillance video was marked, and a marking position as the additional information; a playback control unit that, when playing back the surveillance video, continuously plays back the extended detection section and a marking section in the surveillance video, the marking section being a time section including the marking time, based on the additional information, Video management device.

3. the marking processing unit accepts marking for a person who has not been automatically detected by the person detection unit; The video management device according to claim 1 .

4. the playback control unit displays and outputs the person detection section and the marking section in different modes in one playback section bar. The video management device according to claim 1 .

5. the playback control unit displays the monitoring video in real time; the marking processing unit accepts a marking operation from a user on the monitoring video displayed in real time; 3. The video management device according to claim 1 or 2.

6. the marking processing unit further accepts a marking operation from a user for the video of the person detection section; The video management device according to claim 1 .

7. the marking processing unit further accepts a marking operation from a user for the video of the extended detection section; The video management device according to claim 2 .

8. a setting processing unit that sets a detection frame that determines a detection range of the person detection unit based on a combination of a plurality of shapes input by the user, The video management device according to claim 1 .

9. the playback control unit generates a video playback screen, the video playback screen including a video display area and a clipping image area; the video display area displays the monitoring video, In the clipping image area, clipping images, which are frame images of the surveillance video to which at least one of a person detection position and a marking position is added by the additional information, are displayed side by side, When the clipping image is selected by the user, the surveillance video is played back from the selected time. The video management device according to claim 1 .

10. a first icon is displayed on the clipping image to which the person detection position is added; a second icon different from the first icon is displayed on the clipping image to which the marking position is added; The video management device according to claim 9.

11. The processor: Receives surveillance footage from surveillance cameras, Automatically detecting a person from the surveillance video; recording a person detection section, which is a time section of the surveillance video in which the person was detected, and a person detection position, which is a position where the person was detected, as additional information corresponding to the surveillance video; accepting a marking operation from a user on the surveillance video; recording a marking time, which is the time of the marked surveillance video, and a marking position as the additional information; When the monitoring video is played back, the person detection section and the marking section, which is a time section including the marking time, are continuously played back based on the additional information. Video management methods.

12. The processor: Receives surveillance footage from surveillance cameras, automatically detecting a detection target object other than a person from the surveillance video; recording an extended detection section, which is a time section of the surveillance video in which the object was detected, and an extended detection position, which is a position where the object was detected, in additional information corresponding to the surveillance video; accepting a marking operation from a user on the surveillance video; recording a marking time, which is the time of the marked surveillance video, and a marking position as the additional information; When the monitoring video is played back, a marking section in the monitoring video, which is a time section including the extended detection section and the marking time, is continuously played back based on the additional information. Video management methods.

13. Receives surveillance footage from surveillance cameras, Automatically detecting a person from the surveillance video; recording a person detection section, which is a time section of the surveillance video in which the person was detected, and a person detection position, which is a position where the person was detected, as additional information corresponding to the surveillance video; accepting a marking operation from a user on the surveillance video; recording a marking time, which is the time of the marked surveillance video, and a marking position as the additional information; When the monitoring video is played back, the person detection section and the marking section, which is a time section including the marking time, are continuously played back based on the additional information. A video management program that allows a computer to do this.

14. Receives surveillance footage from surveillance cameras, automatically detecting a detection target object other than a person from the surveillance video; recording an extended detection section, which is a time section of the surveillance video in which the object was detected, and an extended detection position, which is a position where the object was detected, in additional information corresponding to the surveillance video; accepting a marking operation from a user on the surveillance video; recording a marking time, which is the time of the marked surveillance video, and a marking position as the additional information; When the monitoring video is played back, a marking section in the monitoring video, which is a time section including the extended detection section and the marking time, is continuously played back based on the additional information. A video management program that allows a computer to do this.

Images