Video information management device, video authentication system, video authentication method, and program

Abstract

To provide a video authentication technology that is easy to use for a user, i.e., a monitor. [Solution] The video information management device comprises a frame extraction unit that extracts a specified frame from an input video, an authentication result acquisition unit that receives an authentication result for a specified area within the extracted frame, a difference calculation unit that calculates the difference in time between the acquisition time of the extracted frame and the receipt time, which is the time when the authentication result is received, and an output control unit that time-shifts the video based on the difference in time and outputs it together with the authentication result.

Description

[Technical Field]

[0001] The present disclosure relates to a video information management device, a video authentication system, a video authentication method, and a program. [Background technology]

[0002] To ensure security in facilities where many people enter and exit, there are systems that monitor the entrance and exit of people using video captured by surveillance cameras, etc. In such systems, to reduce the burden on the monitor, facial images are extracted from the captured video and compared with pre-stored facial images for authentication or to detect the target of surveillance.

[0003] There are image authentication systems, image authentication methods, and image authentication programs that allow a surveillance person to instantly and easily visually recognize a surveillance target (see, for example, Patent Document 1). The system disclosed in Patent Document 1 "includes an imaging terminal that acquires surveillance images, an imaging-side storage unit that stores the surveillance images, a cropping unit that crops out an input characteristic image contained in the surveillance image and generates input image data that includes the input characteristic image, image identification information that identifies the surveillance image, and position information that identifies the crop position of the input characteristic image in the surveillance image, a matching unit that compares the input image data with a pre-stored registered image database of surveillance targets and detects input image data that matches the registered image database as detected image data, and a monitor that, when detected image data is detected as a result of the comparison by the matching unit, displays at least a portion of the surveillance image identified based on the detected image data, including the input characteristic image, and the position of the input characteristic image (abstract excerpt)." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-153908 Summary of the Invention [Problem to be solved by the invention]

[0005] The following analysis was performed by the inventors of the present disclosure.

[0006] Since the purpose of surveillance is to ensure security, observers want to monitor video captured by surveillance cameras as quickly as possible. The same goes for authentication results. However, the process of extracting, matching, and authenticating facial images from video takes a certain amount of time. Therefore, if the captured video is received and displayed in real time, the person to be authenticated will often be outside the field of view by the time the authentication results are received.

[0007] According to the technology disclosed in Patent Document 1, an input feature image and a registered image are displayed side by side. Furthermore, an example is also shown in which a video can be played on the same screen. Here, the input feature image is a facial image cut out from the video, and the registered image is an image stored in advance for matching. The played video is a video temporarily stored in a storage device and is played at the command of the monitor. In other words, if the monitor wants to display the video together with the authentication result, the monitor must operate the device, which places a heavy burden on the monitor. Furthermore, the process from when the authentication result is obtained to when the video is played involves a human intervention process, i.e., the monitor's operation. Therefore, a significant delay occurs in the video being played.

[0008] The present disclosure has been made in consideration of the above circumstances, and one of its purposes is to contribute to providing a video authentication technology that is easy to use for the user, the monitor. [Means for solving the problem]

[0009] According to a first aspect of the present disclosure, a frame extraction unit that extracts predetermined frames from the input video; an authentication result acquisition unit that receives an authentication result for the extracted predetermined area within the frame; a difference calculation unit that calculates a difference time between an acquisition time of the extracted frame and a reception time that is a time when the authentication result is received; and an output control unit that time-shifts the video based on the difference time and outputs the video together with the authentication result.

[0010] According to a second aspect of the present disclosure, The video information management device; a video capture device for capturing the video; an authentication device that performs authentication on a predetermined area within the frame and outputs the authentication result to the video information management device; There is provided a video authentication system including: a display device that displays the time-shifted video output from the video information management device and the authentication result.

[0011] According to a third aspect of the present disclosure, The computer of the video information management device Extract a specific frame from the input video, receiving an authentication result for the extracted predetermined area within the frame; calculating a differential time between the acquisition time of the extracted frame and the reception time, which is the time when the authentication result is received; A video authentication method is provided in which the video is time-shifted based on the difference time and output together with the authentication result.

[0012] According to a fourth aspect of the present disclosure, A program is provided that causes a computer of the video information management device to execute the video authentication method.

[0013] These programs can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to realize a video authentication technology that is easy to use for the user (monitoring person). [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1A is a diagram showing an example of the overall configuration of a video authentication system according to an embodiment of the present disclosure, and FIG. 1B is a diagram showing an example of functional blocks of a video information management device according to an embodiment of the present disclosure. [Figure 2] FIG. 10 is a diagram illustrating an example of a flowchart of a video authentication output process according to an embodiment of the present disclosure. [Figure 3] FIG. 1A is a diagram for explaining a monitoring target of an embodiment of the present disclosure, FIG. 1B is a diagram showing an example of a functional block of an authentication device of an embodiment of the present disclosure, and FIG. 1C is a diagram showing an example of a display device of an embodiment of the present disclosure. [Figure 4] (a) is a diagram showing an example of a functional block of a video information management device according to an embodiment of the present disclosure, (b) is a diagram showing example items of a frame according to an embodiment of the present disclosure, and (c) is a diagram showing example items of an authentication result according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a diagram illustrating an example of a flowchart of a video authentication output process according to an embodiment of the present disclosure. [Figure 6] FIG. 1A is an explanatory diagram for explaining an example of imaging performed by a conventional system and an embodiment of the present disclosure, and FIGS. 1B and 1C are diagrams for explaining an example of display performed by the conventional system. [Figure 7] 1A and 1B are diagrams illustrating a display example according to an embodiment of the present disclosure. [Figure 8] 10A and 10B are diagrams illustrating a display example of a modified example of the present disclosure. [Figure 9] FIG. 10 is a diagram illustrating an example of management data according to a modified example of the present disclosure. [Figure 10] FIG. 1 is a diagram illustrating an example of a hardware configuration of a video information management device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that reference numerals in the drawings are assigned to each element for convenience as an example to facilitate understanding, and are not intended to limit the present disclosure to the illustrated aspects. Furthermore, connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows are used to schematically indicate the flow of the main signal (data) and do not exclude bidirectionality. Furthermore, in the following description, "A and / or B" means A or B, or A and B.

[0017] <<First Embodiment>> An overview of this embodiment will be described. Fig. 1(a) is a diagram for explaining an overview of the entire video authentication system 100 of this embodiment. As shown in this figure, the video authentication system 100 of this embodiment includes a video acquisition device 110, a video information management device 200, an authentication device 120, and a display device 130.

[0018] The video acquisition device 110 acquires video and transmits it to the video information management device 200. The video acquisition device 110 continuously captures still images of a predetermined area (a monitored area) at predetermined time intervals and transmits them to the video information management device 200 in real time. Each still image is called a frame. Each still image includes, as accompanying information, the time (timestamp) at which the still image was acquired and information identifying the frame (frame number). The video acquisition device 110 is, for example, a surveillance camera. The number of video acquisition devices 110 is not important.

[0019] The video information management device 200 buffers the video transmitted from the video acquisition device 110, and transmits each frame to the authentication device 120 for authentication. Then, the video is time-shifted and output to the display device 130 in synchronization with the authentication result. Details of the video information management device 200 will be described later.

[0020] The authentication device 120 authenticates information in a predetermined area within the frame received from the video information management device 200 and outputs the authentication result to the video information management device 200. For example, it authenticates the facial area of ​​a person within the frame. The authentication is performed, for example, by comparing it with an authentication image stored in advance. For example, if there is a match among the authentication images stored in advance, it outputs an authentication success as the authentication result, and if there is no match, it outputs an authentication failure as the authentication result.

[0021] The display device 130 displays the time-shifted video output from the video information management device 200 and the authentication result side by side. For example, it is a display or the like.

[0022] [Video information management device] 1(b), the video information management device 200 includes a frame extraction unit 210, an authentication result acquisition unit 220, a difference calculation unit 230, and an output control unit 240. It also includes a buffer unit (hereinafter referred to as the buffer) 250 that temporarily stores the video acquired from the video acquisition device 110.

[0023] The frame extraction unit 210 extracts predetermined frames from the input video. In this embodiment, for example, all frames of the input video are extracted in the order in which they were input and transmitted to the authentication device 120. At this time, information identifying each frame (frame identification information) is also transmitted. The frame extraction unit 210 also temporarily stores the input video in the buffer 250.

[0024] For example, the frame extraction unit 210 may be configured to analyze each frame of the input video and extract authentication target information if it is included. If the authentication target is a person's face, for example, the frame may be analyzed using existing face extraction software, and if a person's face is included, the frame may be extracted.

[0025] The authentication result acquisition unit 220 receives the authentication result for a predetermined area in the extracted frame from the authentication device 120. In this embodiment, for example, as described above, the authentication result for the face area of ​​a person in the frame is received. At this time, the authentication result acquisition unit 220 also receives frame identification information for the frame used for authentication.

[0026] The difference calculation unit 230 calculates the difference time between the acquisition time of the extracted frame and the time when the authentication result was received. In this embodiment, the difference calculation unit 230 receives the acquisition time of the frame along with the frame identification information from the frame extraction unit 210. The difference calculation unit 230 also receives the reception time, which is the time when the authentication result was received, along with the frame identification information from the authentication result acquisition unit 220. Then, for those frame identification information that match, the difference is calculated and used as the difference time of the frame. The calculated difference time is output to the output control unit 240. The difference time may be calculated for each frame, for every predetermined number of frames, or at predetermined time intervals.

[0027] The output control unit 240 time-shifts the video stored in the buffer 250 by the latest time difference and outputs it together with the authentication result.

[0028] [Video authentication output processing] Next, the flow of the video authentication output process by the video information management device 200 of this embodiment will be described. Fig. 2 shows the processing flow of the video authentication output process of this embodiment. This process is started, for example, when the video authentication system 100 is started.

[0029] The frame extraction unit 210 extracts frames from the moving image input to the moving image information management device 200 (step S1101).

[0030] The frame extraction unit 210 transmits the extracted frames to the authentication device 120 (step S1102). The frame extraction unit 210 also temporarily stores the input video in the buffer 250 (step S1103).

[0031] When the authentication result acquisition unit 220 receives the authentication result from the authentication device 120 (step S1104; Yes), it notifies the difference calculation unit 230 of the result.

[0032] The difference calculation unit 230 calculates the difference time based on the frame identification information (step S1105) and notifies the output control unit 240 of the calculated difference time.

[0033] The output control unit 240 time-shifts the video by the difference time and outputs it together with the authentication result (step S1106).

[0034] Each unit of the video information management device 200 repeats the above process until an instruction to end the process is received (step S1107).

[0035] As described above, according to this embodiment, when a video and an authentication result are transmitted to the display device 130, the video is time-shifted by the difference time and transmitted. Therefore, the video and the authentication result for the video are displayed on the display device 130 at the same time.

[0036] Generally, authentication processing takes time, and thus, conventionally, there has been a certain time lag between a video and the authentication result of that video. Therefore, if the received video and authentication result are output to the display device 130 without time shifting, the authentication result displayed on the display device 130 will be different from the original video frame. For example, if the target of authentication is a person's face, there is a possibility that the authentication result of a person who does not exist in the video will be displayed at the same time.

[0037] However, according to this embodiment, this situation is improved, and the video and the authentication result of the person in the frame of the video are displayed in synchronization. At this time, the video is displayed in a time-shifted manner. However, this time-shifted period is the time required for the simplified authentication device 120 to perform authentication. For example, delays due to human intervention, such as operations by an observer, are not included. Therefore, the video is displayed in near real time.

[0038] Therefore, according to this embodiment, it is possible to simultaneously view video captured by a video capture device 110 such as a surveillance camera in near real time and display the authentication results in parallel while the authentication target is within the field of view of the video.

[0039] Therefore, according to this embodiment, it is possible to realize a video authentication technology that is easy to use for the monitor, who is the user.

[0040] <<Second embodiment>> Next, a second embodiment will be described. In this embodiment, the video authentication system 100 of the first embodiment will be described as an example in which it is used for room entry management. In this embodiment, the authentication target is a person's face. In addition, the authentication device 120 will be described as an example in which a face image of a person who has been previously permitted to enter the room is stored as a matching image.

[0041] The video authentication system 100 of this embodiment has the same configuration as the video authentication system 100 of the first embodiment shown in FIG. 1(a).

[0042] However, in this embodiment, as shown in Fig. 3(a), the video capturing device 110 is a surveillance camera 110a. This surveillance camera 110a is installed, for example, at the entrance of a specific room to manage entry to the room.

[0043] The surveillance camera 110a captures still images of a predetermined capture range at extremely short intervals and outputs them as video in real time to the video information management device 200. Each still image is called a frame. Each frame has accompanying information. The accompanying information includes a frame number to identify the frame and an acquisition time to indicate the time the image was captured.

[0044] As shown in FIG. 3(b), the authentication device 120 of this embodiment includes an authentication unit 121 and a matching image storage unit 122.

[0045] Facial images of people who are permitted to enter the room are stored in advance in the matching image storage unit 122. Facial images and information for identifying the people in the facial images are stored in association with each other in the matching image storage unit 122. In this embodiment, the information for identifying the people (person identification information) is, for example, a person's name.

[0046] When the authentication unit 121 receives a frame from the video information management device 200, it extracts a facial image (facial area) from the frame and compares it with a matching image stored in the matching image storage unit 122. The facial area in the frame is extracted using an existing face detection algorithm. The detected facial area is then compared with the matching facial image to perform authentication. If a matching image that matches the detected facial area is stored in the matching image storage unit 122, the matching image and person identification information are output as the authentication result.

[0047] When there are multiple face areas in one frame, the authentication unit 121 performs matching for all the face areas and outputs the authentication results for each of them.

[0048] Examples of face detection algorithms that can be used include Haar Cascades (haar-like features), Histogram of Oriented Gradients (HOG), and Deep Learning CNN. Examples of matching and authentication algorithms that can be used include Eigenfaces and FaceNet.

[0049] 3(c), the display device 130 of this embodiment includes, for example, two displays. Here, the two displays are referred to as a first display 131 and a second display 132.

[0050] Display device 130 displays the time-shifted video on first display 131 and the authentication result on second display 132. The displayed authentication result is a match image and the person's name. In this embodiment, as shown in the figure, the video is time-shifted, so the person in the video displayed on first display 131 is displayed as the authentication result.

[0051] The video information management device 200 of this embodiment basically has the same configuration as the video information management device 200 of the first embodiment shown in Fig. 1(b). However, as shown in Fig. 4(a), the video information management device 200 of this embodiment further includes a differential time storage unit 260 that stores differential times.

[0052] When a moving image is input, the frame extraction unit 210 of this embodiment extracts each frame in the order in which it was input, and transmits the frames to the authentication device 120. The input moving image is also temporarily stored in the buffer 250.

[0053] 4(b) shows an example of items included in each frame 310 that constitutes a moving image. Each frame 310 includes an acquisition time 311, a frame number 312, and frame data 313, which is image data.

[0054] The frame extraction unit 210 may extract only those frames 310 that contain a face image to be authenticated from the input frames 310, and transmit them to the authentication device 120. In this case, each time the frame extraction unit 210 acquires a frame 310, it analyzes the frame data 313 and determines whether or not a face area is present. For this determination, an existing face detection algorithm may be used, as in the authentication device 120.

[0055] The authentication result acquisition unit 220 receives the authentication result 320 from the authentication device 120. As shown in FIG. 4(c), the authentication result 320 includes a receipt time 321, which is the time when the authentication result was received, a frame number 322 in which the frame number 312 of the frame 310 where authentication was performed is stored as is, a comparison image 323, and a person's name 324 as person identification information. If the authentication is unsuccessful, the comparison image 323 and the person's name 324 may store information to be displayed when the authentication is unsuccessful. Alternatively, nothing may be stored. The receipt time 321 is set by the authentication result acquisition unit 220.

[0056] If there are multiple face areas in one frame 310 and each face area is authenticated, multiple authentication results 320 having the items shown in FIG. 4(b) are transmitted.

[0057] The difference calculation unit 230 calculates the difference between the acquisition time 311 and the reception time 321 of the frame 310 and the authentication result 320, which have the same frame number (312, 322), as the differential time.

[0058] In this embodiment, the difference calculation unit 230 calculates the difference time at a predetermined time interval or for every predetermined number of frames. The calculated difference time is stored in the difference time storage unit 260. Note that each time a new difference time is calculated, the old difference time is overwritten.

[0059] Specifically, when the difference calculation unit 230 receives the authentication result 320, it extracts the reception time 321 and frame number 322. Then, it searches the buffer 250 for a frame 310 having the same frame number and extracts its acquisition time 311. Then, it uses the reception time 321 and the acquisition time 311 to calculate the difference time.

[0060] At this time, the difference calculation unit 230 may add a predetermined margin to the difference between the reception time 321 and the acquisition time 311 to obtain the difference time.

[0061] The output control unit 240 outputs the video stored in the buffer 250 after time-shifting the video by the difference time stored in the difference time storage unit 260. The output control unit 240 also outputs the matching image 323 and the person's name 324 of the authentication result 320 acquired by the authentication result acquisition unit 220 as the authentication result.

[0062] It should be noted that the output control unit 240 may add a predetermined margin to the difference time calculated by the difference calculation unit 230, and output the moving image after time shifting it.

[0063] [Video authentication output processing] Next, the flow of the video authentication output process by the video information management device 200 of this embodiment will be described. Fig. 5 shows the processing flow of the video authentication output process of this embodiment. This process is basically the same as that of the first embodiment. Only the points that differ from the first embodiment will be described.

[0064] In this embodiment, after receiving the authentication result 320 (S1104; Yes), the difference calculation unit 230 determines whether a predetermined period of time has elapsed since the previous difference calculation (step S2101). The difference calculation unit 230 determines the elapsed time using, for example, a timer or the like provided in the video information management device 200.

[0065] If the predetermined time has elapsed (S2101; Yes), the difference calculation unit 230 proceeds to step S1105 and calculates the difference time. If the predetermined time has not elapsed (S2101; No), the difference calculation unit 230 does not calculate the difference time. That is, the video information management device 200 proceeds to step S1106.

[0066] When the calculation is performed for each predetermined number of frames, the moving image information management device 200 determines in step S2101 whether or not the predetermined number of frames of moving images have been received.

[0067] [Display example] Here, to explain the effect of this embodiment, a display example of a conventional method and a display example of the video information management device 200 of this embodiment will be shown.

[0068] For example, as shown in FIG. 6(a), a case will be described in which surveillance camera 110a captures the scene of people A through E entering a room in order. Here, it is assumed that five people, A through E, are moving from the upper left to the lower right in the figure. A line of five people is partially displayed on first display 131 of display device 130, which displays the video. In other words, at the timing of capturing each frame 310, only people within the angle of view of surveillance camera 110a are captured and displayed.

[0069] Furthermore, if the authentication is successful, second display 132 displays matching image 323 and person's name 324. If the authentication is unsuccessful, "undetected" and a predetermined graphic are displayed as an image indicating failure. Furthermore, it is assumed that person A is a person who is permitted to enter the room, and matching image 323 is stored in matching image storage unit 122, while person B is a person who is not permitted, and matching image 323 is not stored in matching image storage unit 122.

[0070] Because the authentication process takes time, in a conventional system that outputs video without time-shifting, as shown in Fig. 6(b), when Person A appears on First Display 131, the authentication result for Person A cannot be displayed on Second Display 132. On the other hand, as shown in Fig. 6(c), when Person A moves out of the field of view, the authentication result for Person A is displayed on Second Display 132.

[0071] On the other hand, in this embodiment, the output timing of the video is adjusted by the video information management device 200. Therefore, as shown in Fig. 7(a), with person A appearing on first display 131, the authentication result for person A is displayed on second display 132. Similarly, as shown in Fig. 7(b), with person B appearing on first display 131, the authentication result for person B is displayed on second display 132.

[0072] As described above, the video information management device 200 of this embodiment has the same configuration as that of the first embodiment, and therefore has the same effects as those of the first embodiment.

[0073] Furthermore, in this embodiment, the differential time is calculated at predetermined time intervals, thereby reducing the calculation load.

[0074] Therefore, according to this embodiment, it is possible to realize a video authentication technology that is easy to use for the monitor, who is the user.

[0075] <Variation 1> In the above embodiments, the display device 130 displays the video and the authentication result separately, but this is not limiting. For example, the display device 130 may be configured to have a single display and to superimpose the video and the authentication result.

[0076] In this case, the output control unit 240 of the video information management device 200 may generate and output the superimposed image data. Alternatively, the output control unit 240 may output the video and the authentication result, and the display device 130 may generate and display the superimposed image, as in the above embodiments.

[0077] 8(a) and 8(b) show display examples similar to those of the second embodiment. As shown in Fig. 8(a), the authentication result for person A is superimposed on an image of person A on the display. Similarly, as shown in Fig. 8(b), the authentication result for person B is superimposed on an image of person B on the display.

[0078] <Variation 2> In the above-described embodiments, the difference calculation unit 230 calculates the difference time at a predetermined timing, and when a new difference time is calculated, the previously calculated difference time is overwritten. However, the present invention is not limited to this.

[0079] For example, after a new differential time is calculated, it is compared with the previously calculated differential time, and if the difference in the differential time is equal to or greater than a predetermined threshold, the differential time may be changed to smooth the image display. Specifically, for example, a maximum value Δt of the amount of change per unit time is determined in advance. Then, if the difference in the differential time is ΔT, the difference time is changed by Δt per unit time (the integer part of (ΔT / Δt)) times, and the last time it is changed by the remainder ΔT / Δt.

[0080] This modification can reduce the speed or delay of video display when an external factor causes a delay in image information.

[0081] <Variation 3> In addition, in each of the above embodiments, the authentication device 120 extracts face regions from the entire frame and authenticates each face region. However, this is not limited to this. For example, a configuration may be adopted in which a person is detected using motion tracking (object tracking) and matching is performed only on the frame in which the person is first detected. In this case, the display device 130 may be configured to display multiple pieces of information. Images of people who have been successfully authenticated may be displayed for a certain period of time and then erased, while images of people who have not been successfully authenticated may be erased when some action is received. Note that motion tracking may use an algorithm such as Kernelized Correlation Filters (KCF). This reduces the load of the authentication process and, accordingly, shortens the difference time. Therefore, the size of the buffer 250 provided by the video information management device 200 can also be reduced.

[0082] <Variation 4> In the above-described embodiments and modifications, an example has been described in which a person who is permitted to enter a room is authenticated. However, the authentication is not limited to this. For example, authentication may be performed to identify a person who is not permitted to enter a room or a person who requires attention.

[0083] In this case, the matching image storage unit 122 of the authentication device 120 stores facial images of people who are not permitted to enter the room, or people who require attention.

[0084] In this case, the video authentication system 100 may further include an audio output device such as a buzzer, a lamp, etc. For example, when a person who is not permitted to enter or a suspicious person is authenticated, not only a display but also an audio output or a lamp display may be used to call attention.

[0085] <Variation 5> In the second embodiment, a case where one monitoring camera 110a manages entry to one room has been described as an example, but this is not limiting. Multiple monitoring cameras 110a may each manage entry to a different room. In this case, the frame 310 further includes, in addition to the information shown in FIG. 4(b), information identifying the monitoring camera 110a that has acquired the information.

[0086] For example, the video information management device 200 has a table that associates the surveillance camera 110a with information that identifies the room being monitored. When transmitting the frame 310 to the authentication device 120, the video information management device 200 adds the information that identifies the room being monitored and transmits it.

[0087] The authentication device 120 stores a comparison image for each room to be monitored in the comparison image storage unit 122. Then, the authentication device 120 returns the authentication result 320 to the video information management device 200 together with information specifying the room to be monitored.

[0088] <Variation 6> The difference time varies depending on the load on the interface and the load on the device performing the verification (authentication device 120), for example, as the response delays as the number of people to be verified increases. Therefore, the timing of transmitting the image information stored in the buffer 250 to the display device 130 may be adjusted at specific intervals.

[0089] For example, when a differential time is set, image information is delayed and transmitted by the set differential time (set value). At this time, a certain margin may be provided in the set value. If the calculated differential time subsequently becomes larger than the set value, transmission is delayed, so the newly calculated differential time is immediately reflected in the set value. At this time, the transmission timing may be gradually extended until there is no more data buffered in the buffer 250, thereby adjusting the transmission time so that the image can be displayed smoothly while there is buffered data. It is also possible to periodically check whether the differential time is shorter than the set value, and if so, reset it, and gradually shorten the transmission time difference so that the image can be displayed smoothly, while transmitting the image information to match the set time. Note that a threshold may be used to check the differential time. Note that since there is a buffering margin, checks may be performed periodically at predetermined intervals.

[0090] The differential time is used to adjust the frame output time. The video information management device 200 may hold the management data shown in FIG. 9. The management data includes a frame number, a frame acquisition time, frame data, the number of detected faces, a matching completion time, and a matching result for each detected face. The frame number, frame acquisition time, and frame data are set when a video frame is acquired. The number of detected faces is set when a face is detected. The matching completion time is set when matching of all detected faces has been completed. The matching result for each face is set at the time of matching.

[0091] The difference calculation unit 230 uses this management data to calculate the difference between the frame acquisition time and the matching completion time as the difference time. The output control unit 240 transmits the video frames using the calculated difference time. This allows transmission in a state where all data is complete. In this way, the video information management device 200 of this embodiment basically delays and transmits video frames by this difference time. However, the difference time varies. Therefore, for example, the delay may be changed by sampling the data. Alternatively, the data may be sampled and updated periodically. Furthermore, as described above, the update process may be performed in detail using buffered data. By performing the update process in detail, the video can be displayed as smoothly as possible when the delay time is updated.

[0092] [Hardware configuration] The video information management device 200 of each of the above embodiments and modifications may be realized by, for example, a general-purpose information processing device. The general-purpose information processing device includes, for example, a CPU (Central Processing Unit) 291, a main storage device (memory) 292, an auxiliary storage device 293, a communication I / F 294, and an expansion I / F 295, which are interconnected by an internal bus, as shown in FIG.

[0093] The CPU 291 realizes the above functions and comprehensively controls the entire device by, for example, loading a program stored in the auxiliary storage device 293 into the main storage device 292 and executing it. Note that the CPU 291 may be replaced by one or more processors such as an MPU (Micro Processing Unit).

[0094] The main storage device 292 is a memory such as a RAM (Random Access Memory), etc. The main storage device 292 is a work area when the CPU 291 processes programs executed by the video information management device 200, etc.

[0095] The auxiliary storage device 293 is, for example, a read-only memory (ROM), a hard disk drive (HDD), or a solid state drive (SSD). The auxiliary storage device 293 stores various programs executed by the video information management device 200. In addition, in each of the above embodiments and / or modifications, the buffer 250 and the differential time storage unit 260 may be constructed on the auxiliary storage device 293. The auxiliary storage device 293 may include a storage medium such as a flexible disk, a hard disk, an optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, or a DVD.

[0096] The programs stored in the auxiliary storage device 293 can be provided as program products recorded on non-transitory computer-readable recording media. The auxiliary storage device 293 can be used for medium- to long-term storage of various programs recorded on non-transitory computer-readable recording media.

[0097] The communication I / F 294 is an interface for inputting and outputting signals and data via wired or wireless communication. In this embodiment, data may be transmitted and received via this interface to and from the video capture device 110, the authentication device 120, and / or the display device 130.

[0098] The expansion I / F 295 is an interface for connecting an input device, an output device, etc. In this embodiment, for example, the video capture device 110, the authentication device 120, and / or the display device 130 may be connected to this interface.

[0099] Note that the hardware configuration of the video information management device 200 is not limited to this. Furthermore, each function (server) of each device may be implemented, for example, by an integrated circuit (IC) dedicated to each process, an application specific integrated circuit (ASIC), a system on chip (SOC), a field programmable gate array (FPGA), or the like.

[0100] In addition, a program for realizing each of the above functions of each device can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product.

[0101] Similarly, other components such as the authentication device 120 may also include a CPU 291 , a main storage device 292 , an auxiliary storage device 293 , a communication I / F 294 , and an expansion I / F 295 .

[0102] In the process flow used in the above explanation, multiple steps (processes) are described in order, but the order in which each step is performed is not limited to the order described. For example, the order of the steps shown in the figure can be changed to the extent that the content is not affected, such as performing each process in parallel.

[0103] Although the embodiments and modifications of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be modified in various ways that would be understandable to a person skilled in the art. Each embodiment and modification can be combined with other embodiments as appropriate. Furthermore, for example, the network configurations and element configurations shown in the drawings are examples intended to aid in understanding the present disclosure and are not limited to the configurations shown in these drawings.

[0104] Finally, preferred embodiments of the present disclosure will be summarized. Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. (Appendix 1) The video information management device a frame extraction unit that extracts predetermined frames from the input video; an authentication result acquisition unit that receives an authentication result for the extracted predetermined area within the frame; a difference calculation unit that calculates a difference time between an acquisition time of the extracted frame and a reception time that is a time when the authentication result is received; and an output control unit that time-shifts the video based on the difference time and outputs the video together with the authentication result. (Appendix 2) In the video information management device according to claim 1, It is desirable that the output control unit superimposes the authentication result on the time-shifted video and outputs the result. (Appendix 3) In the video information management device according to claim 1 or 2, the difference calculation unit calculates the difference time at predetermined time intervals, It is preferable that, when the differential time is calculated, the output control unit time-shifts the video based on the latest differential time. (Appendix 4) In the video information management device according to any one of Supplementary Note 1 to 3, further comprising a buffer unit for temporarily storing the input video; the buffer unit stores the acquisition time and frame number of the extracted frame; the authentication result acquisition unit receives the authentication result together with the frame number of the frame that is the basis of the authentication result; It is desirable that the difference calculation unit calculates the difference time using the acquisition time of the frame having the same frame number and the reception time of the authentication result. (Appendix 5) In the video information management device according to any one of Supplementary Note 1 to 4, It is desirable that the output control unit add a predetermined margin to the difference time to determine the amount of time shift. (Appendix 6) In the video information management device according to any one of Supplementary Note 1 to 5, When the differential time difference, which is the difference between the calculated differential time and the previously calculated differential time, is equal to or greater than a predetermined threshold, it is desirable that the output control unit repeats changing the differential time by an amount equal to or less than a predetermined change amount per unit time until the total amount of change becomes the differential time difference. (Appendix 7) The video authentication system A video information management device according to any one of Supplementary Notes 1 to 6; a video capture device for capturing the video; an authentication device that performs authentication on a predetermined area within the frame and outputs the authentication result to the video information management device; and a display device that displays the time-shifted video output from the video information management device and the authentication result. (Appendix 8) 8. The video authentication system of claim 7, It is desirable that the display device displays the authentication result by superimposing it on the moving image. (Appendix 9) Video authentication methods: The computer of the video information management device Extract a specific frame from the input video, receiving an authentication result for the extracted predetermined area within the frame; calculating a differential time between the acquisition time of the extracted frame and the reception time, which is the time when the authentication result is received; The video is time-shifted based on the difference time and output together with the authentication result. (Appendix 10) The program causes the computer of the video information management device to execute the video authentication method described in Supplementary Note 9. (Appendix 11) In the video information management device according to any one of Supplementary Note 1 to 6, It is desirable that the difference calculation unit add a predetermined margin to the calculated difference time. In addition, the forms of Supplementary Notes 9 and 10 can be expanded into the forms of Supplementary Notes 2-6 and 11, just like Supplementary Note 1.

[0105] The disclosures of the above-mentioned patent documents, etc. are incorporated herein by reference. Modifications and adjustments of the embodiments and variations are possible within the scope of this disclosure (including the claims), and further based on its basic technical concept. Furthermore, various combinations and selections of the various disclosed elements (including each element of each claim, each element of each embodiment or variation, each element of each drawing, etc.) are possible within the scope of this disclosure. In other words, this disclosure naturally includes various modifications and alterations that would be possible by a person skilled in the art in accordance with the entire disclosure, including the claims, and the technical concept. In particular, with regard to the numerical ranges described herein, any numerical value or subrange included within the range should be construed as being specifically described, even if not otherwise specified. [Explanation of symbols]

[0106] 100: Video authentication system, 110: Video acquisition device, 110a: Surveillance camera, 120: Authentication device, 121: Authentication unit, 122: Matching image storage unit, 130: Display device, 131: First display, 132: Second display, 200: Video information management device, 210: Frame extraction unit, 220: Authentication result acquisition unit, 230: Difference calculation unit, 240: Output control unit, 250: Buffer, 260: Difference time storage unit, 291: CPU, 292: Main storage unit, 293: Auxiliary storage unit, 294: Communication I / F, 295: Expansion I / F, 310: Frame, 311: Acquisition time, 312: Frame number, 313: Frame data, 320: Authentication result, 321: Receipt time, 322: Frame number, 323: Verification image, 324: Person's name

Claims

1. a frame extraction unit that extracts predetermined frames from the input video; an authentication result acquisition unit that receives an authentication result for the extracted predetermined area within the frame; a difference calculation unit that calculates a difference time between an acquisition time of the extracted frame and a reception time that is a time when the authentication result is received; an output control unit that time-shifts the video based on the difference time and outputs the video together with the authentication result.

2. 2. The video information management device according to claim 1, The output control unit superimposes the authentication result on the time-shifted video and outputs the superimposed result.

3. 2. The video information management device according to claim 1, the difference calculation unit calculates the difference time at predetermined time intervals, When the differential time is calculated, the output control unit time-shifts the video based on the latest differential time.

4. 2. The video information management device according to claim 1, further comprising a buffer unit for temporarily storing the input video; the buffer unit stores the acquisition time and frame number of the extracted frame; the authentication result acquisition unit receives the authentication result together with the frame number of the frame that is the basis of the authentication result; The difference calculation unit calculates the difference time using the acquisition time of the frame having the same frame number and the reception time of the authentication result.

5. 2. The video information management device according to claim 1, The video information management device, wherein the output control unit adds a predetermined margin to the difference time to determine the amount of time shift to be performed.

6. 2. The video information management device according to claim 1, When the differential time difference, which is the difference between the calculated differential time and the previously calculated differential time, is greater than or equal to a predetermined threshold, the output control unit changes the time by an amount equal to or less than a predetermined change amount per unit time, and repeats this process until the total amount of change becomes the differential time difference.

7. The video information management device according to claim 1; a video capture device for capturing the video; an authentication device that performs authentication on a predetermined area within the frame and outputs the authentication result to the video information management device; A video authentication system comprising: a display device that displays the time-shifted video output from the video information management device and the authentication result.

8. The video authentication system according to claim 7, The display device displays the authentication result by superimposing it on the video.

9. The computer of the video information management device Extract a specific frame from the input video, receiving an authentication result for the extracted predetermined area within the frame; calculating a differential time between the acquisition time of the extracted frame and the reception time, which is the time when the authentication result is received; The video authentication method includes time-shifting the video based on the difference time and outputting the video together with the authentication result.

10. A program causing a computer of a video information management device to execute the video authentication method according to claim 9.

Images