Video archive generation method, apparatus, and storage medium

Abstract

The embodiment of the application discloses a video archive generation method and device and a storage medium, and belongs to the field of computer vision. In the embodiment of the application, a video set of each target person is determined through a video image of each target person captured by a plurality of monitoring devices in a place, and a video archive of each target person is generated and stored according to a video image in the video set of each target person and track information contained in the video image. In this way, not only the storage space can be saved, but also when the position information and the behavior information of a target person at any moment in the place are traced back, only the video archive of the target person needs to be checked, the query time is greatly shortened, and the query efficiency is improved.

Description

[0001] This application claims priority to Chinese Patent Application No. 202111086363.8, filed on September 16, 2021, entitled “Video File Generation Method, Apparatus and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer vision, and in particular to a method, apparatus and storage medium for generating video files. Background Technology

[0003] Currently, some event venues need to record the people entering the venue to generate video archives, so as to facilitate the tracking of a person's location and behavior information at any time within the venue.

[0004] In related technologies, multiple monitoring devices are used to capture video of the location, and the videos captured by the multiple monitoring devices are used as video archives for all personnel in the location.

[0005] However, the video files obtained by the above methods have storage redundancy and occupy a large amount of storage space. Moreover, when it is necessary to query the trajectory of a specific person, it can only be done by manually replaying the video files, which is time-consuming and laborious. Summary of the Invention

[0006] This application provides a video file generation method, apparatus, and storage medium, which can improve the problems of existing video files having storage redundancy, large storage space occupation, and time-consuming and laborious querying processes. The technical solution is as follows:

[0007] On the one hand, a method for generating video archives is provided, the method comprising:

[0008] Acquire multiple video images containing target personnel captured by multiple monitoring devices, and trajectory information of the target personnel contained in each video image, wherein the target personnel refers to personnel located within the deployment location of the multiple monitoring devices;

[0009] Based on the target person and their trajectory information contained in each video image, a video set corresponding to each target person is determined, wherein the video set includes video images of the corresponding target person captured by different monitoring devices;

[0010] Based on the video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images, a video file of the corresponding target person is generated.

[0011] Optionally, the trajectory information includes trajectory points and the corresponding time points. Based on the video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images, a video file of the corresponding target person is generated, including:

[0012] If the movement trajectory of the first target person is determined to be complete based on multiple trajectory points and the time point corresponding to each trajectory point, then the video images in the video set corresponding to the first target person are stitched together in chronological order to obtain the video file of the first target person, where the first target person is any target person.

[0013] Optionally, based on the video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images, a video file for the corresponding target person is generated, including:

[0014] If the movement trajectory of the first target person is incomplete based on multiple trajectory points of the first target person and the time point corresponding to each trajectory point, then an estimated trajectory point is generated based on the multiple trajectory points of the first target person, where the first target person is any target person.

[0015] Based on multiple trajectory points of the first target person, the estimated trajectory points, and the video set corresponding to the first target person, a video file of the first target person is generated.

[0016] Optionally, based on multiple trajectory points of the first target person, the estimated trajectory points, and the video set corresponding to the first target person, a video file of the first target person is generated, including:

[0017] Identify the target monitoring device whose coverage area includes the estimated trajectory points;

[0018] Determine the first time point corresponding to the previous trajectory point of the estimated trajectory point among the plurality of trajectory points, and the second time point corresponding to the next trajectory point of the estimated trajectory point;

[0019] Based on the identifier of the target monitoring device, extract the target video image located between the first time point and the second time point from the video captured by the target monitoring device;

[0020] By stitching together the video images from the video set corresponding to the first target person and the target video images in chronological order, a video file of the first target person is obtained.

[0021] Optionally, the method further includes:

[0022] If the difference between the time points corresponding to any two adjacent trajectory points among the multiple trajectory points of the first target person is not greater than the first threshold, then the movement trajectory of the first target person is determined to be complete.

[0023] If the difference between the time points corresponding to any two adjacent trajectory points of the first target person is greater than the first threshold, then the movement trajectory of the first target person is determined to be incomplete.

[0024] Optionally, determining the video set corresponding to each target person based on the target person and their trajectory information contained in each video image includes:

[0025] The feature data and trajectory information of the target person contained in the first video image and the feature data and trajectory information of the target person contained in the second video image are processed by a hybrid association model to obtain the similarity between the target person contained in the first video image and the target person contained in the second video image. The first video image and the second video image are any two video images among multiple video images containing the target person captured by the multiple monitoring devices.

[0026] If the similarity is greater than the second threshold, then the target person in the first video image and the target person in the second video image are determined to be the same target person, and the first video image and the second video image are added to the video set corresponding to the target person.

[0027] Optionally, the method further includes:

[0028] Obtain the registration information for each target person, the registration information including the facial image of the corresponding target person;

[0029] Based on the facial image in the registration information of each target person, the registration information and video file of each target person are matched;

[0030] The registration information and video file of each matched target person are stored accordingly.

[0031] On the other hand, a video file generation apparatus is provided, the apparatus comprising:

[0032] The first acquisition module is used to acquire multiple video images containing target personnel captured by multiple monitoring devices and trajectory information of the target personnel contained in each video image. The target personnel refers to personnel located in the deployment location of the multiple monitoring devices.

[0033] The determination module is used to determine the video set corresponding to each target person based on the target person and the trajectory information of the target person contained in each video image. The video set includes video images of the corresponding target person captured by different monitoring devices.

[0034] The generation module is used to generate video files for each target person based on the video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images.

[0035] Optionally, the trajectory information includes trajectory points and corresponding time points, and the generation module is used for:

[0036] If the movement trajectory of the first target person is determined to be complete based on multiple trajectory points and the time point corresponding to each trajectory point, then the video images in the video set corresponding to the first target person are stitched together in chronological order to obtain the video file of the first target person, where the first target person is any target person.

[0037] Optionally, the trajectory information includes trajectory points and corresponding time points, and the generation module is further configured to:

[0038] If the movement trajectory of the first target person is determined to be incomplete based on multiple trajectory points of the first target person and the time point corresponding to each trajectory point, then an estimated trajectory point is generated based on the multiple trajectory points of the first target person, where the first target person is any target person.

[0039] Based on multiple trajectory points of the first target person, the estimated trajectory points, and the video set corresponding to the first target person, a video file of the first target person is generated.

[0040] Optionally, the generation module is mainly used for:

[0041] Identify the target monitoring device whose coverage area includes the estimated trajectory points;

[0042] Determine the first time point corresponding to the previous trajectory point of the estimated trajectory point among the plurality of trajectory points, and the second time point corresponding to the next trajectory point of the estimated trajectory point;

[0043] Based on the identifier of the target monitoring device, extract the target video image located between the first time point and the second time point from the video captured by the target monitoring device;

[0044] By stitching together the video images from the video set corresponding to the first target person and the target video images in chronological order, a video file of the first target person is obtained.

[0045] Optionally, the device is also used for:

[0046] If the difference between the time points corresponding to any two adjacent trajectory points among the multiple trajectory points of the first target person is not greater than the first threshold, then the movement trajectory of the first target person is determined to be complete.

[0047] If the difference between the time points corresponding to any two adjacent trajectory points of the first target person is greater than the first threshold, then the movement trajectory of the first target person is determined to be incomplete.

[0048] Optionally, the determining module is mainly used for:

[0049] The feature data and trajectory information of the target person contained in the first video image and the feature data and trajectory information of the target person contained in the second video image are processed by a hybrid association model to obtain the similarity between the target person contained in the first video image and the target person contained in the second video image. The first video image and the second video image are any two video images among multiple video images containing the target person captured by the multiple monitoring devices.

[0050] If the similarity is greater than the second threshold, then the target person in the first video image and the target person in the second video image are determined to be the same target person, and the first video image and the second video image are added to the video set corresponding to the target person.

[0051] Optionally, the device further includes:

[0052] The second acquisition module is used to acquire the registration information of each target person, the registration information including the facial image of the corresponding target person;

[0053] The matching module is used to match the registration information and video files of each target person with the facial image in the registration information;

[0054] The storage module is used to store the registration information and video files of each matched target person.

[0055] On the other hand, a video file generation apparatus is provided, the apparatus comprising:

[0056] processor;

[0057] Memory used to store processor-executable instructions;

[0058] The processor executes executable instructions in the memory to perform the video file generation method described above.

[0059] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a computer, it implements the steps of the video file generation method described above.

[0060] On the other hand, a computer program product containing instructions is provided that, when run on a computer, causes the computer to perform the steps of the video file generation method described above.

[0061] The beneficial effects of the technical solutions provided in this application include at least the following:

[0062] In this embodiment, a video set for each target person is determined by video images captured by multiple monitoring devices within the venue. A video file for each target person is generated and stored based on the video images in the video set and the trajectory information contained in the video images. This not only saves storage space, but also allows for tracing the location and behavior information of a target person at any time within the venue by simply viewing the target person's video file, significantly shortening the query time and improving query efficiency. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a system architecture diagram involving a video file generation method provided in an embodiment of this application;

[0065] Figure 2 This is a flowchart of a video file generation method provided in an embodiment of this application;

[0066] Figure 3 This is a flowchart provided in an embodiment of the present application for generating a video file of a first target person based on video images in a video set corresponding to the first target person and the trajectory information of the first target person contained in the video images;

[0067] Figure 4 This is a schematic diagram of the structure of a video file generation device provided in an embodiment of this application;

[0068] Figure 5 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0070] Before providing a detailed explanation of the embodiments of this application, the system architecture involved in the embodiments of this application will be introduced first.

[0071] Figure 1 This is a system architecture diagram involving a video file generation method provided in an embodiment of this application. For example... Figure 1 As shown, the system includes multiple monitoring devices 101, a server 102, and a registration device 103. The multiple monitoring devices 101 refer to monitoring devices deployed in the same location, such as in an office, supermarket, or other activity venue. Furthermore, the multiple monitoring devices 101 can be connected to the server 102 via a wired or wireless network.

[0072] It should be noted that each of the multiple monitoring devices 101 can capture video images of each target person within its corresponding coverage area, and then send the video images of each target person to the server 102.

[0073] Server 102 receives video images of each target person sent by each monitoring device, and generates video files of each target person based on the video images of each target person and the trajectory information of the corresponding target person contained in the video images, using the method provided in this application embodiment.

[0074] The registration device 103 can be installed at the entrance of the location where multiple monitoring devices 101 are deployed, and it can be connected to the server 102. The registration device 103 is used to acquire the registration information of a target person when they enter the deployment location and sends this information to the server 102. The server 102 receives the registration information from the registration device 103 and, after subsequently identifying the target person's video file, matches and stores the registration information against the video file. The registration information includes the target person's facial image, and may also include the target person's name, identification, and other information. It should be noted that the facial image can be a human body image containing a face, or it can be an image containing only a face.

[0075] The monitoring device 101 can be a device with image capture capabilities, such as a face camera, a smart camera, or a regular camera. Furthermore, the monitoring device 101 can also have data processing capabilities. The server 102 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. The registration device 103 can be a user terminal device; for example, it can be a smartphone, tablet, or other terminal device capable of image capture. This embodiment of the application does not limit the specific device used.

[0076] Optionally, in some possible implementations, the system may also include one or more storage devices for storing video files of the various target individuals.

[0077] The video file generation method provided in the embodiments of this application will be described next.

[0078] Figure 2 This is a flowchart of a video file generation method provided in an embodiment of this application. This method can be applied to the server in the aforementioned system architecture. See also... Figure 2 The method includes the following steps:

[0079] Step 201: Acquire multiple video images containing the target person captured by multiple monitoring devices and the trajectory information of the target person contained in each video image.

[0080] In one implementation, each of the multiple monitoring devices can capture video images within its coverage area and extract facial images and the trajectory information of the target person corresponding to each facial image from each frame of the captured video image. Then, it sends the captured video images, extracted facial images, and corresponding trajectory information to the server. Correspondingly, the server receives the video images containing the target person, the target person's facial image, and the corresponding trajectory information sent by each monitoring device. Here, the target person refers to any person located within the deployment area of ​​the multiple monitoring devices.

[0081] Each monitoring device, after capturing video images, can perform facial recognition on the captured images and extract facial images from the corresponding video images based on the facial recognition results. After extracting the facial images, the monitoring device can also determine the position coordinates of the facial images in the video images and the timestamp of the video images. Then, the position coordinates of the facial images in the video images are used as the trajectory points of the target person represented by the facial images, and the timestamp of the video images is used as the time point corresponding to the trajectory points. Finally, the determined trajectory points of the target person represented by the facial images and the corresponding time points are used as the trajectory information corresponding to the facial images.

[0082] Optionally, the face image extracted from each frame of video image can be replaced with a human body image, or it can include both a face image and a human body image, or it can be other feature data that can characterize human features extracted from each frame of video image. This application embodiment does not limit this.

[0083] Alternatively, in another implementation, each of the multiple monitoring devices can capture video within its own coverage area and directly send the captured video images to the server. The video images captured by each monitoring device will include video images of target personnel entering the coverage area of ​​that device. Correspondingly, after receiving the video images sent by each monitoring device, the server can extract video images of different target personnel captured by each monitoring device, and extract the trajectory information of the target personnel from the video images of the different target personnel captured by each monitoring device.

[0084] In this implementation, the server can receive registration information from the registration devices regarding personnel entering the deployment area of ​​the multiple monitoring devices. Then, for each video image sent by a monitoring device, the server can extract video images containing the same face image from the video images captured by the corresponding monitoring device, based on the face image included in the registration information, thereby obtaining multiple frames of video images of the target person. Alternatively, the server can perform face recognition on the video images sent by each monitoring device to extract multiple frames containing faces from the received video images, and use these extracted frames as video images of the target person captured by the multiple monitoring devices. Then, the server can determine the trajectory information of the target person in each video image containing the target person using the aforementioned method.

[0085] Optionally, in another implementation, each of the multiple monitoring devices can capture video images of target individuals entering its coverage area, thereby obtaining video segments for each target individual, and then extracting the corresponding trajectory information of the target individual from each video segment. Each target individual's video segment contains multiple frames of video images of that target individual. Subsequently, each monitoring device can send its obtained video segments of each target individual and the corresponding trajectory information extracted from those video segments to a server. Accordingly, the server receives the video segments and trajectory information of each target individual sent by each monitoring device.

[0086] For example, in this embodiment of the application, when any target person enters the deployment area of ​​the multiple monitoring devices, the registration device located at the entrance of the deployment area can obtain the target person's registration information. This registration information may include the person's facial image. In addition, the registration information may also include the person's name, identification information, etc. For example, the person can enter the area by swiping their ID card on the registration device. In this case, the registration device can capture an image of the ID card and obtain the facial image and ID card number contained in the ID card image, thereby using the extracted facial image and ID card number as the person's registration information. Then, the registration device can send the captured facial image of the person entering the deployment area to each monitoring device. At this point, the person entering the deployment area is the target person.

[0087] After receiving facial images of various target personnel entering the premises, the monitoring equipment can identify each target personnel entering the coverage area of ​​the monitoring equipment based on the received facial images, and capture video images of the target personnel entering the coverage area to obtain video segments of the corresponding target personnel.

[0088] Optionally, the registration device may not send facial images of each target person entering the premises to the monitoring device. In this case, each monitoring device may also capture video images of the target person based on other characteristics of the target person entering the premises to obtain video segments of each target person. These other characteristics may include features such as body features and gait characteristics, which are not limited to specific features in this embodiment.

[0089] After obtaining the video segments of the target personnel it captured, each monitoring device can extract the trajectory information of the corresponding target personnel from each frame of the video image in each video segment, referring to the method described above.

[0090] Step 202: Based on the target personnel and their trajectory information contained in each video image, determine the video set corresponding to each target personnel. The video set corresponding to each target personnel includes video images of the corresponding target personnel captured by different monitoring devices.

[0091] In one implementation, the server can analyze video images captured by multiple monitoring devices using a hybrid association model to obtain a video set corresponding to each target person.

[0092] For example, the server can process the feature data and trajectory information of the target person in the first video image and the target person in the second video image using a hybrid association model to obtain the similarity between the target person in the first video image and the target person in the second video image. If the similarity is greater than a second threshold, it is determined that the target person in the first video image and the target person in the second video image are the same target person, and the first video image and the second video image are added to the video set corresponding to the target person. Here, the first video image and the second video image are any two video images containing the target person captured by multiple monitoring devices.

[0093] When the monitoring device sends captured video images, extracted facial images, and corresponding trajectory information to the server, the server first randomly selects two video images from multiple video images, designated as the first video image and the second video image. The first facial image extracted from the first video image is used as the feature data of a target person extracted from the first video image, and the second facial image extracted from the second video image is used as the feature data of a target person extracted from the second video image. Next, the server inputs the first facial image, the trajectory information of the target person corresponding to the first facial image, the second facial image, and the trajectory information of the target person corresponding to the second facial image into a hybrid association model. This model processes the input information and outputs the similarity between the target person corresponding to the first facial image and the target person corresponding to the second facial image. Then, the server compares the similarity output by the hybrid association model with a pre-set second threshold. If the similarity output by the hybrid association model is greater than the pre-set second threshold, it is determined that the target person in the first video image and the target person in the second video image are the same target person. In this case, both the first and second video images can be added to the video set of this target person. The server can then continue comparing other video images with those in the video set using the same method to determine if the target person in the other video images is the same person as the target person in the video set. If so, the corresponding video image is added to the video set, and this process continues until all other video images have been traversed. Afterward, the server can select two video images from the remaining video images and repeat the above process.

[0094] Optionally, if the similarity between the target person corresponding to the first face image and the target person corresponding to the second face image is less than or equal to the second threshold, it is considered that the target person contained in the first video image and the target person contained in the second video image are not the same target person. In this case, the similarity between the target person in the first video image and the target person in other video images can be re-determined by the above method, or the similarity between the target person in the second video image and the target person in other video images can be re-determined until other video images containing the same target person as the first video image or the second video image are found. Then, the steps described above are repeated to obtain a video set of the target person. The embodiments of this application will not be described in detail here.

[0095] It should be noted that the preset second threshold can be a value such as 80% or 90%, and this application embodiment does not limit it.

[0096] Optionally, in some possible cases, the feature data of the target person contained in the first video image and the second video image may also be a human body image of the target person or other data that can characterize the person's features. In this case, the analysis and processing can be performed in a manner similar to that of a facial image, which will not be elaborated further in this embodiment of the application.

[0097] Optionally, when analyzing video images using the hybrid association model described above, some video images may not be categorized into a specific video set due to blurry facial images of the target individuals or other reasons. In such cases, the facial images of the target individuals contained in the registration information obtained by the aforementioned registration device can also be used to associate the various video images.

[0098] For example, while inputting two video images and the trajectory information contained in the video images into the aforementioned hybrid association model, the facial image of the target person entering the venue, obtained by the registration device, can also be input into the hybrid association model. Then, the hybrid association model can output the similarity between the two video images and the similarity between each video image and the input facial image. If the similarity between the two video images is greater than a second threshold, these two video images can be added to the video set of the corresponding target person. If the similarity between the two video images is not greater than the second threshold, but the similarity between each video image and an input facial image is greater than a certain threshold, these two video images can also be considered as video images of the same target person and added to that target person's video set.

[0099] Optionally, if the monitoring equipment does not extract feature data from the captured video images, but instead directly sends the captured video images to the server, the server can first extract the feature data and trajectory information of the target personnel contained in the video images from the received video images from each monitoring equipment, and then generate a video set for each target personnel by referring to the above method.

[0100] Optionally, if the monitoring equipment obtains video segments of the target person, the server can analyze the video segments of each target person captured by multiple monitoring devices using a hybrid association model to determine the video set corresponding to each target person. For the method of determining the video set of the target person through video segments, the server can refer to the method described above for determining the similarity between two video images to determine the similarity between the two video segments, and then determine the video set for each target person based on the similarity of the target person's video segments; this will not be elaborated further here.

[0101] In another implementation, after the server obtains multiple video images containing target personnel captured by multiple monitoring devices and the trajectory information of the target personnel contained in the corresponding video images, it can also generate a video set for each target personnel through feature clustering.

[0102] For example, the server analyzes the features contained in each video image, and then uses one or more of these features as the basis for determining whether the target person in the video images is the same target person. Subsequently, video images with the same one or more features are regarded as video images of the same target person, and then the video images of the same target person are used to form a video set of that target person.

[0103] For example, if the monitoring device sends the captured video images, facial images extracted from the video images, and corresponding trajectory information to the server, the server can perform feature analysis on the received facial images and then identify video images with the same facial features as those of the same person. Alternatively, the server can also perform feature analysis on the facial images and their corresponding trajectory information, and then identify video images with the same facial features and spatially related trajectory information as those of the same target person.

[0104] Of course, if the monitoring equipment also extracts other feature data from the video images, such as human images and gait features, cluster analysis can be performed based on these features. This application embodiment will not be elaborated here.

[0105] Optionally, if the monitoring equipment does not extract feature data from the captured video images, but instead sends the captured video images directly to the server, the server can first extract the feature data and trajectory information of the target personnel contained in the video images from the received video images from each monitoring equipment, and then generate a video set for each target personnel by referring to the above feature clustering method.

[0106] Optionally, if the monitoring equipment obtains video segments of the target person, the server can also perform cluster analysis on the video segments of each target person captured by each monitoring equipment through feature clustering, thereby obtaining one or more video segments corresponding to the same target person, and the video set of the target person is composed of one or more video segments of the same target person.

[0107] In another implementation, the server can also directly obtain video images containing the target person that are similar to the face image from multiple video images based on the face image of each target person obtained by the aforementioned registration device, and use these video images as the video images of the target person corresponding to the face image, thereby forming a video set of the target person.

[0108] Step 203: Generate a video file for each target person based on the video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images.

[0109] After obtaining the video set corresponding to each target person, the server can generate a video file for the corresponding target person based on the video set and the trajectory information extracted from the video images contained in the video set.

[0110] The following explanation uses any target person in the location as an example. For ease of description, this target person will be referred to as the first target person. For this first target person, the server can... Figure 3 Steps 2031-2034 shown are used to obtain the video file of the first target person.

[0111] 2031: Determine whether the movement trajectory of the first target person is complete based on multiple trajectory points of the first target person and the time point corresponding to each trajectory point.

[0112] The server first arranges multiple trajectory points of the target person in chronological order according to their corresponding time points. Then, it calculates the difference between the time points corresponding to any two adjacent trajectory points of the first target person. If the difference between the time points corresponding to any two adjacent trajectory points is not greater than a first threshold, that is, if all calculated differences are not greater than the first threshold, then the movement trajectory of the first target person is determined to be complete. At this point, the server can execute step 2032.

[0113] Optionally, if the difference between the time points corresponding to any two adjacent trajectory points of the first target person is greater than the first threshold, that is, if there is a difference greater than the first threshold among the calculated differences, then it is determined that the movement trajectory of the first target person is incomplete. At this time, the server can execute steps 2033 and 2034.

[0114] For example, among multiple trajectory points of the first target person, any two adjacent trajectory points are the first trajectory point and the second trajectory point, respectively. If the time point corresponding to the first trajectory point differs from the time point corresponding to the second trajectory point by 2 seconds, and the first threshold is 1 second, it indicates that the movement trajectory of the target person is incomplete, and steps 2033 and 2034 can be executed. Optionally, if the difference between the time points corresponding to all adjacent trajectory points is less than 1 second, it indicates that the movement trajectory of the target person is complete.

[0115] 2032: The video images in the video set corresponding to the first target person are stitched together in chronological order to obtain the video file of the first target person.

[0116] Once the trajectory of the first target person is confirmed to be complete through the above step 2031, the server can stitch together the video images in the video set corresponding to the first target person in chronological order.

[0117] In some cases, two surveillance devices may have overlapping coverage areas. In such situations, a target person may be captured by both devices simultaneously within that overlapping area, resulting in overlapping segments in the video set of the target person. Therefore, in this embodiment, when stitching together the video images from the first target person's video set, duplicate frames captured at the same time can be removed, and then the frames are stitched together in chronological order.

[0118] 2033: Generate estimated trajectory points based on multiple trajectory points of the first target personnel.

[0119] If the server determines in step 2031 that the movement trajectory of the first target person is incomplete, the server can complete the trajectory of the first target person.

[0120] In one implementation, the server uses trajectory interpolation to generate estimated trajectory points to complete the movement trajectory of the first target person.

[0121] Specifically, the server can obtain the coordinates of two adjacent trajectory points whose time difference is greater than the first threshold, as determined in step 2031 above. Interpolation is then performed on these two adjacent trajectory points to obtain an estimated trajectory point between them. The number of estimated trajectory points between these two adjacent trajectory points can be one or more.

[0122] In another implementation, the server can also use trajectory prediction to complete the missing trajectory points, thereby obtaining estimated trajectory points.

[0123] The server can predict the coordinates of missing trajectory points based on the trajectory trends formed by multiple current trajectory points of the first target person, so as to obtain estimated trajectory points.

[0124] 2034: Generate a video profile of the first target person based on multiple trajectory points, estimated trajectory points, and the video set corresponding to the first target person.

[0125] After generating the estimated trajectory points, the server can identify the target monitoring device whose coverage area includes the estimated trajectory points; determine the first time point corresponding to the previous trajectory point of the estimated trajectory point and the second time point corresponding to the next trajectory point of the estimated trajectory point; based on the identifier of the target monitoring device, extract the target video image located between the first time point and the second time point from the video captured by the target monitoring device; and stitch the video images in the video set corresponding to the first target person and the target video image in chronological order to obtain the video file of the first target person.

[0126] The server stores a mapping relationship between the identifier of each of the multiple monitoring devices and its coverage area. The coverage area can be a coordinate region. The identifier of the monitoring device can be information that uniquely identifies the device. Based on this, the server can determine the coverage area containing the coordinates of the estimated trajectory points from this mapping relationship, and then use the identifier corresponding to the determined coverage area as the identifier of the target monitoring device.

[0127] Additionally, the server can determine the trajectory point whose corresponding time point is before and closest to the estimated trajectory point among multiple trajectory points extracted from the video set of the first target person, and use the time point corresponding to that trajectory point as the first time point. It can also determine the trajectory point whose corresponding time point is after and closest to the estimated trajectory point among multiple trajectory points extracted from the video set of the first target person, and use the time point corresponding to that trajectory point as the second time point.

[0128] Subsequently, the server can obtain the target video image located between the first time point and the second time point from the video captured by the target monitoring device based on the identifier of the target monitoring device. This target video image is the missing video image obtained based on the estimated trajectory points.

[0129] After obtaining the target video image, the server can refer to the method described in step 2032 above to stitch together the video images in the video set corresponding to the first target person and the target video image in chronological order to obtain the video file of the first target person.

[0130] For video sets of multiple target individuals, the server can refer to the above method to generate video files for each target individual, which will not be repeated here in the embodiments of this application.

[0131] Therefore, in this embodiment of the application, the missing motion trajectory in the target person's video file is completed by using the trajectory information in the target person's video image, and the video image is captured from the corresponding monitoring device by using the completed trajectory points. This can improve the problem of incomplete video file caused by incomplete detection of the target person's video image and ensure the integrity of the target person's video file.

[0132] After generating video files, the server can also store the video files of each target person.

[0133] As described in step 201 above, the registration device can acquire the registration information of each target person. In this case, the registration device can send the acquired registration information of each target person to the server. This registration information includes the facial image of the corresponding person. In addition, it may include various other information such as body image, fingerprints, name, and identification. Based on this, after generating video files for each target person, the server can match the registration information and video files of each target person according to the facial image in their registration information, and store the matched registration information and video files for each target person accordingly.

[0134] Taking the first target person as an example, the server can compare the face image of the first target person with the face images in each frame of the generated video file. If the number of video images in the video file that are similar to the face image of the first target person is greater than a certain threshold, then the video file can be used as the video file of the first target person, and the registration information of the first target person can be stored in correspondence with the video file.

[0135] In summary, the embodiments of this application determine the video set of each target person by capturing video images of each target person in multiple monitoring devices within the venue, and generate and store the video file of each target person based on the video images in the video set and the trajectory information contained in the video images. This not only saves storage space, but also allows for tracing the location and behavior information of a target person at any time within the venue by simply viewing the target person's video file, significantly shortening the query time and improving query efficiency.

[0136] In addition, in this embodiment of the application, the missing trajectory points in the target person's video file are completed by using the trajectory information in the target person's video image, and the video image is captured from the corresponding monitoring device by using the completed trajectory points. In this way, the problem of incomplete video file caused by incomplete detection of the target person's video image can be improved, and the integrity of the target person's video file can be guaranteed.

[0137] Next, the video file generation apparatus provided in the embodiments of this application will be described.

[0138] See Figure 4 This application provides a video file generation device 400, which includes:

[0139] The first acquisition module 401 is used to acquire multiple video images containing target personnel captured by multiple monitoring devices and trajectory information of the target personnel contained in each video image. The target personnel refers to the personnel located in the deployment location of the multiple monitoring devices.

[0140] The determination module 402 is used to determine the video set corresponding to each target person based on the target person and the trajectory information of the target person contained in each video image. The video set includes video images of the corresponding target person captured by different monitoring devices.

[0141] The generation module 403 is used to generate a video file of the target person based on the video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images.

[0142] Optionally, the trajectory information includes trajectory points and the corresponding time points, and the generation module 403 is used for:

[0143] If the movement trajectory of the first target person is determined to be complete based on multiple trajectory points and the time point corresponding to each trajectory point, then the video images in the video set corresponding to the first target person are stitched together in chronological order to obtain the video file of the first target person, where the first target person is any target person.

[0144] Optionally, the trajectory information includes trajectory points and the corresponding time points, and the generation module 403 is further used for:

[0145] If the movement trajectory of the first target person is incomplete based on multiple trajectory points of the first target person and the time point corresponding to each trajectory point, then an estimated trajectory point is generated based on the multiple trajectory points of the first target person, and the first target person is any target person.

[0146] Based on multiple trajectory points of the first target person, estimated trajectory points, and the video set corresponding to the first target person, a video file of the first target person is generated.

[0147] Optionally, the generation module 403 is mainly used for:

[0148] Identify the target monitoring equipment whose coverage area includes the estimated trajectory points;

[0149] Determine the first time corresponding to the previous trajectory point and the second time corresponding to the next trajectory point among multiple trajectory points;

[0150] Based on the identifier of the target monitoring device, extract the target video image located between the first time and the second time from the video captured by the target monitoring device;

[0151] By stitching together the video images from the video set corresponding to the first target person and the target video images in chronological order, a video file of the first target person is obtained.

[0152] The device 400 is also used for:

[0153] If the time difference between any two adjacent trajectory points of the first target person is not greater than the first threshold, then the movement trajectory of the first target person is determined to be complete.

[0154] If the time difference between any two adjacent trajectory points of the first target person is greater than a first threshold, then the movement trajectory of the first target person is determined to be incomplete.

[0155] Optionally, the determining module 402 is mainly used for:

[0156] The feature data and trajectory information of the target person contained in the first video image and the feature data and trajectory information of the target person contained in the second video image are processed by a hybrid association model to obtain the similarity between the target person contained in the first video image and the target person contained in the second video image. The first video image and the second video image are any two video images among multiple video images containing the target person captured by the multiple monitoring devices.

[0157] If the similarity is greater than the second threshold, the target person in the first video image and the target person in the second video image are determined to be the same target person, and the first video image and the second video image are added to the video set corresponding to the target person.

[0158] Optionally, the device 400 further includes:

[0159] The second acquisition module is used to acquire the registration information of each target person, including the facial image of the corresponding target person;

[0160] The matching module is used to match the registration information and video files of each target person with the facial image in the registration information;

[0161] The storage module is used to store the registration information and video files of each matched target person.

[0162] In summary, in this embodiment, a video set for each target person is determined by capturing video images of each target person using multiple monitoring devices within the venue. A video file for each target person is generated and stored based on the video images in the video set and the trajectory information contained in the video images. This not only saves storage space, but also allows for tracing the location and behavior information of a target person at any time within the venue by simply viewing the target person's video file, significantly shortening the query time and improving query efficiency.

[0163] It should be noted that the video file generation device provided in the above embodiments is only illustrated by the division of the above functional modules when generating video files. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video file generation device and the video file generation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0164] Figure 5This is a schematic diagram of a server structure according to an exemplary embodiment. The server in the above embodiment can perform functions such as... Figure 5 The server shown is used to implement this. This server can be a server in a backend server cluster. Specifically:

[0165] Server 500 includes a Central Processing Unit (CPU) 501, a system memory 504 including Random Access Memory (RAM) 502 and Read-Only Memory (ROM) 503, and a system bus 505 connecting the system memory 504 and the CPU 501. Server 500 also includes a basic input / output system (I / O system) 506 that facilitates the transfer of information between various devices within the computer, and a mass storage device 507 for storing the operating system 513, application programs 514, and other program modules 515.

[0166] The basic input / output system 506 includes a display 508 for displaying information and an input device 509 for user input, such as a mouse or keyboard. Both the display 508 and the input device 509 are connected to the central processing unit 501 via an input / output controller 510 connected to the system bus 505. The basic input / output system 506 may also include the input / output controller 510 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 510 also provides output to a display screen, printer, or other types of output devices.

[0167] Mass storage device 507 is connected to central processing unit 501 via a mass storage controller (not shown) connected to system bus 505. Mass storage device 507 and its associated computer-readable media provide non-volatile storage for server 500. That is, mass storage device 507 may include computer-readable media (not shown) such as hard disk or CD-ROM (CompactDisc Read-Only Memory) drive.

[0168] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage devices, CD-ROM, DVD (Digital Versatile Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 504 and mass storage device 507 described above can be collectively referred to as memory.

[0169] According to various embodiments of this application, server 500 can also be connected to a remote computer on a network, such as the Internet. That is, server 500 can be connected to network 512 via network interface unit 511 connected to system bus 505, or it can use network interface unit 511 to connect to other types of networks or remote computer systems (not shown).

[0170] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU. The one or more programs contain instructions for performing the video file generation method provided in the embodiments of this application.

[0171] This application also provides a computer-readable storage medium that, when executed by a server's processor, enables the server to perform the live video data generation method provided in the above embodiments. For example, the computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device. It is worth noting that the computer-readable storage medium mentioned in this application embodiment may be a non-volatile storage medium; in other words, it may be a non-transient storage medium.

[0172] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0173] That is, in some embodiments, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the video file generation method provided in the above embodiments.

[0174] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the video images, trajectory information, feature data, registration information, etc. involved in the embodiments of this application were all obtained with full authorization.

[0175] The above description is not intended to limit the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for generating video archives, characterized in that, The method includes: The method acquires multiple video images containing target personnel captured by multiple monitoring devices, and trajectory information of the target personnel contained in each video image. The target personnel refers to personnel located within the deployment location of the multiple monitoring devices. The trajectory information includes trajectory points of the target personnel and the time points corresponding to the trajectory points. The trajectory points are the position coordinates of the target personnel's feature data in the video images, and the time points corresponding to the trajectory points are the time points corresponding to the video images. The feature data includes at least one of facial images and human body images. Based on the target person and their trajectory information contained in each video image, a video set corresponding to each target person is determined, wherein the video set includes video images of the corresponding target person captured by different monitoring devices; If the difference between the time points corresponding to any two adjacent trajectory points of the first target person is greater than a first threshold, then the movement trajectory of the first target person is determined to be incomplete; then, based on the multiple trajectory points of the first target person, an estimated trajectory point is generated using trajectory interpolation or trajectory prediction, where the first target person is any target person. Identify the target monitoring device whose coverage area includes the estimated trajectory points; Determine a first time point corresponding to the trajectory point that is before and closest to the estimated trajectory point among the plurality of trajectory points, and a second time point corresponding to the trajectory point that is after and closest to the estimated trajectory point; Based on the identifier of the target monitoring device, extract the target video image located between the first time point and the second time point from the video captured by the target monitoring device; The video images in the video set corresponding to the first target person and the target video images are spliced ​​together in chronological order to obtain the video file of the first target person; The registration information of each target person entering the deployment site of the multiple monitoring devices is obtained, and the registration information includes the facial image of the corresponding target person; Based on the facial image in the registration information of each target person, the registration information and video file of each target person are matched; If the number of video images in the video file of the first target person that are similar to the facial images in the registration information of the first target person exceeds a certain threshold, the video file of the first target person will be stored in correspondence with the registration information of the first target person.

2. The method according to claim 1, characterized in that, The trajectory information includes trajectory points and corresponding time points. The step of generating a video file for each target person based on video images in the video set corresponding to each target person and the trajectory information of the target person contained in the corresponding video images includes: If the movement trajectory of the first target person is determined to be complete based on multiple trajectory points and the time point corresponding to each trajectory point, then the video images in the video set corresponding to the first target person are stitched together in chronological order to obtain the video file of the first target person, where the first target person is any target person.

3. The method according to claim 1 or 2, characterized in that, The method further includes: If the difference between the time points corresponding to any two adjacent trajectory points among the multiple trajectory points of the first target person is not greater than the first threshold, then the movement trajectory of the first target person is determined to be complete.

4. The method according to claim 1, characterized in that, The step of determining the video set corresponding to each target person based on the target person and the corresponding trajectory information contained in each video image includes: The feature data and trajectory information of the target person contained in the first video image and the feature data and trajectory information of the target person contained in the second video image are processed by a hybrid association model to obtain the similarity between the target person contained in the first video image and the target person contained in the second video image. The first video image and the second video image are any two video images among multiple video images containing the target person captured by the multiple monitoring devices. If the similarity is greater than the second threshold, then the target person in the first video image and the target person in the second video image are determined to be the same target person, and the first video image and the second video image are added to the video set corresponding to the target person.

5. A video file generation device, characterized in that, The device includes: The first acquisition module is used to acquire multiple video images containing target personnel captured by multiple monitoring devices and trajectory information of the target personnel contained in each video image. The target personnel refers to personnel located in the deployment location of the multiple monitoring devices. The trajectory information includes the trajectory points of the target personnel and the time points corresponding to the trajectory points. The trajectory points are the position coordinates of the target personnel's feature data in the video images, and the time points corresponding to the trajectory points are the time points corresponding to the video images. The feature data includes at least one of facial images and human body images. The determination module is used to determine the video set corresponding to each target person based on the target person and the trajectory information of the target person contained in each video image. The video set includes video images of the corresponding target person captured by different monitoring devices. The generation module is configured to: determine that the motion trajectory of the first target person is incomplete if the difference between the time points corresponding to any two adjacent trajectory points among the multiple trajectory points of the first target person is greater than a first threshold; generate estimated trajectory points based on the multiple trajectory points of the first target person, where the first target person is any target person; determine the identifier of the target monitoring device whose coverage area includes the estimated trajectory points; determine the first time point corresponding to the trajectory point that is before and closest to the estimated trajectory point among the multiple trajectory points, and the second time point corresponding to the trajectory point that is after and closest to the estimated trajectory point; extract the target video image located between the first time point and the second time point from the video captured by the target monitoring device based on the identifier of the target monitoring device; and stitch the video images in the video set corresponding to the first target person and the target video image in chronological order to obtain the video file of the first target person. The second acquisition template is used to acquire the registration information of each target person entering the deployment site of the multiple monitoring devices, and the registration information includes the facial image of the corresponding target person; The matching module is used to match the registration information and video files of each target person based on the facial image in the registration information of each target person; The storage module is configured to store the video file of the first target person in correspondence with the registration information of the first target person if the number of video images in the video file of the first target person that are similar to the facial images in the registration information of the first target person exceeds a certain threshold.

6. The apparatus according to claim 5, characterized in that, The determining module is mainly used for: The feature data and trajectory information of the target person contained in the first video image and the feature data and trajectory information of the target person contained in the second video image are processed by a hybrid association model to obtain the similarity between the target person contained in the first video image and the target person contained in the second video image. The first video image and the second video image are any two video images among multiple video images containing the target person captured by the multiple monitoring devices. If the similarity is greater than the second threshold, then the target person in the first video image and the target person in the second video image are determined to be the same target person, and the first video image and the second video image are added to the video set corresponding to the target person.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a computer, implements the steps of the method according to any one of claims 1-4.

Images