Video data structured sample processing method and system and readable medium

Abstract

The invention belongs to the field of video processing, and relates to a video data structured sample processing method and system and a readable medium, and the method comprises the steps: extracting a video file in a sample library according to the requirements of a packaging sample library, and taking the video file as first-level metadata; the method comprises the following steps: performing global structured processing on a to-be-packaged video file to generate a global structured description file, packaging the global structured description file and the video file together to obtain a primary packaging file, and endowing the primary packaging file with a unique identifier; selecting a plurality of target areas from the video file, performing target area structuring processing on each target area, creating an area structuring description file for each target area, and performing secondary packaging on the area structuring description file and the primary packaging file to obtain a secondary packaging file; and according to the unique identifier of the first-level packaging file, performing encryption processing on the second-level packaging file to complete the packaging of the sample library video file. The video file is converted into data information which is visual in form and concise in content, so that the feature expression of the video file is more comprehensive and specific.

Description

Technical Field

[0001] The invention relates to a method, system and readable medium for processing structured samples of video data, and belongs to the technical field of video processing. Background Art

[0002] Video data structuring is to label the targets in the unstructured data of the original video with corresponding labels, and turn them into structured data that can be searched by certain conditions. This process involves classifying the target objects in the video and extracting the trajectory information of the moving targets, so as to achieve structured analysis of the video content. Through such processing, video data can be effectively retrieved and analyzed, supporting multiple search methods, including type search, trajectory search, color search, and similarity search by target appearance. In addition, video data structuring also includes color correction to overcome the color cast problem that may be caused by different cameras. In traditional video surveillance systems, camera video information retrieval requires a lot of manpower and time, and the accuracy is not guaranteed. Summary of the invention

[0003] In view of the above problems, the purpose of the present invention is to provide a video data structured sample processing method, system and readable medium, which converts video files into data information with intuitive form and concise content, so that the feature description of the video file is more comprehensive and specific, and can be retrieved from the video file feature library according to the characteristics of the object to be retrieved.

[0004] To achieve the above-mentioned purpose, the present invention proposes the following technical scheme: a method for processing structured samples of video data, comprising the following steps: according to the requirements of the encapsulation sample library, extracting the video files in the sample library and using them as the first-level metadata; according to the first-level metadata, performing global structural processing on the video files to be encapsulated, generating a global structured description file, encapsulating the global structured file together with the video file to obtain a first-level encapsulation file, and assigning a unique identifier to the first-level encapsulation file; according to the first-level metadata, selecting several target areas from the video file, performing target area structural processing on each target area respectively, and creating a regional structured description file for each target area, performing secondary encapsulation on the first-level encapsulation file and the regional structured description file to obtain a second-level encapsulation file; according to the unique identifier of the first-level encapsulation file, encrypting the second-level encapsulation file to complete the encapsulation of the sample library video file.

[0005] Furthermore, the method for obtaining a first-level encapsulation file is: selecting at least one metadata from the first-level metadata; searching for a video frame corresponding to the selected metadata in the video file, taking the video frame as a target frame, and recording the position of the target frame; extracting metadata attribute information in each of the target frames, and generating a global structured description file for each of the target frames; encapsulating the video file and the global structured description file to obtain a first-level encapsulation file.

[0006] Furthermore, the method for generating a global structured description file is as follows: when the metadata appears for the first time, its attribute value is set to 1; the attribute value of the metadata in all target frames is calculated according to a ratio; and according to the time sequence of the target frames, a change curve corresponding to the target frames and the attribute values ​​is formed to obtain a global structured description file.

[0007] Furthermore, the attribute information includes at least one of position information, time information and size information.

[0008] Furthermore, the method of assigning a unique identifier to the first-level encapsulation file includes: performing SnowFlake algorithm processing on the attribute information to obtain an integer string, wherein the integer string is the unique identifier ID of the first-level encapsulation file.

[0009] Furthermore, the method for assigning the secondary encapsulation file is as follows: when the number of video frames in which the metadata appears continuously exceeds a preset frame number threshold, the segment of video frames is set as a target area; a region description file is created for each of the target areas, and an index is created for the region description files of all target areas in the same video file, the index is packaged with the region description file, and a regional structured description file is generated; the primary encapsulation file and the regional structured description file are packaged to obtain a secondary encapsulation file.

[0010] Furthermore, the zone description file includes: the position of the start frame, the position of the end frame and the attribute information of the metadata; the index is created based on the attribute information of the start frame or the metadata; the metadata includes at least one form of text, image, mark or sound.

[0011] Furthermore, the method for encrypting the secondary encapsulation file according to the unique identifier of the primary encapsulation file is as follows: encrypting the zone description file of each target zone respectively, each zone description file corresponds to a unique zone key; integrating all zone keys in the same video file into a zone key set; converting the unique identifier of the primary encapsulation file into a set of strings, and adding them to the zone key set as an encapsulation stamp; packaging the zone key set with the secondary encapsulation file to complete the video file encapsulation.

[0012] The present invention also discloses a video data structured sample processing system, comprising: a video extraction module, used for extracting video files in a sample library according to the requirements of the encapsulation sample library, and taking it as the first-level metadata; a first-level encapsulation file generation module, used for performing global structural processing on the video files to be encapsulated according to the first-level metadata, generating a global structured description file, encapsulating the global structured file with the video file to obtain a first-level encapsulation file, and giving a unique identifier to the first-level encapsulation file; a second-level encapsulation file generation module, used for selecting a number of target areas from the video file according to the first-level metadata, performing target area structural processing on each target area respectively, creating a regional structured description file for each target area, performing secondary encapsulation on the first-level encapsulation file and the regional structured description file to obtain a second-level encapsulation file; an encryption processing module, used for performing encryption processing on the second-level encapsulation file according to the unique identifier of the first-level encapsulation file, and completing the encapsulation of the sample library video file.

[0013] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned video data structured sample processing methods.

[0014] The technical solution of the present invention has at least the following technical effects or advantages: the present invention can quantitatively characterize the features of the video from the overall and local aspects respectively by performing a dual structured description on the video file, and convert the video file into data information with intuitive form and concise content, so that the feature description of the video file is more comprehensive and specific, and can be retrieved from the video file feature library according to the features of the object to be retrieved, and matched to the source video file, accurately and efficiently. And it provides a more efficient and accurate sample library for the machine learning technology of the later video analysis algorithm, so the video data structured sample encapsulation method is also very necessary. After the present invention extracts and stores the structured information of the camera video, it can directly search from the database by text information according to the features of the object to be retrieved, and match it to the source video or source image, accurately and efficiently. Therefore, the extraction and storage of structured information of the camera video is also very necessary. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a method for processing structured samples of video data in one embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or implying relative importance.

[0017] In order to solve the problems existing in the prior art that the camera video information retrieval in the traditional video surveillance system needs to consume a lot of manpower and time, and the accuracy cannot be guaranteed, the present invention relates to a video data structured sample packaging method, firstly, the video file is structurally described at the global level, and the overall characteristics of the video file are reflected by quantifiable features; then the target detection and target tracking are performed according to the selected target area, wherein, when tracking the target, the target tracking strategy is dynamically adjusted according to the target detection result; according to the target detection and target tracking results, the video file is structurally described at the regional level, and the regional characteristics are reflected by the position and attributes of the target area. The double structural description of the video file can be quantitatively characterized from the overall and local aspects respectively, and the video file is converted into data information with intuitive form and concise content, so that the feature description of the video file is more comprehensive and specific, and it can be retrieved from the video file feature library according to the characteristics of the object to be retrieved, and matched to the source video file, which is accurate and efficient. And it provides a more efficient and accurate sample library for the machine learning technology of the later video analysis algorithm, so the video data structured sample packaging method is also very necessary. The scheme of the present invention is described in detail by embodiments below.

[0018] Embodiment 1 This embodiment discloses a method for processing structured samples of video data, such as Figure 1 As shown, the following steps are included: S1 extracts the video files in the sample library according to the requirements of the encapsulated sample library and uses them as the first-level metadata.

[0019] The primary metadata includes at least one of text, images, tags or sounds, and its objects include people, animals, motor vehicles, etc.

[0020] S2 performs global structural processing on the video file to be encapsulated according to the primary metadata, generates a global structural description file, encapsulates the global structural file and the video file together to obtain a primary encapsulation file, and assigns a unique identifier to the primary encapsulation file.

[0021] The method to obtain the first-level packaging file is: S2.1 Select at least one metadata from the primary metadata.

[0022] S2.2 searches for the video frame corresponding to the selected metadata in the video file, takes the video frame as the target frame, and records the position of the target frame; extracts the metadata attribute information in each target frame, and generates a global structured description file for each target frame.

[0023] The attribute information includes at least one of position information, time information, and size information.

[0024] The method for generating a global structured description file is as follows: when metadata appears for the first time, its attribute value is set to 1; the attribute value of metadata in all target frames is calculated according to the ratio; according to the time sequence of the target frames, a change curve corresponding to the target frames and the attribute value is formed to obtain a global structured description file.

[0025] The method of this attribute value is: by inputting the image into the model, the convolutional neural network CNN is used to extract the important features in the image, which include but are not limited to edges, textures, colors, etc. The obtained features are input into the classifier, and the probability of each category is output. The model outputs a probability distribution value to indicate the possibility that the image belongs to a certain category. The label with the highest probability corresponding to the category will be used as the classification result of the image, and the classification result will be used as the attribute value of the metadata in the target frame.

[0026] S2.3 encapsulates the video file and the global structured description file to obtain a first-level encapsulation file.

[0027] The method of assigning a unique identifier to the first-level package file includes: performing SnowFlake algorithm processing on the attribute information to obtain an integer string, the integer string being the unique identifier ID of the first-level package file. In this embodiment, the unique identifier ID preferably adopts a 64-bit long integer string.

[0028] S3 selects several target areas from the video file according to the primary metadata, performs target area structuring processing on each target area, creates a regional structured description file for each target area, and re-encapsulates the primary encapsulation file and the regional structured description file to obtain a secondary encapsulation file.

[0029] The method for structuring the target area is to generate multiple candidate areas that may contain targets in the image. These areas can be obtained by sliding windows or feature-based region extraction methods. Each candidate area is classified to determine whether it contains a specific target, and the position of the target is accurately located, and a bounding box is output. The algorithm outputs a category label containing metadata, a threshold score, and the position of the target in the image (usually the coordinates of a rectangular box) to create a region description file.

[0030] The method to obtain the secondary packaging file is: S3.1 When the number of video frames in which the metadata appears continuously exceeds a preset frame number threshold, the segment of video frames is set as a target area; S3.2 creates a region description file for each target region, creates an index for the region description files of all target regions in the same video file, packages the index and the region description file, and generates a regional structured description file; The region description file includes: the position of the start frame, the position of the end frame and the attribute information of the metadata; the index is created according to the attribute information of the start frame or the metadata.

[0031] S3.3 encapsulates the first-level encapsulation file and the regional structured description file to obtain a second-level encapsulation file.

[0032] S4 encrypts the second-level encapsulation file according to the unique identifier of the first-level encapsulation file to complete the encapsulation of the sample library video file.

[0033] According to the unique identifier of the first-level package file, the encryption method for the second-level package file is as follows: S4.1 encrypts the zone description file of each target zone respectively, each zone description file corresponds to a unique zone key; S4.2 integrates all the zone keys in the same video file into a zone key set; S4.3 convert the unique identifier of the first-level packaged file into a set of hexadecimal strings and add them to the zone key set as a package stamp; S4.4 packages the zone key set and the secondary packaging file to complete the video file packaging.

[0034] Embodiment 2 Based on the same inventive concept, this embodiment discloses a video data structured sample processing system, including: The video extraction module is used to extract the video files in the sample library according to the requirements of the encapsulated sample library and use them as the first-level metadata; A first-level encapsulation file generation module is used to perform global structural processing on the video file to be encapsulated according to the first-level metadata, generate a global structured description file, encapsulate the global structured file with the video file to obtain a first-level encapsulation file, and give a unique identifier to the first-level encapsulation file; A secondary encapsulation file generation module is used to select several target areas from the video file according to the primary metadata, perform target area structuring processing on each target area, create a regional structured description file for each target area, and perform secondary encapsulation on the primary encapsulation file and the regional structured description file to obtain a secondary encapsulation file; The encryption processing module is used to encrypt the second-level encapsulation file according to the unique identifier of the first-level encapsulation file to complete the encapsulation of the sample library video file.

[0035] Embodiment 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned video data structured sample processing methods.

[0036] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0037] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0038] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0039] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for processing structured samples of video data, characterized in that: The following steps are involved: According to the requirements of the encapsulated sample library, extract the video files in the sample library and use them as the first-level metadata; According to the primary metadata, the video file to be encapsulated is globally structured to generate a global structured description file, the global structured file is encapsulated with the video file to obtain a primary encapsulation file, and a unique identifier is assigned to the primary encapsulation file; According to the primary metadata, several target areas are selected from the video file, target area structuring processing is performed on each target area, and a regional structured description file is created for each target area, and the primary encapsulation file and the regional structured description file are encapsulated twice to obtain a secondary encapsulation file; According to the unique identifier of the first-level encapsulation file, the second-level encapsulation file is encrypted to complete the encapsulation of the sample library video file.

2. The method for processing structured samples of video data according to claim 1, characterized in that: The method for obtaining the first-level packaging file is: selecting at least one metadata from the primary metadata; Searching for a video frame corresponding to the selected metadata in the video file, taking the video frame as a target frame, and recording a position of the target frame; Extracting metadata attribute information from each of the target frames, and generating a global structured description file for each of the target frames; The video file and the global structured description file are encapsulated to obtain a first-level encapsulation file.

3. The method for processing structured samples of video data according to claim 2, characterized in that: The method for generating a global structured description file is: When the metadata appears for the first time, its attribute value is set to 1; Calculate the attribute value of the metadata in all target frames according to the ratio; According to the time sequence of the target frames, a change curve corresponding to the target frames and the attribute values ​​is formed to obtain a global structured description file.

4. The video data structured sample processing method according to claim 2 or 3, characterized in that: The attribute information includes at least one of position information, time information and size information.

5. The video data structured sample processing method according to claim 2 or 3, characterized in that: The method for assigning a unique identifier to the first-level package file includes: performing SnowFlake algorithm processing on the attribute information to obtain an integer string, and the integer string is the unique identifier ID of the first-level package file.

6. The method for processing structured samples of video data according to claim 2, characterized in that: The method of assigning the secondary packaging file is: When the number of video frames in which the metadata appears continuously exceeds a preset frame number threshold, the segment of video frames is set as a target area; Creating a region description file for each target region respectively, creating an index for the region description files of all target regions in the same video file, packaging the index with the region description file, and generating a regional structured description file; The first-level encapsulation file and the regional structured description file are encapsulated to obtain a second-level encapsulation file.

7. The method for processing structured samples of video data according to claim 6, characterized in that: The zone description file includes: the position of the start frame, the position of the end frame and the attribute information of the metadata; the index is created according to the attribute information of the start frame or the metadata; the metadata includes at least one form of text, image, mark or sound.

8. The method for processing structured samples of video data according to claim 6, characterized in that: The method for encrypting the secondary encapsulation file according to the unique identifier of the primary encapsulation file is: Encrypting the zone description file of each target zone respectively, each zone description file corresponding to a unique zone key; All the zone keys in the same video file are grouped into a zone key set; Convert the unique identifier of the first-level packaged file into a set of character strings, and add the character strings as a package stamp to the zone key set; The zone key set and the secondary encapsulation file are packaged to complete the video file encapsulation.

9. A video data structured sample processing system, characterized in that: include: The video extraction module is used to extract the video files in the sample library according to the requirements of the encapsulated sample library and use them as the first-level metadata; A first-level encapsulation file generation module, used to perform global structural processing on the video file to be encapsulated according to the first-level metadata, generate a global structured description file, encapsulate the global structured file with the video file to obtain a first-level encapsulation file, and give a unique identifier to the first-level encapsulation file; A secondary encapsulation file generation module is used to select a number of target areas from the video file according to the primary metadata, perform target area structuring processing on each target area, create a regional structured description file for each target area, and perform secondary encapsulation on the primary encapsulation file and the regional structured description file to obtain a secondary encapsulation file; The encryption processing module is used to encrypt the second-level encapsulation file according to the unique identifier of the first-level encapsulation file to complete the encapsulation of the sample library video file.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the video data structured sample processing method according to any one of claims 1 to 8.

Images