Video identification information addition method, apparatus and device, and storage medium and product

Abstract

Provided in the present application are a video identification information addition method, apparatus and device, and a storage medium and a product. The method comprises: acquiring dynamic / static classification information of a group of pictures in a video, wherein the dynamic / static classification information is obtained by means of performing classification according to the degree of motion of the group of pictures in the video; on the basis of the dynamic / static classification information, classifying the group of pictures in the video into a group of dynamic pictures and a group of static pictures; on the basis of a preset dynamic picture addition frequency, adding identification information to specific frames in the group of dynamic pictures; and on the basis of a preset static picture addition frequency, adding the identification information to specific frames in the group of static pictures, so as to obtain a video to which the identification information has been added.

Description

Method, device, equipment, storage medium and product for adding video identification information

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202410340204.3, application date March 25, 2024, and application name “Method, device, equipment, storage medium and product for adding video identification information”, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application by introduction. Technical Field

[0003] The present application relates to the field of network security technology, and in particular to a method, apparatus, device, storage medium and product for adding video identification information. Background Art

[0004] With the development of network technology and the improvement of network infrastructure, services such as live streaming, video conferencing, and real-time home security monitoring are becoming increasingly sophisticated. However, the risk of data leakage is also increasing, which may lead to various issues related to information privacy and copyright protection.

[0005] Currently, videos embed identification information, such as invisible watermarks, to trace their source and address data leaks. Identification information has important applications in copyright protection, content authentication, and source tracking. For example, adding an invisible watermark to a video does not affect the user's viewing experience. However, real-time video services such as live broadcasts, video conferencing, and real-time home security monitoring are longer than traditional videos and contain enormous amounts of data. Embedding an invisible watermark requires calculations for each frame, consuming computing resources and time, resulting in a significant increase in computing power costs. This also fails to meet the real-time requirements of the service, making invisible watermarking inefficient.

[0006] Therefore, adding authentication information in the prior art cannot meet the business demand of high real-time performance, and the efficiency of adding authentication information is relatively low. Summary of the Invention

[0007] The present application provides a method, apparatus, device, storage medium and product for adding video identification information, which can improve the efficiency of adding identification information and meet the real-time requirements of the business.

[0008] In a first aspect, the present application provides a method for adding video identification information, the method comprising:

[0009] Obtaining dynamic and static classification information of an image group in a video, wherein the dynamic and static classification information is obtained by classifying the image group according to a degree of motion in the video;

[0010] According to the dynamic and static classification information, the image group in the video is divided into a dynamic image group and a static image group;

[0011] According to a preset dynamic image adding frequency, identification information is added to specific frames of the dynamic image group; according to a preset static image adding frequency, identification information is added to specific frames of the static image group to obtain a video with added identification information, wherein the preset dynamic image adding frequency and the preset static image adding frequency are less than the number of video frames.

[0012] The above-described embodiment of the present application obtains dynamic and static classification information of image groups in a video and then divides the video information into dynamic image groups and static image groups based on the dynamic and static classification information. Then, based on the dynamic image and static image addition frequencies, identification information is added to specific frames of the dynamic image group and the static image group, respectively, to obtain a video with the identification information added. Because the preset dynamic image addition frequency and the preset static image addition frequency are less than the number of video frames, the number of added frames is reduced, thereby reducing the computing power consumption of adding identification information and the time required to add identification information. This can improve the efficiency of adding identification information and meet the high real-time requirements of the business.

[0013] In some possible implementations, before obtaining dynamic and static classification information of the image group in the video, the method further includes:

[0014] Acquiring video information of a video, the video information including motion vector information generated during encoding of a group of images in the video, the motion vector information representing a degree of motion of the group of images;

[0015] The image groups in the video are divided according to the motion vector information to obtain dynamic and static classification information of the image groups in the video.

[0016] The above-mentioned implementation mode of the present application classifies the dynamic situation of the video by directly obtaining the motion vector information of the image group in the video during the encoding process. The process of processing information is faster and more direct, which can speed up the processing of information.

[0017] In some possible implementations, obtaining video information of a video includes:

[0018] Obtaining the positional relationship between each image group and the corresponding reference image group in the video information;

[0019] determining whether there is a motion relationship between each image group and the corresponding reference image group based on the positional relationship;

[0020] In the case where there is a motion relationship, motion vector information corresponding to each image group is obtained according to the corresponding positional relationship.

[0021] The above-mentioned embodiment of the present application compares the positional relationship between each different image block in the image group and the image blocks in the reference image group to determine whether there is a motion relationship, and then obtains the motion vector information corresponding to each image group. The motion vector information can be accurately obtained.

[0022] In some possible implementations, dividing the image groups in the video according to the motion vector information to obtain the image groups in the video includes:

[0023] The motion vector information is input into a dynamic and static video classification model, and the dynamic and static video classification model is used to classify the image groups in the video according to the degree of motion represented by the motion vector information to obtain dynamic and static classification information corresponding to the image groups.

[0024] The above embodiment of the present application uses a pre-trained static and dynamic video classification model to classify the image groups in the video into static and dynamic based on motion vector information, thereby obtaining classification results quickly and accurately.

[0025] In some possible implementations, inputting the motion vector information into a dynamic and static video classification model, classifying the image groups in the video according to the degree of motion represented by the motion vector information using the dynamic and static video classification model, and obtaining dynamic and static classification information corresponding to the image groups includes:

[0026] determining the total number of motion vectors in the motion vector information by using a dynamic and static video classification model;

[0027] Calculating an average value of each motion displacement value in the motion vector information;

[0028] The dynamic and static classification sub-model in the dynamic and static video classification model is used to perform dynamic and static classification on the image group in the video based on the total number of the motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information to obtain the dynamic and static classification information corresponding to the image group.

[0029] The above-described embodiment of the present application obtains the total number of motion vectors and the average value of each motion displacement value, and then classifies the video into static and dynamic based on four features: the total number of motion vectors, the average value of the displacement, the maximum displacement value in the motion vector information, and the minimum displacement value in the motion vector information. By using more comprehensive motion vector information, dynamic and static classification information corresponding to the image group can be more accurately obtained.

[0030] In some possible implementations, before using the dynamic and static classification submodel in the dynamic and static video classification model to perform dynamic and static classification on the image group in the video based on the total number of motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information, and obtaining the dynamic and static classification information corresponding to the image group, the method further includes:

[0031] Get historical video data;

[0032] Dividing the historical video data into a plurality of image groups, and obtaining annotations of the plurality of image groups, wherein the annotations are dynamic image groups or static image groups;

[0033] According to the motion vector information of each of the image groups, a total number of corresponding motion vectors, a maximum value of displacement, a minimum value of displacement, and an average value of displacement are obtained;

[0034] The total number of motion vectors, the maximum value of the displacement, the minimum value of the displacement and the average value of the displacement of each image group are used as input data, and the initial support vector machine is trained according to the corresponding labels of the image group to obtain a dynamic and static classification sub-model.

[0035] The above-mentioned embodiment of the present application obtains historical video data and annotates the dynamic image group or static image group in each image group in the historical video data, and then obtains the total number of corresponding motion vectors, the maximum displacement, the minimum displacement, and the average displacement based on the motion vector information of each image group. These are then used to train an initial support vector machine, thereby obtaining a dynamic and static classification sub-model. The initial support vector machine can be trained using historical video data to obtain an accurate dynamic and static classification sub-model.

[0036] In some possible implementations, the preset dynamic image adding frequency includes adding identification information to each frame whose interval number from the reference frame is an integer multiple of a first value;

[0037] The preset static image adding frequency includes adding identification information in each frame whose interval number with the reference frame is an integer multiple of a second value, and the second value is greater than the first value.

[0038] The above embodiment of the present application adds identification information to frames that are integer multiples of the interval between the dynamic image group and the static image group, which can make the identification information more evenly distributed in the video.

[0039] In some possible implementations, before obtaining the dynamic and static classification information of the image group in the video, the method further includes:

[0040] Acquiring video information of an original video, the video information including motion vector information generated during encoding of a group of images in the original video, the motion vector information representing a degree of motion of the group of images;

[0041] Dividing the image groups in the original video according to the motion vector information to obtain initial dynamic and static classification information of the image groups in the original video;

[0042] Adding the initial dynamic and static classification information to the supplementary enhancement information frame;

[0043] The initial supplementary enhancement information frame is added to the original video information to obtain the video, and the initial dynamic and static classification information is the dynamic and static information.

[0044] The above-mentioned embodiment of the present application adds the dynamic and static classification information to the supplementary enhancement information frame and then adds the original video information, which makes the dynamic and static classification information easier to obtain and use.

[0045] In a second aspect, the present application provides a device for adding video identification information, the device comprising:

[0046] an acquisition module configured to acquire dynamic and static classification information of an image group in a video, wherein the dynamic and static classification information is obtained by classifying the image group according to a degree of motion in the video;

[0047] a classification module configured to divide the image group in the video into a dynamic image group and a static image group according to the dynamic and static classification information;

[0048] An adding module is configured to add identification information to specific frames of the dynamic image group according to a preset dynamic image adding frequency; and to add identification information to specific frames of the static image group according to a preset static image adding frequency, to obtain a video with added identification information, wherein the preset dynamic image adding frequency and the preset static image adding frequency are less than the number of video frames.

[0049] In a third aspect, the present application provides a device for adding video identification information, the device comprising: a processor, and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the method for adding video identification information as described above.

[0050] In a fourth aspect, the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the method for adding video identification information as described above is implemented.

[0051] In a fifth aspect, the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method for adding video identification information as described above.

[0052] The above-described embodiment of the present application obtains dynamic and static classification information of image groups in a video and then divides the video information into dynamic image groups and static image groups based on the dynamic and static classification information. Then, based on the dynamic image and static image addition frequencies, identification information is added to specific frames of the dynamic image group and the static image group, respectively, to obtain a video with the identification information added. Because the preset dynamic image addition frequency and the preset static image addition frequency are less than the number of video frames, the number of added frames is reduced, thereby reducing the computing power consumption of adding identification information and the time required to add identification information. This can improve the efficiency of adding identification information and meet the high real-time requirements of the business. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present application can be better understood from the following description of specific embodiments of the present application in conjunction with the accompanying drawings, in which:

[0054] Other features, objects and advantages of the present application will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings, in which the same or similar reference numerals represent the same or similar features.

[0055] FIG1 is a flow chart of a method for adding video identification information provided by one embodiment of the present application;

[0056] FIG2 is a schematic diagram of motion vectors in one embodiment of the present application;

[0057] FIG3 is a flow chart of another method for adding video identification information provided by an embodiment of the present application;

[0058] FIG4 is a schematic structural diagram of an apparatus for adding video identification information provided by one embodiment of the present application;

[0059] FIG5 is a schematic diagram of the hardware structure of a device for adding video identification information provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the application. However, it will be apparent to those skilled in the art that the application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the application by illustrating the examples of the present application. The application is by no means limited to any specific configuration and algorithm proposed below, but covers any modification, replacement and improvement of elements, parts and algorithms without departing from the spirit of the application. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid causing unnecessary ambiguity to the application.

[0061] Supplemental Enhancement Information (SEI): defined in the video bitstream, providing a method for adding information to the video bitstream.

[0062] With the rapid development of technology and the increasing needs of consumers, the current scale of home security video cloud storage is enormous. However, the risk of data leakage and tampering is also increasing. The massive amount of video poses a challenge to maintaining video data security. The frequent incidents of security video leakage have led users to question the safety of home security systems.

[0063] To address these issues, invisible information can be embedded in videos to trace their origins. This invisible video watermarking technology has important applications in copyright protection, content authentication, and source tracing. Furthermore, this invisible video watermark does not affect the user's viewing experience. However, due to the massive scale of security video, embedding watermarks consumes computing resources and time, significantly increasing computing power costs. This can also cause video playback to freeze and long wait times when downloading or sharing videos.

[0064] In addition, the downloading and playback of security videos need to meet real-time requirements. However, adding invisible watermarks to videos is time-consuming and computationally intensive. Current existing technologies generally focus on invisible watermark algorithms and do not consider real-time issues during video playback.

[0065] Research into the above issues has revealed that static video accounts for a significant proportion of surveillance scenarios. Since static video is often a still image and provides less information, a distinction between dynamic and static video is necessary to conserve computing power and ensure real-time performance. The purpose of adding watermarks is to facilitate source tracing. Since static video, with its low information content, is unlikely to be circulated, it is unnecessary to consume a large amount of resources by adding watermarks. Furthermore, since individual video frames are rarely circulated, adding a watermark to every frame is unnecessary.

[0066] In one embodiment of the method for adding video identification information of the present application, reference may be made to FIG. 1 . In this embodiment, the method may be executed by the above-mentioned apparatus for adding video identification information, and the method includes the following steps S101 to S103 .

[0067] Step S101: obtaining dynamic and static classification information of an image group in a video, wherein the dynamic and static classification information is obtained by dividing the image group in the video according to the degree of motion.

[0068] A GOP is a collection of images that are closely connected in time, typically a continuous sequence of frames in a video. During video encoding, the video is typically divided into multiple GOPs.

[0069] Specifically, a decoding operation is performed to convert the video file into a series of image groups. The corresponding dynamic and static classification information for each image group is then obtained. For example, custom data containing dynamic and static classification information can be inserted into the video information. After receiving the video, the dynamic and static classification information can be extracted.

[0070] Step S102: dividing the image group in the video into a dynamic image group and a static image group according to the dynamic and static classification information.

[0071] Specifically, the acquired dynamic and static classification information can be used to classify each image group into dynamic and static images. The image group with a large motion amplitude is classified as a dynamic image group, and the image group with a small motion amplitude is classified as a static image group.

[0072] Step S103: adding identification information to specific frames of the dynamic image group according to a preset dynamic image adding frequency; adding identification information to specific frames of the static image group according to a preset static image adding frequency, to obtain a video with added identification information.

[0073] The identification information can be a specific mark, watermark or other form of identification code.

[0074] Specifically, specific frames are selected from the dynamic image group according to a preset dynamic image addition frequency, and then the identification information is added to these selected frames. A specific selection method can be to randomly select several frames from the image group according to a probability value. In particular, if the video contains sound information, the sound information of the video can be obtained, and the watermark is preferably added to the image corresponding to the sound information.

[0075] Similarly, specific frames are selected from the static image group according to a preset static image addition frequency, and identification information is then added to these selected frames. A sequence of video frames after the addition of identification information is obtained, and the video with the added identification information is reconstructed. Specifically, the preset dynamic image addition frequency and the preset static image addition frequency are both smaller than the number of video frames, eliminating the need to add identification information to every frame of the video.

[0076] The above-described embodiment of the present application obtains dynamic and static classification information of image groups in a video and then divides the video information into dynamic image groups and static image groups based on the dynamic and static classification information. Then, based on the dynamic image and static image addition frequencies, identification information is added to specific frames of the dynamic image group and the static image group, respectively, to obtain a video with the identification information added. Because the preset dynamic image addition frequency and the preset static image addition frequency are less than the number of video frames, the number of added frames is reduced, thereby reducing the computing power consumption of adding identification information and the time required to add identification information. This can improve the efficiency of adding identification information and meet the high real-time requirements of the business.

[0077] In some embodiments, before obtaining dynamic and static classification information of the image group in the video, the method further includes:

[0078] Video information of a video is obtained, where the video information includes motion vector information generated during encoding of a group of images in the video, where the motion vector information represents a degree of motion of the group of images.

[0079] Motion vector information is vector data used in video coding to represent the motion relationship between two adjacent frames. To reduce the redundancy of video data, video coding often uses compression methods based on motion compensation. This involves predicting the current frame using one or more previous frames, and then encoding and transmitting the prediction error. To accurately describe the difference between the current frame and a reference frame, the motion vector of the current frame relative to the reference frame must be calculated, encoded, and transmitted.

[0080] Specifically, video information is encoded, and during this process, motion estimation is performed on each image frame. By comparing the current frame with adjacent frames, the motion vector information of the image group is obtained. Common methods include optical flow and global motion estimation.

[0081] The image groups in the video are divided according to the motion vector information to obtain dynamic and static classification information of the image groups in the video.

[0082] Specifically, after obtaining the motion information of each frame, an appropriate motion vector threshold is set based on the application requirements and actual conditions. This threshold can be determined based on the size, direction, or other characteristics of the motion vector to distinguish between "moving" and "still" image groups.

[0083] For example, a threshold value can be applied to the motion vector of each image group, classifying image groups with motion vectors greater than the threshold as dynamic image groups, while image groups with motion vectors less than the threshold as static image groups. Alternatively, a difference calculation can be performed on the motion vectors of adjacent frames in an image group. If the difference value is greater than a threshold, the image group is classified as a dynamic image group; otherwise, it is classified as a static image group.

[0084] The above-mentioned implementation mode of the present application classifies the dynamic situation of the video by directly obtaining the motion vector information of the image group in the video during the encoding process. The process of processing information is faster and more direct, which can speed up the processing of information.

[0085] In some implementations, obtaining video information of a video includes:

[0086] Obtain the positional relationship between each image group and the corresponding reference image group in the video information.

[0087] Specifically, for each image group, a reference image group is selected. This reference image group can be the previous frame, an image group multiple frames ago, or a frame in the current image group. Based on the differences between the image group and the corresponding reference image group, their positional relationship is determined.

[0088] It is determined whether there is a motion relationship between each image group and the corresponding reference image group according to the positional relationship.

[0089] Specifically, based on the position relationship, it can be determined whether the position difference between the image group and the corresponding reference image group exceeds a preset threshold. If the position difference exceeds the preset threshold, it is determined that there is a motion relationship between it and the reference image group.

[0090] In the case where there is a motion relationship, motion vector information corresponding to each image group is obtained according to the corresponding positional relationship.

[0091] Specifically, feature points can be extracted from each image group. The feature points of each image group are then matched with those of the corresponding reference image group, for example using a descriptor-based matching algorithm. The feature point matching information is then used to calculate the motion transformation between the image group and the corresponding reference image group using a motion estimation algorithm. Motion transformations include, but are not limited to, translation and rotation. Based on the motion transformation parameters obtained from the motion estimation, motion vector information corresponding to each image group can be calculated, such as the magnitude and direction of the displacement vector and the rotation angle.

[0092] The above-mentioned embodiment of the present application compares the positional relationship between each different image block in the image group and the image blocks in the reference image group to determine whether there is a motion relationship, and then obtains the motion vector information corresponding to each image group. The motion vector information can be accurately obtained.

[0093] In some implementations, dividing the image groups in the video according to the motion vector information to obtain the image groups in the video includes:

[0094] The motion vector information is input into a dynamic and static video classification model, and the dynamic and static video classification model is used to classify the image groups in the video according to the degree of motion represented by the motion vector information to obtain dynamic and static classification information corresponding to the image groups.

[0095] The dynamic and static video classification model can be a machine learning algorithm or a deep learning algorithm, which is used to classify image groups in a video according to their dynamic or static characteristics.

[0096] Specifically, for each image group, corresponding feature vectors may be calculated using motion vector information, and these feature vectors may include but are not limited to the average value and variance of the motion vectors.

[0097] Then, build a suitable dynamic and static video classification model. Machine learning algorithms (such as support vector machines, decision trees, and random forests) or deep learning algorithms (such as convolutional neural networks) can be used to train the model. Using a labeled training dataset, the classification model is trained using feature vectors as input and dynamic and static classification labels as output. Cross-validation and regularization techniques can be used during training to improve model performance.

[0098] The feature vectors of the image group are input into a trained classification model, and the dynamic and static classification information of the image group is determined based on the output of the model. The specific classification method can be based on threshold, probability or multi-class classification.

[0099] The above embodiment of the present application uses a pre-trained static and dynamic video classification model to classify the image groups in the video into static and dynamic based on motion vector information, thereby obtaining classification results quickly and accurately.

[0100] In some embodiments, inputting the motion vector information into a dynamic and static video classification model, classifying the image groups in the video according to the degree of motion represented by the motion vector information using the dynamic and static video classification model, and obtaining dynamic and static classification information corresponding to the image groups includes:

[0101] The total number of motion vectors in the motion vector information is determined by a dynamic and static video classification model.

[0102] Specifically, the number of motion vectors in each image group may be accumulated to obtain the total number of motion vectors.

[0103] Calculate the average value of each motion displacement value in the motion vector information.

[0104] Specifically, for each image group, its motion vector information is extracted, including the magnitude (i.e., displacement value) and direction of the motion vector. For each image group's motion vector information, all its displacement values ​​are accumulated and then divided by the total number of displacement values ​​to obtain the average motion displacement value.

[0105] The dynamic and static classification sub-model in the dynamic and static video classification model is used to perform dynamic and static classification on the image group in the video based on the total number of the motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information to obtain the dynamic and static classification information corresponding to the image group.

[0106] Specifically, for each image group, the motion vector information is used to calculate the required features, including but not limited to the total number of motion vectors, the average displacement, and the maximum and minimum displacement values. These features can be used to describe the dynamic characteristics of the image group. The trained dynamic and static video classification model is applied to the new image group. The image group features are input into the trained classification model, which then outputs the corresponding dynamic and static classification information.

[0107] The above-described embodiment of the present application obtains the total number of motion vectors and the average value of each motion displacement value, and then classifies the video into static and dynamic based on four features: the total number of motion vectors, the average value of the displacement, the maximum displacement value in the motion vector information, and the minimum displacement value in the motion vector information. By using more comprehensive motion vector information, dynamic and static classification information corresponding to the image group can be more accurately obtained.

[0108] In some embodiments, before using the dynamic and static classification submodel in the dynamic and static video classification model to perform dynamic and static classification on the image group in the video based on the total number of motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information to obtain the dynamic and static classification information corresponding to the image group, the method further includes:

[0109] Get historical video data.

[0110] The historical video data is divided into a plurality of image groups, and labels of the plurality of image groups are obtained, where the labels are dynamic image groups or static image groups.

[0111] Specifically, the required video data is first obtained from historical video data sources and converted into image sequences. Each video image sequence is segmented into multiple image groups, each containing a certain number of consecutive image frames. Typically, the number of frames in each image group is fixed. For each segmented image group, corresponding dynamic or static annotations are obtained, which can be accomplished manually.

[0112] According to the motion vector information of each of the image groups, the total number of corresponding motion vectors, the maximum value of the displacement, the minimum value of the displacement and the average value of the displacement are obtained.

[0113] The total number of motion vectors, the maximum value of the displacement, the minimum value of the displacement and the average value of the displacement of each image group are used as input data, and the initial support vector machine is trained according to the corresponding labels of the image group to obtain a dynamic and static classification sub-model.

[0114] Specifically, the total number of motion vectors, the maximum value of displacement, the minimum value of displacement, the average value of displacement, and the corresponding annotations of each image group form a training data set. These features are used as input data, and the annotations are used as output data. Based on the training data set, a support vector machine classifier is initialized. The initialized support vector machine is trained using the training data set. During the training process, the support vector machine adjusts its model parameters based on the characteristics and annotation information of the input data to achieve better classification results. After the training is completed, a trained support vector machine model is obtained, which can classify the input data (the total number of motion vectors, the maximum value of displacement, the minimum value of displacement, and the average value of displacement) and the corresponding annotations. This trained support vector machine model can be regarded as a dynamic and static classification sub-model.

[0115] The above-mentioned embodiment of the present application obtains historical video data and annotates the dynamic image group or static image group in each image group in the historical video data, and then obtains the total number of corresponding motion vectors, the maximum displacement, the minimum displacement, and the average displacement based on the motion vector information of each image group. These are then used to train an initial support vector machine, thereby obtaining a dynamic and static classification sub-model. The initial support vector machine can be trained using historical video data to obtain an accurate dynamic and static classification sub-model.

[0116] In some embodiments, the preset dynamic image adding frequency includes adding identification information to each frame whose interval number with the reference frame is an integer multiple of a first value.

[0117] Specifically, a base frame is first selected as a reference frame. This is typically a frame in a video sequence, typically the first frame or another key frame. For each frame, the number of frames between it and the base frame is calculated to determine the interval between frames. Based on a preset dynamic image addition frequency, the frames to which identification information should be added are determined. If the interval is an integer multiple of the first value, the identification information is added to that frame.

[0118] The preset static image adding frequency includes adding identification information in each frame whose interval number with the reference frame is an integer multiple of a second value, and the second value is greater than the first value.

[0119] The specific execution process of this step is similar to the above and will not be repeated here. Specifically, the second value is greater than the first value, that is, the second value corresponding to the static image is larger, and the frequency of adding identification information is correspondingly greater, then the frequency of adding dynamic images is less than the frequency of adding static images.

[0120] The above embodiment of the present application adds identification information to frames that are integer multiples of the interval between the dynamic image group and the static image group, which can make the identification information more evenly distributed in the video.

[0121] In some embodiments, before obtaining the dynamic and static classification information of the image group in the video, the method further includes:

[0122] Video information of an original video is obtained, where the video information includes motion vector information generated during encoding of a group of images in the original video, where the motion vector information represents a degree of motion of the group of images.

[0123] In a specific implementation, motion vector information between images is extracted for each image group.

[0124] The image groups in the original video are divided according to the motion vector information to obtain initial dynamic and static classification information of the image groups in the original video.

[0125] In a specific implementation, the image groups in the original video can be divided according to a set motion vector threshold. Generally speaking, if the motion vector in a certain image group exceeds the set threshold, the image group is considered to belong to the dynamic image group, otherwise it is considered to belong to the static image group.

[0126] The initial dynamic and static classification information is added to the supplementary enhancement information frame.

[0127] In a specific implementation, each image group of the original video may be traversed, and the initial dynamic and static classification information of the image group may be embedded in the corresponding supplementary enhancement information frame.

[0128] The initial supplementary enhancement information frame is added to the original video information to obtain the video, and the initial dynamic and static classification information is the dynamic and static information.

[0129] In a specific implementation, the supplementary enhancement information frame embedded with the classification information is added to the corresponding position of the original video. In this way, the original video information processed by the supplementary enhancement information frame is merged to generate a final video containing dynamic and static classification information.

[0130] The above-mentioned embodiment of the present application adds the dynamic and static classification information to the supplementary enhancement information frame and then adds the original video information, which makes the dynamic and static classification information easier to obtain and use.

[0131] As another implementation of the present application, after capturing the video, it is encoded using a conventional encoder, such as High Efficiency Video Coding (HEVC). During the encoding process, the motion vector information obtained by motion estimation is stored. Motion estimation is the process of searching for a matching block in a reference image during encoding, using a motion vector to represent the positional relationship between the reference block and the current block. If there is no motion relationship between the reference block and the current block, the motion vector is empty. If there is a motion relationship between the reference block and the current block, a motion vector is present.

[0132] As shown in Figure 2, a motion vector diagram is shown. The grid in the figure represents the result of matching block division, and the arrow in the figure represents the motion vector. Motion vectors only exist when there are moving blocks in the figure. Therefore, by determining the motion vector information within a video frame, it can be determined whether the current frame is a static frame.

[0133] In order to perform dynamic and static classification, it is necessary to train a machine learning model. In the early training stage, the video database in the home security system is used to divide the video data into groups of pictures (Group of Pictures, GOP), and the data is manually labeled, with 1 being a dynamic image group and 0 being a static image group. Before training, features are extracted from the n image groups to form a feature vector [a, b, c, d]*n, where "a, b, c, d" represents "the total number of motion vectors, the maximum value of displacement, the minimum value of displacement, and the average value of displacement". More features can be added according to actual conditions. This embodiment takes into account that some videos have only a few current blocks with motion vectors, and some slight motion. The current block motion is very small, but it is still static to the naked eye. In fact, these can all be regarded as static videos, so features related to motion vectors are selected. Afterwards, a support vector machine with a Gaussian radial basis function as the kernel function is used for training to obtain the final model.

[0134] During video encoding, motion vector information for a complete image group is collected. The total number of motion vectors, maximum displacement, minimum displacement, and average displacement are extracted as features. The previously trained support vector machine model is then loaded to determine the status of the current image group: 1 indicates a dynamic image group, and 0 indicates a static image group. The results are then encapsulated in an SEI frame in JSON format, using the format {svmResult:0 / 1}. The SEI frame is then added to the encoded bitstream, and the video is saved.

[0135] When a user chooses to play, download, or share a video, the watermark server pulls the codestream from the cloud storage server, parses the information within the codestream's SEI frames, and determines whether the current image group is dynamic or static. If it is a dynamic image group, invisible watermark frames are added every M frames; if it is a static image group, invisible watermark frames are added every N frames. For details, see Figure 3, where the frequency of adding invisible watermark frames is higher for dynamic image groups, that is, within the same frame, dynamic image groups have more invisible watermark frames. This is because when watching security videos, users focus on dynamic videos. For example, when no one is active at night, security videos are static (similar to a picture). Most users will skip this kind of video. In security scenarios, these static videos are large and account for a high proportion. Because it is just one picture, if too many watermark frames are added, computing power will be wasted, time-consuming, and the actual application is not great. Therefore, adaptive video invisible watermark frame adding technology is selected. For dynamic event videos, more watermark frames are added. If computing power permits, all video frames can be watermarked. For static videos, fewer watermark frames can be added, or even no watermark frames can be added. You can freely choose according to the actual situation.

[0136] Due to the massive scale of security video, embedding a hidden watermark in every frame requires significant computing resources, dramatically increasing computing power costs. Therefore, this embodiment utilizes adaptive video hidden watermark frame addition technology to hierarchically process dynamic and static videos, reducing the time and computing power required to add hidden watermark frames.

[0137] When judging whether a video is dynamic or static, this embodiment directly uses the motion vector information in the encoding stage as a feature, obtains the result through support vector machine classification, and judges the dynamic or static state of the GOP without the need to extract other features, thereby reducing the amount of calculation.

[0138] This embodiment encapsulates the algorithm results into SEI frames in JSON format and adds them to the bitstream. The decoding end only needs to parse the SEI frame information to obtain the dynamic and static results of the current GOP. Usually, video processing is done at the decoding end. This embodiment places the video processing and analysis at the encoding end, reducing the computational complexity at the decoding end.

[0139] Based on the method for adding video identification information provided in the above embodiment, the present application also provides a specific implementation of a device for adding video identification information. Please refer to the following embodiment.

[0140] First, referring to FIG4 , an apparatus 400 for adding video identification information provided in an embodiment of the present application includes the following modules:

[0141] The acquisition module 401 is configured to acquire dynamic and static classification information of an image group in a video, wherein the dynamic and static classification information is obtained by classifying the image group in the video according to the degree of motion.

[0142] The classification module 402 is configured to divide the image group in the video into a dynamic image group and a static image group according to the dynamic and static classification information.

[0143] The adding module 403 is configured to add identification information to specific frames of the dynamic image group according to a preset dynamic image adding frequency; add identification information to specific frames of the static image group according to a preset static image adding frequency, and obtain a video with added identification information.

[0144] The above-described embodiment of the present application obtains dynamic and static classification information of image groups in a video and then divides the video information into dynamic image groups and static image groups based on the dynamic and static classification information. Then, based on the dynamic image and static image addition frequencies, identification information is added to specific frames of the dynamic image group and the static image group, respectively, to obtain a video with the identification information added. Because the preset dynamic image addition frequency and the preset static image addition frequency are less than the number of video frames, the number of added frames is reduced, thereby reducing the computing power consumption of adding identification information and the time required to add identification information. This can improve the efficiency of adding identification information and meet the high real-time requirements of the business.

[0145] As an implementation of the present application, the apparatus 400 for adding video identification information further includes:

[0146] an acquisition module configured to acquire video information of a video, wherein the video information includes motion vector information generated during encoding of a group of images in the video, wherein the motion vector information represents a degree of motion of the group of images;

[0147] The division module is configured to divide the image groups in the video according to the motion vector information to obtain dynamic and static classification information of the image groups in the video.

[0148] The above-mentioned implementation mode of the present application classifies the dynamic situation of the video by directly obtaining the motion vector information of the image group in the video during the encoding process. The process of processing information is faster and more direct, which can speed up the processing of information.

[0149] As an implementation of the present application, the apparatus 400 for adding video identification information further includes:

[0150] The acquisition module is configured to acquire the positional relationship between each image group and the corresponding reference image group in the video information.

[0151] The determination module is configured to determine whether there is a motion relationship between each image group and the corresponding reference image group according to the positional relationship.

[0152] The acquisition module is further configured to obtain motion vector information corresponding to each image group according to the corresponding positional relationship when there is a motion relationship.

[0153] The above-mentioned embodiment of the present application compares the positional relationship between each different image block in the image group and the image blocks in the reference image group to determine whether there is a motion relationship, and then obtains the motion vector information corresponding to each image group. The motion vector information can be accurately obtained.

[0154] As an implementation of the present application, the apparatus 400 for adding video identification information further includes:

[0155] The classification module is configured to input the motion vector information into a dynamic and static video classification model, and classify the image group in the video according to the degree of motion represented by the motion vector information through the dynamic and static video classification model to obtain dynamic and static classification information corresponding to the image group.

[0156] The above embodiment of the present application uses a pre-trained static and dynamic video classification model to classify the image groups in the video into static and dynamic based on motion vector information, thereby obtaining classification results quickly and accurately.

[0157] As an implementation of the present application, the apparatus 400 for adding video identification information further includes:

[0158] The determination module is configured to determine the total number of motion vectors in the motion vector information through a dynamic and static video classification model.

[0159] The calculation module is configured to calculate an average value of each motion displacement value in the motion vector information.

[0160] The classification module is configured to use the dynamic and static classification sub-model in the dynamic and static video classification model to perform dynamic and static classification on the image group in the video based on the total number of the motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information, so as to obtain dynamic and static classification information corresponding to the image group.

[0161] The above-described embodiment of the present application obtains the total number of motion vectors and the average value of each motion displacement value, and then classifies the video into static and dynamic based on four features: the total number of motion vectors, the average value of the displacement, the maximum displacement value in the motion vector information, and the minimum displacement value in the motion vector information. By using more comprehensive motion vector information, dynamic and static classification information corresponding to the image group can be more accurately obtained.

[0162] As an implementation of the present application, the apparatus 400 for adding video identification information further includes:

[0163] The acquisition module is configured to acquire historical video data.

[0164] The segmentation module is configured to segment the historical video data into a plurality of image groups and obtain labels of the plurality of image groups, wherein the labels are dynamic image groups or static image groups.

[0165] The calculation module is configured to obtain the total number of corresponding motion vectors, the maximum value of displacement, the minimum value of displacement and the average value of displacement according to the motion vector information of each of the image groups.

[0166] The training module is configured to take the total number of motion vectors, the maximum value of the displacement, the minimum value of the displacement and the average value of the displacement of each image group as input data, train the initial support vector machine according to the corresponding labels of the image group, and obtain a dynamic and static classification sub-model.

[0167] The above-mentioned embodiment of the present application obtains historical video data and annotates the dynamic image group or static image group in each image group in the historical video data, and then obtains the total number of corresponding motion vectors, the maximum displacement, the minimum displacement, and the average displacement based on the motion vector information of each image group. These are then used to train an initial support vector machine, thereby obtaining a dynamic and static classification sub-model. The initial support vector machine can be trained using historical video data to obtain an accurate dynamic and static classification sub-model.

[0168] As an implementation of the present application, the apparatus 400 for adding video identification information further includes:

[0169] The acquisition module is configured to acquire video information of the original video, wherein the video information includes motion vector information generated by the image group in the original video during the encoding process, and the motion vector information represents the degree of motion of the image group.

[0170] The division module is configured to divide the image groups in the original video according to the motion vector information to obtain initial dynamic and static classification information of the image groups in the original video.

[0171] The adding module is configured to add the initial dynamic and static classification information to the supplementary enhancement information frame.

[0172] The adding module is further configured to add the initial supplementary enhancement information frame to the original video information to obtain the video, and the initial dynamic and static classification information is the dynamic and static information.

[0173] The above-mentioned embodiment of the present application adds the dynamic and static classification information to the supplementary enhancement information frame and then adds the original video information, which makes the dynamic and static classification information easier to obtain and use.

[0174] The various modules in the apparatus for adding video identification information provided in the embodiment of the present application can implement the various steps in the above-mentioned method for adding video identification information and achieve the corresponding effects. For the sake of brevity, they will not be repeated here.

[0175] FIG5 shows a schematic structural diagram of the hardware for adding video identification information provided in an embodiment of the present application.

[0176] The device for adding video identification information may include a processor 501 and a memory 502 storing computer program instructions.

[0177] Specifically, the processor 501 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0178] The memory 502 may include a large capacity memory configured as data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 502 may include removable or non-removable (or fixed) media. Where appropriate, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0179] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method for adding video identification information according to any one embodiment of the present disclosure.

[0180] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any one of the methods for adding video identification information in the above embodiments.

[0181] In one example, the device for adding video identification information may further include a communication interface 503 and a bus 510. As shown in FIG5, the processor 501, the memory 502, and the communication interface 503 are connected via the bus 510 and communicate with each other.

[0182] The communication interface 503 is mainly configured to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0183] Bus 510 includes hardware, software or both, and the components of online data flow metering equipment are coupled to each other. For example, but not limitation, bus can include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 can include one or more buses. Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0184] In addition, in conjunction with the method for adding video identification information in the above embodiments, embodiments of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the methods for adding video identification information in the above embodiments is implemented.

[0185] However, it should be understood that the present application is not limited to the specific configurations and processes described above and shown in the figures. Furthermore, for the sake of brevity, a detailed description of known methods and techniques is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0186] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0187] The present application can be implemented in other specific forms without departing from its spirit and essential characteristics. For example, the algorithm described in the specific embodiment can be modified, and the system architecture does not depart from the basic spirit of the present application. Therefore, the current embodiment is considered to be exemplary and not restrictive in all aspects, and the scope of the present application is defined by the appended claims rather than the above description, and all changes that fall within the scope of the meaning and equivalents of the claims are thereby included within the scope of the present application.

[0188] Those skilled in the art should understand that the above embodiments are exemplary rather than restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, the specification and the claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other devices or steps; the indefinite article "a" does not exclude a plurality; the terms "first" and "second" are used to identify names rather than to indicate any specific order. Any figure marks in the claims should not be understood as limiting the scope of protection. The functions of multiple parts appearing in the claims can be implemented by a separate hardware or software module. The fact that certain technical features appear in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A method for adding video identification information, the method comprising: Obtaining dynamic and static classification information of an image group in a video, wherein the dynamic and static classification information is obtained by classifying the image group according to a degree of motion in the video; According to the dynamic and static classification information, the image group in the video is divided into a dynamic image group and a static image group; adding identification information to specific frames of the dynamic image group according to a preset dynamic image adding frequency; According to a preset static image adding frequency, identification information is added to specific frames of the static image group to obtain a video with the identification information added.

2. The method for adding video identification information according to claim 1, wherein: Before obtaining the dynamic and static classification information of the image group in the video, the method further includes: Acquiring video information of a video, the video information including motion vector information generated during encoding of a group of images in the video, the motion vector information representing a degree of motion of the group of images; The image groups in the video are divided according to the motion vector information to obtain dynamic and static classification information of the image groups in the video.

3. The method for adding video identification information according to claim 2, wherein: The step of obtaining video information of a video includes: Obtaining the positional relationship between each image group and the corresponding reference image group in the video information; determining whether there is a motion relationship between each image group and the corresponding reference image group based on the positional relationship; In the case where there is a motion relationship, motion vector information corresponding to each image group is obtained according to the corresponding positional relationship.

4. The method for adding video identification information according to claim 2, wherein: The dividing the image groups in the video according to the motion vector information to obtain the image groups in the video includes: The motion vector information is input into a dynamic and static video classification model, and the dynamic and static video classification model is used to classify the image groups in the video according to the degree of motion represented by the motion vector information to obtain dynamic and static classification information corresponding to the image groups.

5. The method for adding video identification information according to claim 4, wherein: Inputting the motion vector information into a dynamic and static video classification model, classifying the image groups in the video according to the degree of motion represented by the motion vector information using the dynamic and static video classification model, and obtaining dynamic and static classification information corresponding to the image groups, includes: determining the total number of motion vectors in the motion vector information by using a dynamic and static video classification model; Calculating an average value of each motion displacement value in the motion vector information; The dynamic and static classification sub-model in the dynamic and static video classification model is used to perform dynamic and static classification on the image group in the video based on the total number of the motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information to obtain the dynamic and static classification information corresponding to the image group.

6. The method for adding video identification information according to claim 5, wherein: Before performing dynamic and static classification on the image groups in the video using the dynamic and static classification submodel in the dynamic and static video classification model based on the total number of motion vectors, the average value of the displacement, the maximum value of the displacement in the motion vector information, and the minimum value of the displacement in the motion vector information to obtain the dynamic and static classification information corresponding to the image groups, the method further includes: Get historical video data; Dividing the historical video data into a plurality of image groups, and obtaining annotations of the plurality of image groups, wherein the annotations are dynamic image groups or static image groups; According to the motion vector information of each of the image groups, a total number of corresponding motion vectors, a maximum value of displacement, a minimum value of displacement, and an average value of displacement are obtained; The total number of motion vectors, the maximum value of the displacement, the minimum value of the displacement and the average value of the displacement of each image group are used as input data, and the initial support vector machine is trained according to the corresponding labels of the image group to obtain a dynamic and static classification sub-model.

7. The method for adding video identification information according to any one of claims 1 to 6, wherein: The preset dynamic image adding frequency includes adding identification information to each frame whose interval number with the reference frame is an integer multiple of the first value; The preset static image adding frequency includes adding identification information in each frame whose interval number with the reference frame is an integer multiple of a second value, and the second value is greater than the first value.

8. The method for adding video identification information according to claim 2, wherein: Before obtaining the dynamic and static classification information of the image group in the video, the method further includes: Acquiring video information of an original video, the video information including motion vector information generated during encoding of a group of images in the original video, the motion vector information representing a degree of motion of the group of images; Dividing the image groups in the original video according to the motion vector information to obtain initial dynamic and static classification information of the image groups in the original video; Adding the initial dynamic and static classification information to the supplementary enhancement information frame; The initial supplementary enhancement information frame is added to the original video information to obtain the video, and the initial dynamic and static classification information is dynamic and static information.

9. A device for adding video identification information, the device comprising: an acquisition module configured to acquire dynamic and static classification information of an image group in a video, wherein the dynamic and static classification information is obtained by classifying the image group according to a degree of motion in the video; a classification module configured to divide the image group in the video into a dynamic image group and a static image group according to the dynamic and static classification information; an adding module configured to add identification information to specific frames of the dynamic image group according to a preset dynamic image adding frequency; According to a preset static image adding frequency, identification information is added to specific frames of the static image group to obtain a video with the identification information added.

10. A device for adding video identification information, the device comprising: a processor, and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the method for adding video identification information according to any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method for adding video identification information according to any one of claims 1 to 8.

12. A computer program product, wherein when instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method for adding video identification information according to any one of claims 1 to 8.

Images