A generative video coding method

Abstract

This invention belongs to the field of video processing and is a generative video encoding and decoding method, comprising the following steps: S1: In a video frame, an inter-frame is generated based on the temporal structure and content complexity of the video frame. The inter-frame is set as a recognition frame, and key information of the recognition frame is extracted. The key information includes the temporal information of the recognition frame and the correlation information between the preceding and following frames; S2: According to the information of the recognition frame, the preceding and following frame images of the recognition frame are extracted, and after feature processing, an information code carrying the feature information of the preceding and following frame images is generated by a hash algorithm. The information code is fused with the feature-processed preceding and following frame images to obtain a recognition image; S3: The recognition image is inserted into the original video frame sequence corresponding to the temporal position of the original recognition frame, and the video frame sequence after inserting the recognition image is re-encoded to generate a new video.

Description

Technical Field

[0001] This invention belongs to the field of video processing, and in particular to a generative video encoding and decoding method. Background Technology

[0002] Existing video watermarking technologies often employ fixed frame positions or static carriers for watermark embedding, failing to design dedicated recognition frames that incorporate the temporal characteristics of video frames. This makes the watermarks susceptible to tampering or failure due to frame loss. Furthermore, video type recognition frequently requires additional calls to independent recognition modules, achieved by parsing video metadata or content features. This lack of integration with the encoding / decoding process results in recognition delays and reliance on external data support. Additionally, while some technologies attempt to combine watermarks with frames, they do not use hash codes to associate information between preceding and following frames, making it difficult to trace the video type backward from the recognized frame, leading to insufficient accuracy in type recognition.

[0003] This method primarily addresses two major technical issues: First, it solves the problem of poor watermark stability and the inability to associate video types through the lack of a recognition frame design in existing watermark settings. By generating a dedicated recognition frame and embedding an information code containing hash information of the preceding and following frames, the recognition frame functions as both a watermark carrier and a basis for type identification. Second, it addresses the problem of the disconnect between video type identification and the encoding / decoding process. By binding the recognition frame information with video type features, watermark embedding and type identification basis implantation are completed during the re-encoding process, eliminating the need for an additional recognition module, simplifying the process, and improving recognition efficiency. Summary of the Invention

[0004] This invention proposes a generative video encoding and decoding method.

[0005] A generative video encoding / decoding method includes the following steps: S1: In a video frame, an inter-frame is generated based on the temporal structure and content complexity of the video frame. The inter-frame is set as an identification frame, and key information of the identification frame is extracted. The key information includes the temporal information of the identification frame and the correlation information between the preceding and following frames. S2: Based on the information of the recognition frame, extract the previous frame image and the next frame image of the recognition frame, perform feature processing, generate an information code carrying the feature information of the previous and next frame images through a hash algorithm, and fuse the information code with the feature-processed previous and next frame images to obtain the recognition image. S3: Insert the identified image into the time sequence position of the original identified frame in the original video frame sequence, and re-encode the video frame sequence after inserting the identified image to generate a new video.

[0006] Preferably, in step S1, generating inter-frames based on the temporal structure and content complexity of video frames includes the following: obtaining the key frame interval and frame type distribution in the temporal structure of video frames, and performing content complexity analysis on the original video frame sequence. The content complexity analysis includes calculating the texture density and motion vector of each frame, wherein the texture density is calculated through the gray-level co-occurrence matrix, and the motion vector is calculated by obtaining the coordinate difference between the target block and the matching block in the previous and subsequent frames through a block matching algorithm. Adjust the number of inter-frames generated based on the content complexity analysis results: reduce the number of inter-frames generated when the texture density is greater than the preset threshold and the motion vector magnitude is greater than the preset vector value; increase the number of inter-frames generated when the texture density is less than or equal to the preset threshold and the motion vector magnitude is less than or equal to the preset vector value.

[0007] Preferably, step S1 further includes dynamically adjusting the inter-frame generation interval based on the video frame rate, including: Obtain the frame rate parameters of the original video, set the inter-frame generation interval according to the frame rate parameters, and set the generation interval to the first preset interval or above when the frame rate is greater than or equal to the preset high frame rate threshold. When the frame rate is within the preset frame rate range, the generation interval is set to the second preset interval; When the frame rate is less than or equal to the preset low frame rate threshold, the generation interval is set to the third preset interval or below; Furthermore, the generated inter-frames avoid keyframe positions in the original video frame sequence.

[0008] Preferably, the key information extracted from the recognition frame in S1 includes the following: the timestamp of the recognition frame, the index of the preceding and following frames, and the content features. The timestamp is obtained from the temporal metadata of the video stream, the index of the preceding and following frames is determined according to the position of the timestamp in the original frame sequence, and the content features include the average value of the RGB channels of the recognition frame and the edge features. The edge features are extracted by the Canny edge detection algorithm, and the high threshold and low threshold of the Canny edge detection are dynamically set according to the grayscale standard deviation of the recognition frame.

[0009] Preferably, the feature processing performed on the previous frame image and the next frame image in S2 includes the following: S21: Perform detail compensation processing on the previous frame image and the next frame image. The detail compensation processing includes blur region sharpening and brightness equalization. The blur region sharpening adopts the Laplacian sharpening algorithm, and the brightness equalization adopts the histogram equalization algorithm. S22: Calculate a first time difference and a second time difference based on the timestamps of the identification frame and the preceding and following frames. The first time difference is the difference between the timestamp of the identification frame and the timestamp of the preceding frame, and the second time difference is the difference between the timestamp of the following frame and the timestamp of the identification frame. S23: Calculate the interpolation weights of the previous frame image and the next frame image based on the first time difference and the second time difference, apply the interpolation weights to the previous and next frame images after detail compensation processing, perform weighted interpolation operation on the image pixel values, and complete the temporal correlation optimization of the previous and next frame images.

[0010] Preferably, in step S2, an information code carrying the image feature information of the preceding and following frames is generated using a hash algorithm, including the following: ST21: Extract pixel features from the temporally correlated and optimized images of the preceding and following frames, and determine the complexity of the pixel features; ST22: Select the corresponding hash algorithm based on the complexity determination result: when the number of edge pixels in the pixel feature is greater than or equal to the preset edge number threshold, it is determined to be a high complexity feature, and the pixel feature is encrypted using the 256-bit SHA-256 hash algorithm; when the number of edge pixels is less than the preset edge number threshold, it is determined to be a low complexity feature, and the pixel feature is encrypted using the 128-bit MD5 hash algorithm. ST23: Generate an information code based on the encryption operation result, and then perform reverse decoding on the generated information code to obtain the decoded feature data; ST24: Match the decoded feature data with the original pixel features and calculate the matching ratio. If the matching ratio is greater than or equal to the preset matching threshold, the information code is deemed valid; if the matching ratio is less than the preset matching threshold, the information code generation step is re-executed.

[0011] Preferably, in step S2, the information code is fused with the preceding and following frame images after feature processing, including the following: performing texture complexity analysis on the preceding and following frame images after temporal correlation optimization, and determining the texture complexity by calculating the variance of the image grayscale values: when the variance is greater than or equal to a preset variance threshold, it is determined to be a high-texture region; when the variance is less than the preset variance threshold, it is determined to be a low-texture region. The valid information code is embedded into the low texture area. The embedding process adopts a semi-transparent overlay method and ensures that the distance between the edge of the information code and the edge of the image is greater than or equal to the preset distance threshold. The size of the fusion region is determined based on the information capacity of the valid information code, wherein the information capacity is determined by the version of the information code and the error correction level. The information code is scaled according to the determined fusion area size to ensure that the information code is fully adapted to the fusion area.

[0012] Preferably, in step S3, the video frame sequence after inserting the recognition image is re-encoded, including the following: extracting the coordinates of the embedding region of the information code in the recognition image; during the re-encoding process, setting a first encoding bitrate for the frame region corresponding to the embedding region and setting a second encoding bitrate for the frame region of the non-embedded region, wherein the first encoding bitrate is a preset multiple of the base bitrate and the second encoding bitrate is the base bitrate, and the base bitrate is determined according to the bitrate parameters of the original video. The re-encoding process includes intra-frame prediction, transform coding, and entropy coding. For the frame regions corresponding to the embedded regions, I-frame coding mode is used, and independent decoding is achieved through intra-frame prediction. For the frame regions of non-embedded regions, P-frame coding mode and B-frame coding mode are used respectively according to the frame type distribution of the original video frame sequence. P-frame coding mode achieves compression through forward inter-frame prediction, while B-frame coding mode achieves compression through bidirectional inter-frame prediction. The encoding mode switching is automatically triggered by the encoding control module, and the trigger condition is the recognition result of the embedded area coordinates.

[0013] Preferably, step S3, inserting the recognized image into the original video frame sequence at the corresponding temporal position of the original recognized frame, includes the following: determining the insertion position of the recognized image in the video frame sequence based on the timestamp of the recognized frame, wherein the insertion position is the temporal coordinate of the original recognized frame in the original video frame sequence; after replacing the original recognized frame with the recognized image, aligning the temporal sequence of the video frame sequence to ensure that the deviation between the adjusted video frame sequence frame rate and the original video frame rate is less than a preset frame rate deviation threshold.

[0014] Preferably, step S3 further includes encoded video encapsulation, comprising the following: The re-encoded video stream is then encapsulated. During the encapsulation process, video metadata is added, which includes encoding standard, frame rate, resolution, and bitrate parameters. The packaged video file undergoes format compatibility testing to ensure it plays normally in mainstream players and that there is no distortion in the embedded areas of the recognized images during playback.

[0015] The present invention has the following beneficial effects: 1. By generating a dedicated recognition frame as a watermark carrier and embedding a hash information code generated from the preceding and following frame images, the watermark is deeply bound to the temporal features of the video frame, making it less susceptible to loss or tampering due to a single frame. At the same time, the feature information of the preceding and following frames carried by the information code can serve as the core basis for video type identification, realizing the integration of watermark setting and type identification basis, and providing reliable data support for subsequent video type determination.

[0016] 2. By associating the identification frame information with the video type features and embedding them into the encoding stream during the video re-encoding stage, there is no need to call an independent identification module, reducing data interaction links, shortening the type identification response time, and avoiding identification failure caused by dependence on external metadata, thereby improving the convenience and reliability of video type identification.

[0017] 3. By adaptively selecting low-texture areas to embed information code watermarks and dynamically adjusting the size of the fusion area, the watermark is prevented from obscuring key video content. While ensuring the watermark and type recognition functions, the original visual effect of the video is maintained, meeting the requirements of actual playback scenarios for picture quality. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the steps of a generative video encoding and decoding method according to the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions in the embodiments of this invention will be clearly described below in conjunction with the examples.

[0020] like Figure 1 As shown, this invention proposes a generative video encoding and decoding method, the specific method being as follows: S1, In the video frame, randomly generate an inter-frame frame, set the inter-frame frame as the recognition frame, and determine the information of the frame, including the time in which the frame is located; S2, Generate a recognition image based on the recognition frame information. The recognition image consists of the previous frame image, the next frame image, and the information code generated by the hash algorithm. The information code is then fused with the previous and next frame images to obtain the recognition image. S3, Video re-encoding: The recognized image is fused into the corresponding position in the video frame and re-encoded to generate a new video.

[0021] In one feasible embodiment, the information code includes QR codes, barcodes, RFID tags, etc.

[0022] Furthermore, step S1 also includes sub-step S1-1, which generates inter-frames based on the temporal structure and content complexity of the video frames. Specifically, this involves obtaining the keyframe intervals and frame type distribution in the temporal structure of the video frames, and simultaneously performing content complexity analysis on the original video frame sequence. Content complexity analysis includes calculating the texture density and motion vector of each frame. The texture density is calculated using the gray-level co-occurrence matrix, and the motion vector is obtained using a block matching algorithm. The calculation of the motion vector must satisfy the formula... ,in Represents the magnitude of the motion vector. , These are the x and y coordinates of the target block in the current frame, respectively. , These are the x and y coordinates of the matching block in the next frame, respectively; the number of inter-frames generated is determined based on the content complexity analysis results, when the texture density is greater than a preset threshold and When the vector value is greater than the preset value, reduce the number of inter-frame generation; when the texture density is less than or equal to the preset threshold, and If the value is less than or equal to the preset vector value, increase the number of inter-frames generated.

[0023] The S1-1 sub-step further includes the S1-1-1 sub-step, which dynamically adjusts the inter-frame generation interval based on the video frame rate. Specifically, it involves obtaining the frame rate parameters of the original video. When the frame rate is greater than or equal to 60fps, the inter-frame generation interval is set to 5 frames or more. When the frame rate is greater than 30fps but less than 60fps, the inter-frame generation interval is set to 4 frames. When the frame rate is less than or equal to 30fps, the inter-frame generation interval is set to 3 frames or less. This interval adjustment ensures that the inter-frames are evenly distributed in the video sequence, and that the generated inter-frames avoid key frame positions.

[0024] Furthermore, step S1 also includes sub-step S1-2, which is the extraction of identification frame information. Specifically, it involves collecting information from the marked identification frame. The collected information includes the timestamp of the identification frame, the indexes of the preceding and following frames, and content features. The timestamp is obtained through the temporal metadata of the video stream. The indexes of the preceding and following frames are determined based on the position of the timestamp in the original frame sequence. The content features include the RGB mean and edge features of the identification frame. The RGB mean is obtained by calculating the average value of the R, G, and B channels of all pixels in the frame. The edge features are extracted using the Canny edge detection algorithm, and the high and low thresholds of the Canny edge detection algorithm need to be dynamically set according to the grayscale standard deviation of the identification frame.

[0025] Furthermore, step S2 also includes a sub-step S2-1, which is a feature enhancement for the preceding and following frames. Specifically, based on the content features of the identified frame, detail compensation processing is performed on the extracted preceding and following frame images. The detail compensation processing includes blur region sharpening and brightness equalization. Blur region sharpening uses the Laplacian sharpening algorithm, and brightness equalization uses the histogram equalization algorithm. At the same time, the time difference is calculated based on the timestamps of the identified frame and the preceding and following frames. The time difference includes a first time difference and a second time difference. The first time difference is the difference between the timestamp of the identified frame and the timestamp of the preceding frame, and the second time difference is the difference between the timestamp of the following frame and the timestamp of the identified frame.

[0026] The S2-1 sub-step further includes the S2-1-1 sub-step, which is feature interpolation based on time difference. Specifically, it involves calculating interpolation coefficients based on the first time difference and the second time difference, and the calculation of the interpolation coefficients satisfies the formula... , ,in The interpolation weights of the previous frame image. As the interpolation weight for the next frame image, The first timing difference, The second temporal difference is used; the interpolation weights are applied to the enhanced preceding and following frame images, and weighted interpolation is performed on the image pixel values ​​to obtain the enhanced preceding and following frame images after temporal correlation optimization.

[0027] Furthermore, step S2 also includes sub-step S2-2, which is dynamic hash information code generation. Specifically, it is described as follows: extracting pixel features from the enhanced frame images after temporal correlation optimization, the pixel features include RGB mean and edge features, and determining the complexity of the pixel features. When the number of edge pixels in the edge features is greater than or equal to 500 per frame, it is determined to be a high-complexity feature, and a 256-bit SHA-256 hash algorithm is used to encrypt the pixel features to generate an information code; when the number of edge pixels is less than 500 per frame, it is determined to be a low-complexity feature, and a 128-bit MD5 hash algorithm is used to encrypt the pixel features to generate an information code.

[0028] The S2-2 sub-step further includes the S2-2-1 sub-step, which is the hash information code verification. Specifically, the generated information code is decoded in reverse to obtain the decoded feature data. The decoded feature data is then compared with the original pixel features to calculate the matching degree, which satisfies the formula... ,in For matching degree, This represents the number of data points that match the decoded features with the original features. The total number of data points for the original features; when The information code is considered valid when the percentage is greater than or equal to 95%; when... If the percentage is less than 95%, repeat sub-step S2-2 to generate the information code.

[0029] Furthermore, step S2 also includes sub-step S2-3, which is adaptive information code fusion. Specifically, it is described as follows: performing texture complexity analysis on the enhanced frame images before and after the temporal correlation optimization, determining the texture complexity by calculating the variance of the image grayscale values, and determining the high texture region when the variance is greater than or equal to 100, and the low texture region when the variance is less than 100; embedding the verified valid information code into the low texture region, the embedding process adopts a semi-transparent overlay method, the overlay transparency is set to 50%, and the distance between the edge of the information code and the edge of the image must be greater than or equal to 20 pixels.

[0030] The S2-3 sub-step further includes the S2-3-1 sub-step, which is the adjustment of the fusion region size. Specifically, the sub-step involves calculating the information capacity of the valid information code. The information capacity is determined by the version and error correction level of the information code. When the information capacity is 1kb, the fusion region size is set to 100×100 pixels; when the information capacity is 2kb, the fusion region size is set to 150×150 pixels; and when the information capacity is 3kb, the fusion region size is set to 200×200 pixels. The information code is then scaled according to the set fusion region size to ensure that the information code fully fits the fusion region.

[0031] Furthermore, step S3 also includes sub-step S3-1, which is for optimizing encoding parameters. Specifically, it involves extracting the coordinates of the embedded region of the fused recognition image, setting the encoding bitrate for the frame region corresponding to the embedded region during the video re-encoding process, with the encoding bitrate being 1.2 times the base bitrate, which is determined based on the bitrate parameters of the original video; for the frame region outside the embedded region, encoding is performed while maintaining the base bitrate, using the H.265 standard, and the encoding process includes intra-frame prediction, transform coding, and entropy coding.

[0032] The S3-1 sub-step further includes the S3-1-1 sub-step, which is an encoding mode switching. Specifically, the I-frame encoding mode is used for the frame region corresponding to the embedded region. The I-frame encoding mode achieves independent decoding through intra-frame prediction to ensure the image clarity of the embedded region. For the frame region of the non-embedded region, the P-frame or B-frame encoding mode is used according to the frame type distribution. The P-frame encoding mode achieves compression through forward inter-frame prediction, and the B-frame encoding mode achieves compression through bidirectional inter-frame prediction. The encoding mode switching is automatically triggered by the encoding control module, and the triggering condition is the recognition result of the embedded region coordinates.

[0033] Furthermore, step S1 also includes sub-step S1-3, which is the parsing of the original video frame sequence. Specifically, it is described as follows: the original video file is read using a video parsing tool, and the complete video frame sequence and timing structure are obtained. The video parsing tool supports MP4, MKV, and AVI video files. The timing structure parsing includes the distribution pattern of keyframes (I-frames), prediction frames (P-frames), and bidirectional prediction frames (B-frames). The keyframe interval is obtained by parsing the sequence parameter set in the video stream, and the frame type is determined by the type identifier in the frame header information.

[0034] Furthermore, step S1 also includes sub-step S1-4, which is the identification frame marking. Specifically, the inter-frame generated after the dynamic adjustment interval is marked as an identification frame. The marking process is achieved by adding an identification identifier to the frame header information. The identification identifier is 8-bit binary data and is unique in the video frame sequence. The marked identification frame needs to store its index position in the original frame sequence so that the images of the preceding and following frames can be extracted later.

[0035] Furthermore, step S2 also includes sub-step S2-4, which is the extraction of images from the preceding and following frames. Specifically, based on the preceding and following frame indices of the identified frame, the preceding and following frame images are located and extracted from the original video frame sequence. The extraction process is achieved through frame index mapping, and the mapping relationship is established based on the storage address of the original frame sequence. The extracted preceding and following frame images must maintain the original resolution and color space, including RGB, YUV420, and YUV422. The extraction results must be verified for integrity to ensure that the image has no missing pixels.

[0036] Furthermore, step S3 also includes sub-step S3-2, which is the insertion of the recognition image. Specifically, the insertion position of the recognition image in the video frame sequence is determined based on the timestamp of the recognition frame. The insertion position is the temporal coordinate of the original recognition frame in the frame sequence. The insertion process replaces the original recognition frame with the recognition image. After the replacement, the temporal sequence of the frame sequence needs to be aligned. The temporal alignment is achieved by adjusting the frame interval to ensure that the frame rate of the adjusted frame sequence deviates from the original video frame rate by less than 0.1fps.

[0037] Furthermore, step S3 also includes sub-step S3-3, which is the encapsulation of the encoded video. Specifically, the final video stream after the encoding mode is switched is encapsulated. The encapsulation format includes MP4 and MKV. The encapsulation process requires adding video metadata, which includes encoding standard, frame rate, resolution, and bitrate parameters. The encapsulated video file needs to pass a format compatibility test to ensure that it can be played normally in mainstream players and that there is no distortion in the embedded area of ​​the recognized image during playback.

[0038] Example 1 The generative video encoding and decoding method in this embodiment is a conventional scheme, which includes eight core steps: original video parsing, conventional random identification frame generation, basic information extraction of identification frames, conventional extraction of preceding and following frames, fixed parameter hash information code generation, fixed position image fusion, conventional video re-encoding, and standard format encapsulation. It does not introduce innovative sub-steps such as dynamic inter-frame generation and adaptive fusion, and focuses on enabling short video platforms to implant basic watermarks into user-generated content and perform preliminary identification of video types. Standardized conventional technical means are used to ensure the stability and compatibility of the process.

[0039] 1. Steps for parsing the original video The FFmpeg tool is used to parse the raw short video files. The parsing objects include, but are not limited to, MP4 and MKV formats. The parsing process is implemented by calling the libavformat and libavcodec libraries of FFmpeg. Specifically, first, the format context structure `AVFormatContext` is initialized, and the `avformat_open_input` function is called to open the raw video file and read the video stream header information. Then, the `avformat_find_stream_info` function is called to parse the detailed parameters of the video stream, including the video frame rate, resolution, bitrate, frame type distribution, and pixel format.

[0040] In the parsed frame type distribution, the I-frame interval is set to 30 frames by default. This interval is determined by reading the `pic_order_cnt_type` field in the video stream sequence parameter set. The parsed complete frame sequence is stored in the frame data storage module in timestamp order. This module uses an SQLite database, with each frame data corresponding to one record. The record fields include frame index, frame type identifier, PTS timestamp, and pixel data storage path. At the same time, the video frame rate, resolution, bit rate, and other temporal structure parameters are stored in the temporal information table. The temporal information table is associated with the frame data storage module through the frame index for subsequent recognition frame generation module to call.

[0041] 2. Standard Random Identification Frame Generation Steps The frame generation module reads frame type distribution data from the timing information table, filters out the frame indices of all P-frames and B-frames, and excludes I-frame indices to avoid affecting video encoding and compression efficiency. It uses the Mersenne Twister pseudo-random number algorithm to generate random numbers, maps these random numbers to the inter-frame candidate index range, and determines the generation position of the identified frame. The generation interval is fixed at 10 frames, meaning one inter-frame is generated every 10 inter-frame candidate frames.

[0042] The inter-frame generation method is pixel-level average interpolation. Specifically, it calls the `cv::addWeighted` function of the OpenCV library to perform weighted fusion of the previous and next frame images at the target generation location. The weight coefficients are all set to 0.5. The calculation formula is as follows: in, For the generated inter-frame coordinates Pixel value at that location, For the previous frame image Pixel value at that location, For the next frame image in The pixel value at that location. The generated inter-frame marker is designated as an identification frame. The marking is completed by adding an "Identification Frame Identifier" field to the corresponding record in the frame data storage module. For non-identification frames, the value of this field is binary "0".

[0043] 3. Steps for extracting basic information from the identification frame The information extraction module reads records marked "identification frames" from the frame data storage module and extracts the core information of the identification frames, including the PTS timestamp, preceding and following frame indices, and basic content features. The PTS timestamp is directly read from the "PTS timestamp" field of the corresponding record. The preceding and following frame indices are calculated from the frame index of the identification frame; the preceding frame index is the identification frame index minus 1, and the following frame index is the identification frame index plus 1. If the calculated index exceeds the frame sequence range, such as if the identification frame is the first or last frame, it is adjusted to the nearest valid frame index.

[0044] The basic content features include the RGB channel mean of the recognition frame. The calculation process is as follows: The `cv::mean` function from the OpenCV library is called to read the BMP pixel data of the recognition frame, and the average value is calculated for the R, G, and B channels of all pixels respectively. , , The RGB mean value, along with the PTS timestamp and the preceding and following frame indices, is stored in the identification frame information table. This table uses a MySQL database and is associated with the frame data storage module through the frame index, so that it can be called by the subsequent preceding and following frame extraction module.

[0045] 4. Standard extraction steps for previous and next frames The front and back frame extraction module reads the front and back frame indices of the target recognition frame from the recognition frame information table, locates the corresponding front and back frame records from the frame data storage module according to the index, reads the BMP files of the front and back frame images through the "pixel data storage path" in the records, and calls the `cv::imread` function of the OpenCV library to load the BMP files into an image matrix.

[0046] To ensure the integrity of the extracted image data, an integrity check is required. The check uses the CRC32 cyclic redundancy check algorithm: the pixel data of the loaded image matrix is ​​read row by row, converted into a byte stream, and the CRC32 check value is calculated. This check value is compared with the "CRC32 check code" field of the corresponding record in the frame data storage module. If they match, the image extraction is considered complete; if they do not match, the BMP file is reread, with a maximum of three rereads. If all three rereads fail, an extraction failure signal is output, triggering a re-analysis process of the original video. The extracted complete previous and next frame images are temporarily stored in a memory buffer for subsequent use by the information code generation module.

[0047] 5. Steps for generating fixed-parameter hash information codes The information code generation module reads and extracts the complete previous and next frame images from the memory cache, and uses a fixed MD5 hash algorithm to encrypt the RGB mean values ​​of the two frames. Specifically, it first processes the previous frame image... , , With the next frame image , , Concatenate into a string The format is " " The `MD5` function of the OpenSSL library is called to perform a hash operation on the string, generating a 128-bit hash value.

[0048] The generated 128-bit hash value is used as the information carried by the information code. The ZXing library is used to generate the information code image. The information code version is fixed at 3, corresponding to a 29×29 module matrix. The error correction level is fixed at H level, and the pixel size is fixed at 120×120 pixels. The pixel format of the generated information code image is set to RGB and temporarily stored in the information code cache area for subsequent use by the image fusion module.

[0049] 6. Fixed-position image fusion steps The image fusion module reads the previous and next frame images from the memory cache and the information code image from the information code cache, and performs fusion using a fixed-position embedding method. The fusion position is fixed at the lower right corner of the image, specifically within a rectangular area formed by offsetting 150 pixels to the left and 150 pixels upward from the lower right corner vertex of the image. The lower right corner coordinates are [coordinates missing]. The size of this area is adapted to the size of the information code image.

[0050] The fusion method uses alpha channel mixing, with the alpha value fixed at 0.6. The mixing calculation formula is as follows: in, For the fused image in Pixel value at that location, The alpha mixing coefficient is 0.6. For information code images in Pixel value at that location, For the previous or next frame image in The pixel values ​​at each location are used. After fusion, the resulting image is recognized. The `cv::imwrite` function is called to store the recognized image in BMP format in the recognized image storage directory, and the storage path is recorded for subsequent use by the video re-encoding module.

[0051] 7. Standard Video Re-encoding Steps The video re-encoding module reads the frame sequence of the original video from the frame data storage module. Based on the recognition frame index in the recognition frame information table, it replaces the frame data corresponding to the original recognition frames with the recognition images in the storage directory, forming an adjusted frame sequence. The adjusted frame sequence is then re-encoded using an x264 encoder with H.264 / AVC as the encoding standard. The encoding parameters are set to fixed values: a fixed bitrate of 2 Mbps, a frame rate consistent with the original video frame rate, a GOP size of 30 frames, the default 8×8 DCT transform for intra-frame prediction, and CAVLC for entropy coding.

[0052] During the encoding process, the x264 encoder context `x264_t` is first initialized, and the fields of the encoding parameter structure `x264_param_t` are set. Then, the adjusted frame sequence is converted into YUV420P pixel format according to the timestamp order, and the `x264_encoder_encode` function is called to encode and generate an H.264 bitstream. The encoded bitstream is divided into NALU segments and temporarily stored in the bitstream buffer using a circular buffer with a size of 10MB for subsequent encapsulation modules to call.

[0053] 8. Standard Format Packaging Steps The encapsulation module reads the H.264 bitstream from the bitstream buffer and uses the MP4Box tool to encapsulate the bitstream into MP4 format. During encapsulation, an MP4 file container is first created, and the `MP4Create` function is called to generate the MP4 file header. Then, a video track is created, and parameters such as the track's encoding type, resolution, and frame rate are set. The SPS and PPS parameters of the H.264 bitstream are written to the track's codec configuration box mdia / minf / stbl / stsd / avcC. Finally, the NALU-formatted H.264 bitstream is written to the video track's media data box mdia / minf / stbl / stts / stsz / stz2 / mdat in timestamp order.

[0054] During the encapsulation process, video metadata needs to be added, including a "watermark identifier" and a "video type identifier." This metadata is written to the user's data box in the MP4 file. After encapsulation, the final new video file is generated, stored in the target directory, and its file path is output to the platform management system, completing the basic watermark implantation and type identification criteria implantation for the short video.

[0055] This embodiment is adapted to the UGC content processing scenario of short video platforms. In this scenario, short videos are typically 15-60 seconds long and mostly 720p or 1080p in resolution, requiring high processing efficiency and compatibility. The original video parsing module has optimized its parsing speed for this scenario, using multi-threaded parsing to keep the parsing time of a single video within 1 second. The frame data storage module adopts a hybrid storage method combining local disk and memory, storing frequently used frame data in memory to improve reading speed. The fixed parameter settings of the encoding module refer to the conventional bitrate standards of short video platforms to ensure that the size of the generated new video file is adapted to the platform's storage and transmission requirements, while ensuring basic clarity.

[0056] In addition, to address the potential resolution inconsistency issue in UGC content, the original video parsing module has added a resolution adaptation function. When the input video resolution is lower than 720p, the OpenCV `cv::resize` function is automatically called to scale the frame sequence to 720p. A bilinear interpolation algorithm is used to ensure that the position and size are adapted during subsequent information code fusion, avoiding incomplete information code display or recognition failure due to low resolution.

[0057] Example 2 This embodiment of the generative video encoding / decoding method focuses on the most important steps. While preserving the integrity of the original process, it deepens the technical details of these steps, including four sub-steps: fine extraction of features from preceding and following frames, dynamic hash information code generation, adaptive fusion region selection, and dynamic adjustment of fusion parameters. Other steps adopt a basic adaptation design. This solution focuses on addressing the need for watermarking copyright in long videos. Through high-precision image generation and fusion, it improves the watermark's anti-tampering capability and visual compatibility, while providing richer feature data for accurate video type identification.

[0058] 1. Steps for parsing the original video The FFmpeg tool is used to parse long video files. The parsing module is based on a distributed architecture design, dividing the long video into segments of 10 minutes each, with each segment assigned an independent parsing node. Each parsing node calls FFmpeg's `avformat_open_input` and `avformat_find_stream_info` functions to parse the frame sequence, frame rate, bit rate, and frame type distribution of the corresponding video segment, and extracts video metadata such as title, director, and type tags.

[0059] The parsed frame sequence generates a unique identifier by combining the segment index and the frame index, and stores it in a distributed frame data storage system. The frame data of each video segment is stored in an independent HDFS directory, and the directory path contains the video ID and the segment index. The temporal structure parameters such as frame type distribution, frame rate, and bit rate are stored in a distributed temporal information table. The row key of the table is a combination of the video ID and the segment index, and the column families include "frame_type" and "time_info", which are used by the subsequent frame generation module to call across nodes.

[0060] 2. Recognition Frame Generation and Information Extraction Steps The frame generation module employs a conventional interval generation strategy, reading the frame type distribution of each segment from the distributed time-series information table in HBase. P-frames and B-frames are selected as inter-frame candidates, while I-frames are excluded. The generated inter-frames are generated using weighted interpolation of preceding and following frames, with the weight coefficients dynamically adjusted based on the frame type: if the preceding frame is an I-frame, its weight is set to 0.6, and the following frame's weight is set to 0.4; if the following frame is an I-frame, its weight is set to 0.6, and the preceding frame's weight is set to 0.4; when both are non-I-frames, the weight is 0.5. The calculation formula is as follows: in, The weight is the weight of the previous frame. As the weight for the next frame, and .

[0061] In addition to extracting the PTS timestamp and the indexes of the preceding and following frames, the frame information extraction module also needs to extract the type association features of the recognition frames, including the metadata tags of the video segment where the recognition frame is located, the texture features and motion features of the recognition frame. This information, along with the basic information, is stored in the distributed recognition frame information table. The row key is the unique identifier of the recognition frame, and the column families include "basic_info" and "type_feature", which are then used by the subsequent recognition image generation and fusion module.

[0062] 3. Recognizing the steps of image generation and fusion 3.1 Sub-step for fine feature extraction from previous and subsequent frames The front and back frame extraction module reads the previous and next frame images from the corresponding directory in HDFS based on the front and back frame indices and segment indices in the distributed recognition frame information table, and calls the OpenCV and VLFeat libraries to extract fine features, including color features, texture features, and shape features.

[0063] The color features utilize histogram features from the HSV color space. The H channel is divided into 16 intervals, the S channel into 8 intervals, and the V channel into 8 intervals. The pixel percentage of each interval is calculated, forming a 16×8×8=1024-dimensional color histogram feature vector. Texture features were generated using HOG features. The image was divided into 16×16 pixel cells, and a gradient histogram in 8 directions was calculated for each cell. Every 4 cells formed a block, and the histograms within the block were L2 normalized to generate a 1764-dimensional HOG feature vector. The shape features are obtained using Hu moment features. Seven invariant moments of the image are calculated to form a 7-dimensional Hu moment feature vector. .

[0064] The three feature vectors are concatenated to obtain the combined feature vector of the preceding and following frames. Calculated for the previous frame and the next frame respectively and The data is stored in the feature cache area for subsequent information code generation sub-steps.

[0065] 3.2 Dynamic Hash Information Code Generation Sub-step The information code generation module reads the "type association feature" of the identification frame from the distributed identification frame information table, and determines the hash algorithm and information code parameters according to the type label: if the type label is "movie", the 256-bit SHA-256 hash algorithm is used and the information code version is set to 5; if the type label is "TV series" or "documentary", the 128-bit MD5 hash algorithm is used and the information code version is set to 3.

[0066] Taking the "movie" genre as an example, the hash operation object is the combined feature vector of the previous frame and the next frame. and First, add the two vectors element by element to get... ,right Normalization is performed to obtain the normalized vector. Call the `SHA256` function of the OpenSSL library to... The element value string, in the format " Perform a hash operation to generate a 256-bit hash value. .

[0067] The hash value is concatenated with the type label of the recognition frame to carry information as an information code. The ZXing library is used to generate the information code image. The error correction level is dynamically adjusted according to the video type: "movie" is set to level H, and "TV series" and "documentary" are set to level Q. The pixel size of the information code is calculated according to the version. Version 5 corresponds to 180×180 pixels, and version 3 corresponds to 120×120 pixels. The generated information code image is stored in the information code distributed cache for subsequent fusion sub-steps.

[0068] 3.3 Adaptive fusion region selection sub-step The fusion region selection module performs texture complexity analysis on the previous frame image and calculates texture feature parameters, including energy values, using the gray-level co-occurrence matrix. Entropy Contrast .

[0069] The calculation parameters for the gray-level co-occurrence matrix are set as follows: gray level Step length ,angle The calculation formula is: in, In the gray-level co-occurrence matrix The element value at position represents the grayscale value. and The pixels in the step size ,angle The probability of coexistence.

[0070] Determine low-texture regions based on texture parameters: Set a threshold. , , When a certain image patch , and When the image is in low-texture mode, it is identified as such. The previous frame is divided into 32×32 pixel blocks, and the texture of each block is calculated. , , All low-texture regions are filtered out, the area of ​​each region is calculated, and the region with the largest area is selected as the candidate fusion region. If there are multiple regions with the same largest area, the distance between the center coordinates of each region and the center coordinates of the image is calculated, and the region with the smallest distance is selected as the final fusion region.

[0071] If the area of ​​the selected low-texture region is smaller than the pixel size of the information code, such as an information code of 180×180 pixels, corresponding to 6 32×32 blocks, the fusion region is expanded by edge expansion: with the candidate block as the center, adjacent blocks are expanded outward until the size of the expanded region is not smaller than the size of the information code. During the expansion process, bilinear interpolation is used to supplement edge pixels to ensure the texture continuity of the expanded region.

[0072] 3.4 Sub-step for dynamic adjustment of fusion parameters The blending parameter adjustment module dynamically sets the alpha blending coefficient and blending smoothness based on the texture complexity of the blending region: Alpha coefficient With energy value Positive correlation, the calculation formula is: ,in , The value range is 0.4-0.6 to ensure that the simpler the texture, the higher the transparency of the information code, reducing visual interference; the smoothness of the fusion is achieved through Gaussian blurring, applying Gaussian blurring to the edges of the information code image, with the blur kernel size determined by the contrast. Adjustment, The smaller the value, the larger the fuzzy kernel. The formula is: ,in The size of the fuzzy kernel.

[0073] During the fusion process, the information code image is first scaled to the size of the fusion region. Lanczos interpolation is used to ensure the clarity of the scaled information code. Then, Gaussian blur is applied to the scaled information code image. Subsequently, the OpenCV function `cv::addWeighted` is called to perform alpha blending between the information code image and the previous frame image within the fusion region. The fusion formula is the same as in Example 1, but... The values ​​are dynamically calculated; finally, the edges of the fused region are feathered to eliminate the harshness of the fused edges, resulting in the final recognition image.

[0074] The identified images are named according to "Video ID - Unique Identifier of Identification Frame", stored in the HDFS identification image directory in BMP format, and the storage path is written into the "fusion_info" column family of the distributed identification frame information table for subsequent video re-encoding module to call.

[0075] 4. Video re-encoding and packaging steps The video re-encoding module uses an x265 encoder, reads the adjusted frame sequence from HDFS, and adopts the H.265 / HEVC encoding standard. The base bitrate is set according to the video resolution. To adapt to the fusion region of the recognized image, the encoding module uses fine-grained encoding for the fusion region: the fusion region is divided into 8×8 pixel macroblocks, and the intra-frame prediction mode uses AMP to improve the encoding accuracy of the fusion region; the non-fusion region uses 16×16 macroblocks to maintain the normal encoding efficiency.

[0076] The encoded H.265 stream is stored in the HDFS stream directory. The encapsulation module uses MP4Box's distributed encapsulation function to concatenate the stream segments in timestamp order and encapsulate them into MKV format. During encapsulation, copyright watermark metadata and detailed video type tags are added, and the metadata is written to the "Info" section of the MKV file. The encapsulated new video file is stored in the long video copyright library for the platform to perform copyright tracking and type classification management.

[0077] This embodiment is adapted to the scenario of copyright watermarking and accurate type identification on long video platforms. In this scenario, the videos are long and high-resolution, and the requirements for the watermark's anti-tampering capability and visual compatibility are stringent. The distributed parsing and storage architecture solves the problem of large data volume and slow processing of long videos. The parallel processing of 12 parsing nodes can control the parsing time of 120 minutes of 4K video to within 10 minutes. The adaptive fusion region selection and dynamic parameter adjustment avoid the information code from obscuring the key content of the long video. In the movie scene, the fusion region is mostly selected from low-texture areas such as the background sky and ground, with a visual interference rate of less than 5%. Dynamic hashing and high-precision encoding ensure the watermark's anti-tampering capability. Even if the video has been cropped and compressed, the type characteristics and copyright information can still be obtained by reverse decoding through the information code, meeting the copyright protection requirements of long videos.

[0078] Example 3 This embodiment of the generative video encoding and decoding method focuses on innovative potential steps, including step one, "dynamic inter-frame generation," and step three, "encoding optimization," integrating the basic recognition image generation and fusion in step two to form a complete process. This solution is adapted to the real-time watermarking and rapid type recognition requirements of live streaming platforms. By dynamically generating inter-frames based on content complexity and frame rate, it balances the real-time performance of recognition with the smoothness of live streaming. Through encoding optimization of the recognition area, it ensures the reliability of watermark recognition while controlling the live streaming bitrate, meeting the requirements of low latency (≤500ms) and high stability for live streaming.

[0079] 1. Steps for real-time parsing of raw video FFmpeg is used to parse the live stream in real time. The parsing module adopts a multi-threaded pipeline architecture with three parallel threads: a stream receiving thread, a frame parsing thread, and a data distribution thread. The stream receiving thread receives the live stream using the `avformat_open_input` function, with a timeout of 100ms to ensure rapid reconnection in case of stream interruption. The frame parsing thread calls the `av_read_frame` function to read frame data in real time, parsing the frame type, PTS timestamp, and pixel format, with the parsing speed matched to the live frame rate. The data distribution thread distributes the parsed frame data to the frame buffer in timestamp order, and simultaneously writes real-time parameters such as frame rate, bitrate, and current frame type to the timing information buffer for subsequent use by the dynamic inter-frame generation module.

[0080] To reduce parsing latency, the parsing module employs hardware acceleration to speed up the decoding process of frame data, keeping the parsing time per frame below 10ms. Simultaneously, lightweight preprocessing is performed on the parsed frame data to remove invalid frames, and a threshold for black frames is determined by detecting the RGB mean of the frames. , , And inter-frame difference, including the duplicate frame determination threshold: inter-frame SAD value <100, to ensure that the frame buffer stores valid frame data.

[0081] 2. Dynamic Inter-Frame Generation Steps 2.1 Real-time Analysis of Content Complexity Sub-step The complexity analysis module reads current and historical frame data from the frame buffer and the real-time frame rate from the timing information buffer. It calculates the content complexity of each frame in real time, including texture density and motion vectors.

[0082] Texture density is achieved using the energy value of the gray-level co-occurrence matrix. However, to adapt to real-time requirements, grayscale levels will be... Reduced to 64, step size Reduce computational load per frame The calculation time is controlled within 2ms; the motion vector adopts a simplified version of the block matching algorithm, which calculates the block matching results between the current frame and the previous frame to obtain the displacement of each block. Motion vector magnitude Take all blocks The average value is used as the mean of the motion vector of the current frame. The calculation time is controlled within 3ms.

[0083] according to and Content complexity is divided into three levels: high complexity ( and Medium complexity and Low complexity and Complexity level and corresponding , The value is written to the complexity cache in real time for use in the inter-frame interval adjustment sub-step.

[0084] 2.2 Frame Rate Adaptation Interval Adjustment Sub-step The interval adjustment module caches the real-time frame rate based on timing information. The complexity level of the complexity cache is used to dynamically set the inter-frame generation interval. The specific rules are as follows: High frame rate ( High complexity Frame, medium complexity Frames, low complexity frame; Medium frame rate ( High complexity Frame, medium complexity Frames, low complexity frame; Low frame rate High complexity Frame, medium complexity Frames, low complexity frame.

[0085] The interval adjustment module monitors frame rate changes in real time, and recalculates when the frame rate fluctuation exceeds 5fps. Value; simultaneously, to avoid overlap between inter-frames and I-frames, the interval adjustment module reads the I-frame position from the timing information buffer, if according to the current If the calculated generation position is an I-frame, then the generation position will be offset by 1 frame to ensure that all inter-frames are either P-frames or B-frames, which will not affect the live broadcast encoding compression efficiency.

[0086] 2.3 Inter-frame rapid generation and tagging sub-step The inter-frame generation module adjusts the interval according to the frame. The inter-frame generation position is located in the frame buffer, and a fast pixel interpolation algorithm is used to generate the inter-frames. For high-complexity frames, motion-compensated interpolation is used. Based on the motion vectors of the previous and next frames, displacement compensation is performed on each block before interpolation. The formula is as follows: in, The motion vector of the previous frame block. This is the motion vector for the next frame block; for frames of medium to low complexity, conventional average interpolation is used to ensure generation speed.

[0087] The generated inter-frame markers are used to identify frames. The markers are added to the header of the frame data by adding a 1-byte "identification frame identifier". 0x01 is an identification frame and 0x00 is a non-identification frame. The marked identification frames are still stored in the frame buffer. At the same time, the PTS timestamp, the index of the preceding and following frames, and the complexity level of the identification frame are written into the identification frame information buffer for subsequent recognition image generation modules to call. The total time of the entire inter-frame generation process, "complexity analysis → interval adjustment → marker generation", is controlled within 15ms, which meets the real-time requirements of live streaming.

[0088] 3. Recognizing the steps of image generation and fusion The preceding and following frame extraction module reads the preceding and following frame images from the frame buffer and extracts basic features. It generates a 128-bit hash value using the MD5 hash algorithm and generates an information code carrying the live broadcast type label. The fusion process uses a simplified adaptive region selection, which only calculates the grayscale variance of the image and selects the low-texture region with the largest area to embed the information code. The alpha coefficient is fixed at 0.5, and the fusion time is controlled within 8ms. The generated recognition image replaces the original recognition frame in the frame buffer to form an adjusted live broadcast frame sequence.

[0089] 4. Coding optimization steps 4.1 Sub-steps for identifying the region, locating the location, and configuring parameters The encoding parameter configuration module reads the fusion region coordinates of the recognition frame from the recognition frame information cache, marks the region as the recognition region, and configures the differential encoding parameters: base bitrate. Based on the live stream resolution settings, ROI region bitrate Non-ROI region bitrate Total bitrate controlled at Within a certain range, avoid fluctuations in live streaming bandwidth.

[0090] Meanwhile, the encoding mode is configured as follows: the ROI region adopts I-frame encoding mode, and intra-frame prediction adopts DC prediction to ensure that the ROI region is decoded independently and avoid information code distortion caused by inter-frame prediction; the non-ROI region selects the encoding mode according to the frame type: P-frame adopts forward inter-frame prediction, B-frame adopts bidirectional inter-frame prediction, the motion estimation search range is set to ±16 pixels, entropy coding adopts CABAC, and the encoding parameter configuration information is written to the encoding control buffer for the encoder to read in real time.

[0091] 4.2 Real-time Encoding and Stream Transmission Sub-steps The real-time mode of the x264 encoder is adopted. The encoder reads the adjusted live frame sequence from the frame buffer and the encoding parameters from the encoding control buffer, and performs real-time encoding on each frame: First, the YUV420P frame data is converted into the input format of the x264 encoder, and the basic parameters are loaded by calling the `x264_param_default_preset` function. Then, the bitrate and encoding mode of the ROI area and non-ROI area are adjusted according to the configuration of the encoding control buffer. Subsequently, the `x264_encoder_encode` function is called to encode and generate the H.264 bitstream. The encoding time is controlled within 12ms.

[0092] The encoded bitstream is transmitted to the live streaming server in real time via the RTMP protocol. Timestamp synchronization information is added during transmission to ensure the timing accuracy of image recognition during live playback. At the same time, the encoding parameters and recognition frame information are encapsulated into metadata and transmitted together via the RTMP AMF protocol for use by the live streaming receiver. The receiver can quickly locate the recognition area based on the metadata, improving the efficiency of information code decoding.

[0093] 5. Live Stream Packaging and Monitoring Steps The live stream encapsulation module encapsulates the real-time encoded H.264 stream into FLV format. The encapsulation process only adds the necessary file header and track information to avoid encapsulation delay. The encapsulated FLV stream is distributed to each viewer node through CDN to ensure live coverage and playback smoothness.

[0094] Meanwhile, a live streaming monitoring module is set up to monitor the frequency of recognition frame generation, the success rate of information code recognition, and the encoding delay in real time. The monitoring data is updated every 10 seconds. If the recognition success rate is lower than 90% or the encoding delay exceeds 20ms, parameter adjustments are triggered: increasing the ROI area bitrate or reducing the encoding complexity to ensure the reliability and real-time performance of live watermark recognition.

[0095] This embodiment is adapted to real-time watermarking and type recognition scenarios on live streaming platforms. In such scenarios, the live stream has a high frame rate and stringent latency requirements, demanding extremely high processing speed and stability. The multi-threaded pipeline parsing architecture controls the single-frame parsing time to within 10ms, the total time for the dynamic inter-frame generation process is ≤15ms, the real-time encoding time is ≤12ms, and the total latency of the entire processing chain is ≤40ms, far below the latency requirements of live streaming platforms. Differentiated encoding parameters ensure the clarity of the information code in the recognition area while controlling the total bitrate, avoiding waste of live streaming bandwidth. The real-time monitoring module can dynamically adjust parameters according to the live streaming status, improving the adaptability and stability of the solution.

[0096] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A generative video encoding and decoding method, characterized in that, Includes the following steps: S1: In a video frame, an inter-frame is generated based on the temporal structure and content complexity of the video frame. The inter-frame is set as an identification frame, and key information of the identification frame is extracted. The key information includes the temporal information of the identification frame and the correlation information between the preceding and following frames. S2: Based on the information of the recognition frame, extract the previous frame image and the next frame image of the recognition frame, perform feature processing, generate an information code carrying the feature information of the previous and next frame images through a hash algorithm, and fuse the information code with the feature-processed previous and next frame images to obtain the recognition image. S3: Insert the identified image into the time sequence position of the original identified frame in the original video frame sequence, and re-encode the video frame sequence after inserting the identified image to generate a new video.

2. The generative video encoding and decoding method according to claim 1, characterized in that, In step S1, generating inter-frames based on the temporal structure and content complexity of video frames includes the following: obtaining the key frame interval and frame type distribution in the temporal structure of video frames, and performing content complexity analysis on the original video frame sequence. The content complexity analysis includes calculating the texture density and motion vector of each frame, wherein the texture density is calculated through the gray-level co-occurrence matrix, and the motion vector is calculated by obtaining the coordinate difference between the target block and the matching block in the previous and subsequent frames through a block matching algorithm. Adjust the number of inter-frames generated based on the content complexity analysis results: reduce the number of inter-frames generated when the texture density is greater than the preset threshold and the motion vector magnitude is greater than the preset vector value; increase the number of inter-frames generated when the texture density is less than or equal to the preset threshold and the motion vector magnitude is less than or equal to the preset vector value.

3. The generative video encoding and decoding method according to claim 1, characterized in that, S1 also includes dynamically adjusting the inter-frame generation interval based on the video frame rate, including: Obtain the frame rate parameters of the original video, set the inter-frame generation interval according to the frame rate parameters, and set the generation interval to the first preset interval or above when the frame rate is greater than or equal to the preset high frame rate threshold. When the frame rate is within the preset frame rate range, the generation interval is set to the second preset interval; When the frame rate is less than or equal to the preset low frame rate threshold, the generation interval is set to the third preset interval or below; Furthermore, the generated inter-frames avoid keyframe positions in the original video frame sequence.

4. The generative video encoding and decoding method according to claim 1, characterized in that, The key information extracted from the recognition frame in S1 includes the following: the timestamp of the recognition frame, the index of the preceding and following frames, and the content features. The timestamp is obtained from the temporal metadata of the video stream, the index of the preceding and following frames is determined according to the position of the timestamp in the original frame sequence, and the content features include the average value of the RGB channels of the recognition frame and the edge features. The edge features are extracted by the Canny edge detection algorithm, and the high threshold and low threshold of the Canny edge detection are dynamically set according to the grayscale standard deviation of the recognition frame.

5. The generative video encoding and decoding method according to claim 1, characterized in that, The feature processing performed on the previous and next frame images in S2 includes the following: S21: Perform detail compensation processing on the previous frame image and the next frame image. The detail compensation processing includes blur region sharpening and brightness equalization. The blur region sharpening adopts the Laplacian sharpening algorithm, and the brightness equalization adopts the histogram equalization algorithm. S22: Calculate a first time difference and a second time difference based on the timestamps of the identification frame and the preceding and following frames. The first time difference is the difference between the timestamp of the identification frame and the timestamp of the preceding frame, and the second time difference is the difference between the timestamp of the following frame and the timestamp of the identification frame. S23: Calculate the interpolation weights of the previous frame image and the next frame image based on the first time difference and the second time difference, apply the interpolation weights to the previous and next frame images after detail compensation processing, perform weighted interpolation operation on the image pixel values, and complete the temporal correlation optimization of the previous and next frame images.

6. The generative video encoding and decoding method according to claim 1, characterized in that, In step S2, an information code carrying the image feature information of the preceding and following frames is generated using a hash algorithm, including the following: ST21: Extract pixel features from the temporally correlated and optimized images of the preceding and following frames, and determine the complexity of the pixel features; ST22: Select the corresponding hash algorithm based on the complexity determination result: when the number of edge pixels in the pixel feature is greater than or equal to the preset edge number threshold, it is determined to be a high complexity feature, and the pixel feature is encrypted using the 256-bit SHA-256 hash algorithm; when the number of edge pixels is less than the preset edge number threshold, it is determined to be a low complexity feature, and the pixel feature is encrypted using the 128-bit MD5 hash algorithm. ST23: Generate an information code based on the encryption operation result, and then perform reverse decoding on the generated information code to obtain the decoded feature data; ST24: Match the decoded feature data with the original pixel features and calculate the matching ratio. If the matching ratio is greater than or equal to the preset matching threshold, the information code is deemed valid; if the matching ratio is less than the preset matching threshold, the information code generation step is re-executed.

7. The generative video encoding and decoding method according to claim 1, characterized in that, In step S2, the information code is fused with the preceding and following frame images after feature processing, including the following: texture complexity analysis is performed on the preceding and following frame images after temporal correlation optimization, and texture complexity is determined by calculating the variance of the image grayscale values: when the variance is greater than or equal to a preset variance threshold, it is determined to be a high-texture region; when the variance is less than the preset variance threshold, it is determined to be a low-texture region. The valid information code is embedded into the low texture area. The embedding process adopts a semi-transparent overlay method and ensures that the distance between the edge of the information code and the edge of the image is greater than or equal to the preset distance threshold. The size of the fusion region is determined based on the information capacity of the valid information code, wherein the information capacity is determined by the version of the information code and the error correction level. The information code is scaled according to the determined fusion area size to ensure that the information code is fully adapted to the fusion area.

8. The generative video encoding and decoding method according to claim 1, characterized in that, The re-encoding of the video frame sequence after inserting the recognition image in S3 includes the following: extracting the coordinates of the embedding region of the information code in the recognition image; during the re-encoding process, setting a first encoding bitrate for the frame region corresponding to the embedding region and setting a second encoding bitrate for the frame region of the non-embedded region, wherein the first encoding bitrate is a preset multiple of the base bitrate and the second encoding bitrate is the base bitrate, and the base bitrate is determined according to the bitrate parameters of the original video. The re-encoding process includes intra-frame prediction, transform coding, and entropy coding. For the frame regions corresponding to the embedded regions, I-frame coding mode is used, and independent decoding is achieved through intra-frame prediction. For the frame regions of non-embedded regions, P-frame coding mode and B-frame coding mode are used respectively according to the frame type distribution of the original video frame sequence. P-frame coding mode achieves compression through forward inter-frame prediction, while B-frame coding mode achieves compression through bidirectional inter-frame prediction. The encoding mode switching is automatically triggered by the encoding control module, and the trigger condition is the recognition result of the embedded area coordinates.

9. The generative video encoding and decoding method according to claim 1, characterized in that, In step S3, inserting the recognized image into the original video frame sequence at the corresponding temporal position of the original recognized frame includes the following: determining the insertion position of the recognized image in the video frame sequence based on the timestamp of the recognized frame, wherein the insertion position is the temporal coordinate of the original recognized frame in the original video frame sequence; after replacing the original recognized frame with the recognized image, aligning the temporal sequence of the video frame sequence to ensure that the deviation between the adjusted video frame sequence frame rate and the original video frame rate is less than a preset frame rate deviation threshold.

10. A generative video encoding and decoding method according to claim 1, characterized in that, S3 also includes encoded video encapsulation, including the following: The re-encoded video stream is then encapsulated. During the encapsulation process, video metadata is added, which includes encoding standard, frame rate, resolution, and bitrate parameters. The packaged video file undergoes format compatibility testing to ensure it plays normally in mainstream players and that there is no distortion in the embedded areas of the recognized images during playback.

Images