Face video compression method and system based on key frame prediction

Abstract

The invention relates to the technical field of video signal compression, and specifically discloses a face video compression method and system based on key frame prediction. The method comprises: face video quality analysis, face video key frame extraction, face video compression, and face compressed video decompression and reconstruction. The quality of the initial face video is accurately evaluated by analyzing feature data of the initial face video. The details and changes of the extracted frame content are deeply understood by analyzing the feature data of each extracted frame, so as to provide an accurate basis for subsequent key frame extraction, ensure that the extracted key frame can accurately reflect the changes in the video, use different compression codes for each key frame and each feature frame in the face video, use compression resources more efficiently, and ensure the quality and clarity of the video at the same time. By analyzing the feature data of a second face video, it is judged whether the final quality of the compressed face video meets expectations, so as to ensure that the quality and effect of the face video compression meet the expected requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of video signal compression, and in particular to a face video compression method and system based on key frame prediction. Background Art

[0002] At present, with the popularization of remote office and online education, the demand for the use of video conferencing, distance education and live broadcast platforms is growing rapidly. In order to support high-quality remote interaction, high-definition video transmission has become the mainstream. The hourly data usage of applications such as distance education can reach more than 10GB, and the adoption of 4K video technology has further increased the pressure on data transmission. In this context, "Talking Face Video" technology, because it can truly capture facial dynamic expressions, has gradually become the key to transmitting non-verbal communication information in remote interactions.

[0003] For example, the invention patent with publication number CN114885178A discloses a bidirectional frame prediction method for face videos, including: sampling three frames of face videos in the time domain; obtaining key point heat maps corresponding to each of the three frames; combining the three frames and the key point heat maps to reconstruct an intermediate frame image. A hybrid compression method for face videos with an extremely low bit rate based on the above method is also disclosed.

[0004] For example, the invention patent with announcement number CN112019861B announces a video compression method and device based on key frame guided super-resolution, wherein the method includes: inputting the input video in the form of a frame sequence into a key frame selection network to obtain a high-resolution key frame of the input video; downsampling the input video in a frame sequence to obtain a low-resolution frame sequence of the input video; inputting the high-resolution key frame and the low-resolution frame sequence into a generator to generate a super-resolution video.

[0005] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0006] In existing video compression methods, compression processing is usually performed directly on the acquired video data. Due to the influence of video data quality, the selection of key frames may become inaccurate, and representative or important key frames cannot be accurately identified, which directly affects the prediction and compression effects of subsequent frames, and further affects the efficiency and quality of the entire video compression. Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention provides a face video compression method and system based on key frame prediction, which can effectively solve the problems involved in the above-mentioned background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: In a first aspect, the present invention provides a face video compression method based on key frame prediction, comprising:

[0009] S1. Face video quality analysis: Acquire and analyze the initial face video feature data to obtain the video quality index, and compare it with the video quality index limit value preset in the video compression database to obtain a comparison result, and finally determine whether the initial face video needs to be preprocessed according to the comparison result;

[0010] S2. Extraction of key frames of face videos: Based on the process of extracting frame rate features of face videos by a preset frame rate division method, feature data of each extracted frame is obtained and analyzed to obtain feature indexes of each extracted frame, and each extracted frame is divided into key frames and feature frames by data processing of feature indexes of each extracted frame;

[0011] S3. Face video compression: each key frame and each feature frame in the face video are compressed and encoded respectively, thereby obtaining a face compressed video;

[0012] S4. Decompression and reconstruction of compressed face video: decompress and reconstruct the compressed face video, and record the compressed face video after the decompression and reconstruction operation as the second face video, obtain feature data of the second face video, and perform comprehensive analysis on the face video and each extracted frame feature index to obtain a feature accuracy index of the second face video, and compare it with the feature accuracy threshold of the second face video preset in the video compression database to determine whether to issue an early warning prompt for the frame rate feature extraction and compression encoding of the face video based on the comparison result.

[0013] By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames. The specific processing process is as follows:

[0014] The feature index of each extracted frame is differenced with the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compared with the extracted frame feature deviation index threshold preset in the video compression database. If the feature deviation index of an extracted frame is greater than or equal to the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames. Conversely, if the feature deviation index of an extracted frame is less than the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames.

[0015] As a further method, the video quality index, the specific analysis process is:

[0016] The maximum brightness and the minimum brightness of the initial face video are processed by ratio, so as to obtain the brightness contrast of the initial face video.

[0017] Perform ratio processing on the average signal power and the average noise power of the initial face video to obtain the average signal-to-noise ratio of the initial face video;

[0018] The resolution, frame rate, transmission bit rate, brightness contrast and average signal-to-noise ratio of the initial face video are comprehensively analyzed to obtain the video quality index.

[0019] As a further method, the determination of whether the initial face video needs to be preprocessed is performed based on the comparison result. The specific determination process is:

[0020] If the video quality index is greater than or equal to the video quality index threshold value preset in the video compression database, the comparison result is recorded as the first video quality comparison result;

[0021] If the video quality index is less than the video quality index threshold value preset in the video compression database, the comparison result is recorded as the second video quality comparison result;

[0022] If the comparison result is the first video quality comparison result, the initial face video is marked as a face video, and frame rate features are directly extracted from the face video by a preset frame rate division method;

[0023] If the comparison result is the second video quality comparison result, the initial face video is preprocessed, the preprocessed initial face video is marked as a face video, and frame rate features are extracted from the face video by a preset frame rate division method;

[0024] The initial face video is preprocessed, and the specific preprocessing includes noise removal, contrast enhancement, resolution improvement, face detection and face alignment.

[0025] As a further method, the specific analysis process of each extracted frame feature index is as follows:

[0026] The brightness deviation value, face area, face tilt angle and video quality index of each extracted frame are comprehensively analyzed to obtain the feature index of each extracted frame. The specific analysis method is as follows:

[0027] ;

[0028] In the formula, is the feature index of the nth extracted frame, e is a natural constant, n is the number of each extracted frame, n=1, 2, 3, ..., m, m is the total number of extracted frames, is the brightness deviation value of the nth extracted frame, The brightness tolerance value preset for the video compression database, is the area occupied by the face of the nth extracted frame, The feature impact index corresponding to the unit value of the face area preset in the video compression database, is the face tilt angle of the nth extracted frame, The feature impact index corresponding to the unit value of the face tilt angle preset in the video compression database, is the video quality index, The characteristic impact indicator corresponding to the unit value of the video quality index preset for the video compression database.

[0029] As a further method, the feature accuracy index of the second face video is specifically analyzed as follows:

[0030] Get the video length and resolution of the face video.

[0031] The video duration of the second face video is processed with the video duration of the face video to obtain the compressed deviation duration of the two face videos.

[0032] The video resolution of the second face video is processed differentially with the video resolution of the face video to obtain a compressed resolution deviation value of the two face videos.

[0033] The compression deviation duration, compression resolution deviation value, video brightness contrast, average signal-to-noise ratio and each extracted frame feature index of the second face video are comprehensively analyzed to obtain the feature accuracy index of the second face video.

[0034] As a further method, the judgment of whether to perform an early warning prompt for the frame rate feature extraction and compression coding of the face video is as follows:

[0035] If the feature accuracy index of the second facial video is greater than or equal to the feature accuracy threshold of the second facial video preset in the video compression database, the comparison result is recorded as the first comparison result.

[0036] If the feature accuracy index of the second facial video is less than the feature accuracy threshold of the second facial video preset in the video compression database, the comparison result is recorded as the second comparison result.

[0037] If the comparison result is the second comparison result, it is necessary to give an early warning prompt for frame rate feature extraction and compression encoding of the face video.

[0038] The second aspect of the present invention provides a system for a face video compression method based on key frame prediction, including: a face video quality analysis module, which is used to obtain and analyze the feature data of the initial face video, obtain a video quality index, and compare it with the video quality index limit value preset in the video compression database. If the video quality index is greater than or equal to the video quality index limit value preset in the video compression database, the initial face video is marked as a face video, and the frame rate feature of the face video is directly extracted through a preset frame rate division method. Conversely, if the video quality index is less than the video quality index limit value preset in the video compression database, the initial face video is preprocessed, the preprocessed initial face video is marked as a face video, and the frame rate feature of the face video is extracted through a preset frame rate division method.

[0039] The face video key frame extraction module is used to obtain and analyze the feature data of each extracted frame based on the process of frame rate feature extraction of the face video through a preset frame rate division method, obtain the feature index of each extracted frame, and divide each extracted frame into key frames and feature frames by data processing of the feature index of each extracted frame.

[0040] The face video compression module is used to compress and encode each key frame and each feature frame in the face video, thereby comprehensively obtaining a face compressed video.

[0041] The face compressed video decompression and reconstruction module is used to decompress and reconstruct the face compressed video, and record the face compressed video after the decompression and reconstruction operation as the second face video, obtain the feature data of the second face video, and perform a comprehensive analysis on the face video and the feature indexes of each extracted frame to obtain the feature accuracy index of the second face video, and compare it with the feature accuracy threshold of the second face video preset in the video compression database to obtain and determine whether to issue an early warning prompt for the frame rate feature extraction and compression encoding of the face video based on the comparison result.

[0042] By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames. The specific processing process is as follows:

[0043] The feature index of each extracted frame is differenced with the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compared with the extracted frame feature deviation index threshold preset in the video compression database. If the feature deviation index of an extracted frame is greater than or equal to the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames. Conversely, if the feature deviation index of an extracted frame is less than the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames.

[0044] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0045] (1) The present invention can accurately evaluate the quality of the initial face video by acquiring and analyzing the feature data of the initial face video, obtain the video quality index, and compare it with the video quality index threshold value preset in the video compression database, so as to intelligently determine whether the initial face video needs preprocessing. For the initial face video with higher quality, the frame rate feature extraction can be directly performed; for the initial face video with lower quality, the preprocessing is first performed to improve its quality before the subsequent operation is performed;

[0046] (2) By acquiring and analyzing the feature data of each extracted frame, the present invention can deeply understand the details and changes of the extracted frame content, obtain the feature index of each extracted frame, and provide an accurate basis for subsequent key frame extraction. By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames, ensuring that the extracted key frames can accurately reflect the important information and changes in the video.

[0047] (3) The present invention compresses and encodes each key frame and each feature frame in the face video separately, which can make more efficient use of compression resources while ensuring the quality and clarity of the video. Key frames and feature frames usually contain important information and features in the video, so separate compression encoding of them can ensure that this information is better preserved and presented during the compression process;

[0048] (4) The present invention obtains feature data of the second face video, and comprehensively analyzes the face video and extracts frame feature index to comprehensively evaluate the quality and effect of video compression; obtains the feature accuracy index of the second face video, and compares it with the feature accuracy threshold of the second face video preset in the video compression database, and determines whether the final quality of the compressed face video meets the expectations based on the comparison result. If the feature accuracy index is lower than the threshold, it is necessary to adjust and optimize the frame rate feature extraction and compression encoding process. Through early warning prompts, problems in the video compression process can be discovered and handled in a timely manner, ensuring that the quality and effect of face video compression meet the expected requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention is further described with the aid of the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other accompanying drawings based on the following accompanying drawings without creative work.

[0050] Figure 1 It is a schematic diagram of the method steps of the present invention;

[0051] Figure 2It is a schematic diagram of system module connection of the present invention. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] Reference Figure 1 As shown, the first aspect of the present invention provides a face video compression method based on key frame prediction, including: S1. Face video quality analysis: acquiring and analyzing the initial face video feature data to obtain a video quality index, and comparing it with the video quality index limit value preset in the video compression database to obtain a comparison result, and finally determining whether the initial face video needs to be preprocessed based on the comparison result.

[0054] It needs to be explained that the above-mentioned frame rate feature extraction of the face video through the preset frame rate division method is to divide the frames from the video according to certain rules, so as to further analyze the facial features or perform other video processing tasks. In this embodiment, the video is evenly divided into several segments according to time, and each segment contains the same number of frames. For example, if the total number of video frames is 1000 frames, and the preset division is 10 segments, then each segment contains 100 frames. The specific operation process is as follows: first, the video needs to be decoded and converted into a series of image frames, which can be achieved through a video processing library (such as FFmpeg, OpenCV, etc.). According to the preset frame rate division method, the required frames are extracted from the decoded image frames, and the extracted frames are further processed, such as face detection, feature point extraction, etc., to obtain the required video feature data.

[0055] S2. Face video key frame extraction: Based on the process of frame rate feature extraction of face video through a preset frame rate division method, the feature data of each extracted frame is acquired and analyzed to obtain the feature index of each extracted frame. Then, each extracted frame is divided into key frames and feature frames through data processing of the feature index of each extracted frame.

[0056] S3. Face video compression: compress and encode each key frame and each feature frame in the face video respectively, thereby obtaining a face compressed video.

[0057] It should be explained that the above-mentioned VVC (Versatile Video Coding) encoding method is used for each key frame in the face video. The VVC encoding method can significantly reduce video data while ensuring the same video quality. It is particularly important for key frames in face videos because these frames usually contain important information and details in the video. The VVC encoding method supports multiple resolutions and frame rates, and can well adapt to the face video encoding requirements in different scenarios. Whether it is high-definition, ultra-high-definition or higher-resolution video, VVC can provide efficient compression performance; for each feature frame in the face video, a context-based feature encoding method is used, which can retain more feature information, which is crucial for tasks such as face recognition and expression analysis, because these tasks usually rely on key features in the video for accurate judgment and analysis. By utilizing the correlation between features, the context-based feature encoding method can effectively reduce redundant information, improve encoding efficiency, and help reduce storage and transmission costs while maintaining video quality and watchability.

[0058] S4. Decompression and reconstruction of compressed face video: decompress and reconstruct the compressed face video, and record the compressed face video after the decompression and reconstruction operation as the second face video, obtain feature data of the second face video, and perform comprehensive analysis on the face video and the feature indexes of each extracted frame to obtain the feature accuracy index of the second face video, and compare it with the feature accuracy threshold of the second face video preset in the video compression database to determine whether to issue an early warning prompt for the feature extraction and compression encoding of the face video based on the comparison result.

[0059] It should be explained that the above-mentioned decompression and reconstruction operation of the face compressed video is a crucial link in the processing flow of the face compressed video. Differentiated decoding modes need to be adopted for each key frame and each feature frame. For the key frames in the face compressed video, the VVC (Versatile Video Coding) decoder is used for decompression. The VVC decoder can efficiently process the compressed video data, especially those highly compressed key frames. In the process of decompressing the key frames, the VVC decoder will use the extracted metadata and entropy coding data to restore the original image information of the key frames. For the feature frames, a special feature frame decompression module is designed. The module uses the compact feature information in the decompressed key frames to decompress the compact features of the interlaced frames. This decompression method based on feature information can reduce the processing of redundant information, improve the decompression efficiency, and help to reconstruct the feature frames more accurately. After all the key frames and feature frames are decompressed, these frames are reorganized in the original video order to obtain the decompressed and reconstructed second face video. The second face video should be consistent with the original face video in visual quality and content.

[0060] By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames. The specific processing process is as follows:

[0061] The feature index of each extracted frame is differenced with the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compared with the extracted frame feature deviation index threshold preset in the video compression database. If the feature deviation index of an extracted frame is greater than or equal to the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames. Conversely, if the feature deviation index of an extracted frame is less than the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames.

[0062] Specifically, the initial face video feature data includes the resolution, frame number, transmission bit rate, maximum brightness, minimum brightness, average signal power and average noise power of the initial face video.

[0063] In this embodiment, there are certain correlations among the resolution, frame number, transmission bit rate, maximum brightness, minimum brightness, average signal power and average noise power of the initial face video, and these correlations are crucial for analyzing the quality and performance of the video. In the video, the higher the bit rate, the greater the amount of information transmitted per unit time. A higher bit rate can usually provide richer details and more accurate colors. The higher the resolution and frame number, the higher the bit rate is usually required to transmit in order to maintain the video quality. The stronger the signal power, the richer the brightness information in the video. The noise power will affect the clarity of the image. The greater the noise, the more susceptible the brightness information in the video is to interference. At the same time, the resolution and frame number directly affect the accuracy of face recognition. High resolution and high frame rate can provide more face details and dynamic information, thereby improving the accuracy of recognition. If the bit rate is too low, the detailed information in the video may be lost, resulting in poor face recognition effect.

[0064] The maximum brightness and the minimum brightness of the initial face video are processed by ratio, so as to obtain the brightness contrast of the initial face video.

[0065] Specifically, the video quality index has a specific analysis process as follows:

[0066] The maximum brightness and the minimum brightness of the initial face video are processed by ratio, so as to obtain the brightness contrast of the initial face video.

[0067] The resolution, frame rate, transmission bit rate, brightness contrast and average signal-to-noise ratio of the initial face video are comprehensively analyzed to obtain the video quality index. The specific analysis method is as follows:

[0068] ;

[0069] In the formula, is the video quality index, e is a natural constant, is the resolution of the initial face video, The quality impact index corresponding to the resolution unit value preset in the video compression database. is the number of frames of the initial face video, The quality impact index corresponding to the frame unit value preset in the video compression database. is the transmission bit rate of the initial face video, The reference transmission bit rate preset for the video compression database, is the brightness contrast of the initial face video, Reference brightness contrast preset for video compression database, is the average signal-to-noise ratio of the initial face video, The quality impact indicator corresponding to the average signal-to-noise ratio unit value preset for the video compression database.

[0070] It should be explained that the resolution of the above-mentioned initial face video refers to the frame number of the initial face video, which refers to the size of the video image and can be obtained by viewing the properties of the initial face video through video processing tools; the transmission bit rate of the initial face video refers to the number of frames played per second, which can also be obtained by viewing the properties of the initial face video through video processing tools; the reference transmission bit rate preset in the video compression database refers to the amount of data required per second during the transmission of the video. Use professional video analysis tools (such as FFmpeg multimedia processing tools) to find the encoding information of the video file, find the "bit rate" field in the encoding information, and read its value; the initial face video The brightness contrast of the video refers to the ratio of the highest brightness to the lowest brightness of the initial face video, where the highest brightness and the lowest brightness of the initial face video can be measured by video analysis tools to measure the brightness information in the video frame and calculate the highest and lowest brightness values; the reference brightness contrast preset in the video compression database refers to a reference brightness contrast value set during the video compression process to maintain the clarity and visual quality of the video image; the average signal-to-noise ratio of the initial face video refers to the ratio of the average signal power of the initial face video to the average noise power, where the average signal power and the average noise power are the signal power and noise information power values ​​in the video measured by video analysis tools.

[0071] The quality impact indicator corresponding to the resolution unit value preset in the video compression database, wherein the quality impact indicator corresponding to the resolution unit value can be extracted in the following manner, for example, by synchronously inputting the real-time resolution unit value and the video quality index into the quality impact indicator correspondence relationship stored in the video compression database, and the video compression database automatically extracts the quality impact indicator corresponding to the resolution unit value according to the stored quality impact indicator correspondence relationship, and its value range in this example is [0, 1].

[0072] The quality impact indicator corresponding to the frame unit value preset in the video compression database, wherein the quality impact indicator corresponding to the frame unit value can be extracted in the following manner, for example, by synchronously inputting the real-time frame unit value and the video quality index into the quality impact indicator correspondence relationship stored in the video compression database, and the video compression database automatically extracts the quality impact indicator corresponding to the frame unit value according to the stored quality impact indicator correspondence relationship, and its value range in this example is [0, 1].

[0073] The quality impact indicator corresponding to the average signal-to-noise ratio unit value preset in the video compression database, wherein the quality impact indicator corresponding to the average signal-to-noise ratio unit value can be extracted in the following manner, for example, the real-time average signal-to-noise ratio unit value and the video quality index are synchronously input into the quality impact indicator correspondence relationship stored in the video compression database, and the video compression database automatically extracts the quality impact indicator corresponding to the average signal-to-noise ratio unit value according to the stored quality impact indicator correspondence relationship, and its value range in this example is [0, 1].

[0074] In this embodiment, a lower resolution means less detailed information in the video and a blurred image. In the video quality index, a lower resolution will lead to a decrease in overall quality; a smaller frame rate means that the video will freeze or jump during playback, resulting in a lower video quality index; a larger or smaller transmission bit rate, that is, when the bit rate deviates greatly from the preset reference transmission bit rate, a too high bit rate may cause network congestion and transmission delay, while a too low bit rate may cause a decrease in video quality, such as blurred image, color distortion, etc. Therefore, a large deviation between the transmission bit rate and the preset reference transmission bit rate will result in a lower video quality index; a higher or lower brightness contrast, that is, when the bit rate deviates greatly from the preset reference brightness contrast, a too high brightness may cause the image to be overexposed and details to be lost; while a too low brightness may cause the image to be too dark and difficult to recognize; a lower average signal-to-noise ratio means that there is more noise and interference in the video, affecting the clarity and purity of the image; therefore, through a detailed analysis of each parameter in the video quality index, the quality level of the video can be comprehensively and objectively evaluated, providing a better video foundation for subsequent video compression processing.

[0075] Furthermore, the process of determining whether the initial face video needs to be preprocessed is as follows:

[0076] If the video quality index is greater than or equal to the video quality index threshold value preset in the video compression database, the comparison result is recorded as the first video quality comparison result.

[0077] If the video quality index is less than the video quality index threshold value preset in the video compression database, the comparison result is recorded as the second video quality comparison result.

[0078] If the comparison result is the first video quality comparison result, the initial face video is marked as a face video, and frame rate features are directly extracted from the face video through a preset frame rate division method.

[0079] If the comparison result is the second video quality comparison result, the initial face video is preprocessed, the preprocessed initial face video is marked as a face video, and frame rate features are extracted from the face video through a preset frame rate division method.

[0080] The initial face video is preprocessed, and the specific preprocessing includes noise removal, contrast enhancement, resolution improvement, face detection and face alignment.

[0081] In this embodiment, the video quality index is compared with the video quality index limit value preset in the video compression database. If the video quality index is greater than or equal to the video quality index limit value preset in the video compression database, it means that the video quality is high enough and no additional preprocessing is required. The comparison result is recorded as the first video quality comparison result. Therefore, for the first video quality comparison result, the initial face video is directly marked as a face video, and the frame rate feature extraction is performed on the face video according to the preset frame rate division method. On the contrary, if the video quality index is less than the video quality index limit value preset in the video compression database, it means that the video quality is low, which may include noise, insufficient contrast, low resolution and other problems. The comparison result is recorded as the second video quality comparison result. Therefore, for the second video quality comparison result, the initial face video needs to be preprocessed to improve its quality so that it is more suitable for subsequent face recognition and analysis. The specific preprocessing includes noise removal, contrast enhancement, resolution improvement, face detection, face alignment and video image normalization. The preprocessed initial face video is marked as a face video, and the frame rate feature extraction is performed on the face video through the preset frame rate division method.

[0082] It should be explained that the above-mentioned noise removal can use filtering technology (such as Gaussian filtering, mean filtering, etc.) to remove noise in the video, which helps to reduce random fluctuations and interference in the video and improve the clarity of the video; enhancing contrast can be achieved by adjusting the brightness and contrast parameters of the video or using methods such as histogram equalization. By adjusting the contrast of the video, the details in the video can be made clearer, thereby improving the accuracy of subsequent face recognition, because clearer image features are easier to identify and match; improving resolution can be achieved through face super-resolution technology, using face sample images to reconstruct a high-resolution face video that is most similar to the original low-resolution face image, which can effectively enhance low-quality images. It measures the resolution of face videos, restores the detailed information of facial features, enhances the detailed information of videos, and makes facial features more prominent; face detection refers to the technology of accurately finding the location of faces from images or videos. It scans images through Haar cascade classifiers to identify and locate the face area; face alignment, also known as face registration, refers to further locating the coordinates of the key points of facial features on the basis of face detection. These key points include eyebrows, eyes, nose, mouth, and facial contours. The purpose of face alignment is to accurately align the key points in the face image for subsequent face recognition and analysis. Commonly used alignment methods include ASM (Active Shape Model).

[0083] Specifically, the feature data of each extracted frame includes the brightness contrast, the area occupied by the face, and the tilt angle of the face of each extracted frame.

[0084] In this embodiment, there is a mutual influence relationship between the parameters of the brightness contrast of each extracted frame, the area occupied by the face, and the tilt angle of the face. The brightness contrast directly affects the clarity and visibility of the face. When the brightness is too high, the face area may be overexposed, making it difficult to accurately judge the area occupied by the face, while too low brightness may blur the facial features, which also affects the judgment of the area. Changes in brightness contrast will affect the perception of the tilt angle of the face. For example, under certain lighting conditions, high contrast may make one side of the face appear more prominent, thereby misleading the judgment of the tilt angle. At the same time, uneven brightness may also cause certain areas of the face to appear brighter or darker, further affecting the recognition of the tilt angle.

[0085] The brightness contrast of each extracted frame is subjected to difference processing with the reference brightness contrast preset in the video compression database to obtain the brightness deviation value of each extracted frame.

[0086] Specifically, the specific analysis process of each extracted frame feature index is as follows:

[0087] The brightness deviation value, face area, face tilt angle and video quality index of each extracted frame are comprehensively analyzed to obtain the feature index of each extracted frame. The specific analysis method is as follows:

[0088] ;

[0089] In the formula, is the feature index of the nth extracted frame, e is a natural constant, n is the number of each extracted frame, n=1, 2, 3, ..., m, m is the total number of extracted frames, is the brightness deviation value of the nth extracted frame, The brightness tolerance value preset for the video compression database, is the area occupied by the face of the nth extracted frame, The feature impact index corresponding to the unit value of the face area preset in the video compression database, is the face tilt angle of the nth extracted frame, The feature impact index corresponding to the unit value of the face tilt angle preset in the video compression database, is the video quality index, The characteristic impact indicator corresponding to the unit value of the video quality index preset for the video compression database.

[0090] It needs to be explained that the brightness deviation value of the above-mentioned extracted frame is the difference between the brightness contrast of the extracted frame and the reference brightness contrast preset in the video compression database, wherein the brightness contrast of the extracted frame refers to the brightness and darkness of the extracted frame, and is obtained by measuring the brightness information in the extracted frame through a video analysis tool, and calculating the highest and lowest brightness values, and comparing the highest brightness value with the lowest brightness value; the area occupied by the face refers to the proportion of the face in the image, which can be obtained by detecting the proportion of the number of face pixels to the number of pixels in the entire image through a video analysis tool; the face inclination angle refers to the degree of inclination of the face relative to the horizontal plane, which can be obtained by detecting the position change of the key points of the face (such as eyes, nose, mouth, etc.) and the horizontal key points of the face; the brightness allowable deviation value preset in the video compression database refers to the maximum deviation value between a brightness contrast and a reference brightness set in order to maintain the clarity and visual quality of the video image during the video compression process.

[0091] The video compression database presets the feature influence index corresponding to the unit value of the area occupied by the face, wherein the feature influence index corresponding to the unit value of the area occupied by the face can be extracted in the following manner. For example, the real-time unit value of the area occupied by the face and the extracted frame feature index are synchronously input into the feature influence index correspondence relationship stored in the video compression database. The video compression database automatically extracts the feature influence index corresponding to the unit value of the area occupied by the face based on the stored feature influence index correspondence relationship. At the same time, in this example, its value range is [0, 1].

[0092] The video compression database presets the feature influence index corresponding to the unit value of the face tilt angle, wherein the feature influence index corresponding to the unit value of the face tilt angle can be extracted in the following manner, for example, the real-time unit value of the face tilt angle and the extracted frame feature index are synchronously input into the feature influence index correspondence relationship stored in the video compression database, and the video compression database automatically extracts the feature influence index corresponding to the unit value of the face tilt angle according to the stored feature influence index correspondence relationship, and its value range in this example is [0, 1].

[0093] The video compression database presets the feature influence indicator corresponding to the unit value of the video quality index, wherein the feature influence indicator corresponding to the unit value of the video quality index can be extracted in the following manner, for example, the real-time unit value of the video quality index and the extracted frame feature index are synchronously input into the feature influence indicator correspondence relationship stored in the video compression database, and the video compression database automatically extracts the feature influence indicator corresponding to the unit value of the video quality index according to the stored feature influence indicator correspondence relationship, and its value range in this example is [0, 1].

[0094] In this embodiment, when the brightness deviation value of the extracted frame is large, or even greater than the preset allowable brightness deviation value, the image will be too bright or too dark, and the details in the extracted frame will be lost, affecting the feature recognition and analysis of the extracted frame, resulting in a decrease in the feature index of the extracted frame; similarly, a lower video quality index affects the feature extraction quality of the extracted frame; a lower face area means that the face is not presented clearly or prominently in the video frame, and a lower face inclination angle will affect the recognition effect because the face features are not prominent or deformed, resulting in a decrease in the feature index of the extracted frame; therefore, through a detailed analysis of each parameter in the feature index of the extracted frame, the state of the face in the extracted frame can be comprehensively and objectively evaluated, providing a basis for the division of key frames and feature frames.

[0095] It needs to be explained that the above-mentioned extracted frame feature index is positively correlated with the area occupied by the face and the face inclination angle. The extracted frame feature index is the basis for judging whether the extracted frame is a key frame or a feature frame. If the value of the extracted frame feature index is large, it indicates that the values ​​of the area occupied by the face and the face inclination angle are also large, indicating that the face changes in the extracted frame are large and it may be a key frame.

[0096] In this embodiment, each extracted frame is divided into key frames and feature frames by processing the data of the feature index of each extracted frame. The specific processing process is as follows:

[0097] The feature index of each extracted frame is differenced with the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compared with the extraction frame feature deviation index threshold preset in the video compression database; if the feature deviation index of an extracted frame is greater than or equal to the extraction frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames; conversely, if the feature deviation index of an extracted frame is less than the extraction frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames; in this embodiment, if the extracted frame is numbered m, the extracted frame corresponding to the extracted frame numbered m is directly recorded as the key frame. It should be explained that the extracted frame numbered m is the last extracted frame arranged.

[0098] In this embodiment, if the extracted frame is numbered 3, the feature index of the third extracted frame is subtracted from the feature index corresponding to the fourth extracted frame to obtain the third extracted frame feature deviation index, and compared with the extracted frame feature deviation index threshold preset in the video compression database. If the third extracted frame feature deviation index is greater than or equal to the extracted frame feature deviation index threshold preset in the video compression database, the third extracted frame and the fourth extracted frame corresponding to the third extracted frame feature are recorded as key frames. Otherwise, if the third extracted frame feature deviation index is less than the extracted frame feature deviation index threshold preset in the video compression database, the third extracted frame and the fourth extracted frame corresponding to the third extracted frame feature are recorded as feature frames.

[0099] Furthermore, the second face video feature data specifically includes the video length, video resolution, video brightness contrast, video average signal power and video average noise power of the second face video.

[0100] In this embodiment, there may be a mutually influencing relationship between parameters such as the video length, video resolution, video brightness contrast, video average signal power, and video average noise power of the second facial video. The longer the video length, the more image frames it contains, so the statistical characteristics of the overall signal power and noise power may be more stable. The higher the resolution, the more details in the image, which will make the change in brightness contrast more obvious. The higher the resolution, the greater the amount of information in the image, so the signal power may increase accordingly. An image with a higher brightness contrast may have a higher signal power because the difference between the bright and dark parts is more obvious.

[0101] The video signal power and the video noise power of the second face video are analyzed in ratio to obtain an average signal-to-noise ratio of the second face video.

[0102] Specifically, the feature accuracy index of the second face video is analyzed in the following process:

[0103] Get the video length and resolution of the face video.

[0104] The video duration of the second face video is processed with the video duration of the face video to obtain the compressed deviation duration of the two face videos.

[0105] The video resolution of the second face video is processed differentially with the video resolution of the face video to obtain a compressed resolution deviation value of the two face videos.

[0106] In this embodiment, video compression is a technology for reducing the size of video files for more efficient storage and transmission. However, video distortion or quality degradation may occur during the video compression process, especially when a higher compression ratio is used. The resolution, frame rate or color depth of the video is reduced to reduce the file size. These changes will cause details in the video to be lost or blurred, reducing the overall quality of the video. Decompression is the inverse process of compression, which aims to restore the original video data. However, due to the distortion introduced during the compression process, the decompressed video will not be completely restored to the original quality. At the same time, the decompression process will also be affected by the performance of the decoder, which will further affect the video quality. Therefore, by comparing the second face video with the face video, the specific impact of the compression and decompression processes on the video quality can be evaluated. By analyzing the deviation of the video duration and resolution, the degree of change of the video timeline and picture fineness by the compression algorithm can be revealed.

[0107] The compression deviation duration, compression resolution deviation value, video brightness contrast, average signal-to-noise ratio and feature index of each extracted frame of the second face video are comprehensively analyzed to obtain the feature accuracy index of the second face video. The specific analysis method is as follows:

[0108] ;

[0109] In the formula, is the feature accuracy index of the second face video, is the feature index of the nth extracted frame, n is the number of each extracted frame, n=1, 2, 3, ..., m, m is the total number of extracted frames, The accurate index corresponding to the unit value of the extracted frame feature index preset in the video compression database. is the compression deviation duration of the second face video, The compression tolerance time preset for the video compression database, is the compression resolution deviation value of the second face video, The compression resolution tolerance value preset for the video compression database, is the video brightness contrast of the second face video, Reference brightness contrast preset for video compression database, is the average signal-to-noise ratio of the second face video, A preset reference signal-to-noise ratio for the video compression database.

[0110] It should be explained that the compression deviation duration of the above-mentioned second face video refers to the difference between the video duration of the second face video and the video duration of the face video, wherein the video duration of the second face video and the video duration of the face video are both video durations obtained using the OpenCV video processing library; the compression allowable deviation duration preset by the video compression database refers to the maximum deviation between the second face video duration and the face video duration during the video compression process; the compression resolution deviation value of the second face video refers to the difference between the video resolution of the second face video and the video resolution of the face video, wherein the video resolution of the second face video and the video resolution of the face video are both obtained by viewing the video properties through the video processing tool; the compression resolution allowable deviation value preset by the video compression database refers to the maximum difference between the resolution of the compressed second face video and the resolution of the face video; the video brightness contrast of the second face video refers to the brightness of the second face video, which is measured and extracted by a video analysis tool to measure the brightness information in the second face video, and calculate the highest and lowest brightness values, and compare the highest brightness value with the lowest brightness value, thereby obtaining the second face The brightness contrast of the video; the reference brightness contrast preset in the video compression database refers to a reference brightness contrast value set in order to maintain the clarity and visual quality of the video image during the video compression process; the average signal-to-noise ratio of the second face video refers to the ratio of the average signal power to the average noise power of the second face video, wherein the average signal power and the average noise power are measured by a video analysis tool to measure the signal power and noise information power values ​​in the second face video, thereby obtaining the average signal-to-noise ratio of the second face video; the reference signal-to-noise ratio preset in the video compression database refers to the minimum signal-to-noise ratio of the compressed second face video; the accuracy index corresponding to the unit value of the extracted frame feature index preset in the video compression database, wherein the accuracy index corresponding to the unit value of the extracted frame feature index can be extracted in the following manner, for example, the real-time unit value of the extracted frame feature index and the feature accuracy index of the second face video are synchronously input into the feature accuracy index correspondence relationship stored in the video compression database, and the video compression database automatically extracts the accuracy index corresponding to the unit value of the extracted frame feature index according to the stored feature accuracy index correspondence relationship, and its value range in this example is [0, 1].

[0111] In this embodiment, a larger compression deviation duration, or even greater than the preset compression allowable deviation duration, means that the second face video is significantly inconsistent with the face video on the timeline, which will cause changes in the actions, expressions or event sequence in the video, thereby reducing the feature accuracy index of the second face video; when the compression resolution deviation value is large, or even greater than the preset compression resolution allowable deviation value, the reduction in resolution will cause the loss of detail information in the image, such as the texture, edges and other features of the face becoming blurred; when the video brightness contrast is large or small, that is, when the degree of deviation from the preset reference brightness contrast is large, it will directly affect the clarity and recognizability of the video picture. Too high brightness will cause the picture to be overexposed and details to be lost, while too low brightness may This causes the picture to be too dark and the features to be difficult to identify the content of the video; a lower average signal-to-noise ratio, even lower than the preset reference signal-to-noise ratio, means that there is more noise interference in the video, and the noise will cover up or distort the useful information in the image. Therefore, a lower average signal-to-noise ratio will lead to a significant decrease in the feature accuracy index; a smaller extracted frame feature index indicates that the features extracted from the video frame are not rich or accurate enough, resulting in a decrease in the feature accuracy index; therefore, through a detailed analysis of the parameters in the feature accuracy index of the second face video, we can gain an in-depth understanding of the specific impact of the video compression and decompression process on the video quality, and provide a basis for subsequent optimization of the compression algorithm, improvement of the video quality, and improvement of the feature extraction algorithm, thereby improving the accuracy and reliability of face video compression applications.

[0112] Specifically, the judgment of whether to issue an early warning prompt for frame rate feature extraction and compression coding of face video is as follows:

[0113] If the feature accuracy index of the second facial video is greater than or equal to the feature accuracy threshold of the second facial video preset in the video compression database, the comparison result is recorded as the first comparison result.

[0114] It needs to be explained that when the feature accuracy index of the second face video is greater than or equal to the feature accuracy threshold of the second face video preset in the video compression database, it means that the second face video obtained through compression and decompression still maintains a high accuracy, and the compression and decompression process does not cause significant loss or distortion of the facial features in the video. It is considered that the video has high quality in face recognition and processing, and is suitable for subsequent frame rate feature extraction and compression encoding, and the comparison result is recorded as the first comparison result. Therefore, when the comparison result is the first comparison result, there is no need for an early warning prompt for the frame rate feature extraction and compression encoding of the face video.

[0115] If the feature accuracy index of the second facial video is less than the feature accuracy threshold of the second facial video preset in the video compression database, the comparison result is recorded as the second comparison result.

[0116] It needs to be explained that when the feature accuracy index of the second face video is less than the feature accuracy threshold of the second face video preset in the video compression database, it means that the face video has suffered a certain degree of loss or distortion during the compression and decompression process, resulting in the video feature accuracy index being lower than the preset threshold. This may be because the compression algorithm is too aggressive, resulting in the loss of key information and the inability to fully restore the details in the portrait video. Therefore, the comparison result is recorded as the second comparison result.

[0117] If the comparison result is the second comparison result, it is necessary to give an early warning prompt for frame rate feature extraction and compression encoding of the face video.

[0118] In this embodiment, a warning prompt is given for frame rate feature extraction and compression coding of facial videos. The specific warning prompt can be displayed as a warning text in the video interface, such as "Warning: The current video frame rate is too low or the compression coding is improperly set, affecting the accuracy and fluency of the video content."

[0119] With the above complete and accurate analysis of face video compression prediction, we can use the video sequences in the VoxCeleb (Face Verification and Recognition Research) dataset for training and testing with a resolution of 256×256. The model is trained on an NVIDIA RTX4090 graphics card with the Adam (gradient descent optimization algorithm) optimizer for 100 epochs (a complete forward and backward propagation process of the training dataset through the neural network). The impact of adjusting the number of key reference frames used in the compression process (n=1 and n=2) on the compression effect is analyzed. The proposed method is compared with the VVC encoder and four generative video compression methods: FOMM (Facial Expression Reenactment Method Based on Deep Learning), FOMM2.0 (an improved version of FOMM with improved performance or efficiency), Face_vid2vid (a generative compression method specifically for facial videos) and CFTE (a generative facial expression reenactment method), focusing on evaluating the relationship between bitrate and video quality.

[0120] The above method combines traditional and modern perceptual quality evaluation indicators. Traditional indicators such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) evaluate pixel-level differences and structural information respectively, but may not be completely consistent with image quality perception. In order to solve this problem, modern evaluation indicators designed for learning methods are adopted, such as deep image structure and texture similarity (DISTS), learning perceptual image patch similarity (LPIPS) and Fréchet Inception distance (FID). It is an indicator used to evaluate the quality of images generated by generative models; deep image structure and texture similarity (DISTS), learning perceptual image patch similarity (LPIPS) and Fréchet Inception distance (FID) indicators are all based on deep network features, and the lower the score, the better the perceptual quality.

[0121] In addition, key efficiency indicators in video compression - coding bits per pixel (CodingBPP) and bit rate (kbps) are introduced to measure the trade-off between data transmission rate and video quality.

[0122] PSNR is an indicator used to measure the quality of image reconstruction. It is usually used to evaluate the difference between a compressed or denoised image and the original image. The larger the PSNR, the better the quality of the reconstructed image. The specific formula is as follows:

[0123] ;

[0124] Where R is the maximum possible pixel value of the image, usually 255 (the maximum pixel value of an 8-bit depth image).

[0125] MSE stands for Mean Squared Error, which represents the difference between the original image and the reconstructed image.

[0126] SSIM is an indicator used to evaluate the structural similarity between two images. It takes into account brightness, contrast, and structural information, and is more in line with human perception than simple pixel differences (such as MSE). The specific calculation formula is as follows:

[0127] ;

[0128] In the formula, and are two image blocks (usually local areas), They are image blocks and The mean of They are image blocks and The variance of is an image block and The covariance between is a constant, , is the dynamic range of pixel values ​​(usually 255).

[0129] CodingBPP is a common indicator for measuring image compression ratio, which indicates the number of bits required for each pixel. It is usually used in the field of image compression. The specific calculation formula is:

[0130] ;

[0131] Where TotalBits is the total number of bits required for the compressed image, ImageWidth is the width of the image, and ImageHeight is the height of the image.

[0132] Kbps is a unit for measuring data transmission rate, indicating how many bits (kilobits) are transmitted per second. It is usually used to describe audio and video streams, network transmission speeds, etc. The specific calculation formula is:

[0133] ;

[0134] Where TotalBitsTransferred is the total number of bits transferred, and TimeinSeconds is the time used for transmission.

[0135] Table 1 compares the bit rate savings of this method with the other five methods in terms of Rate-DISTS (bit rate savings), Rate-LPIPS (perceptual similarity) and Rate-FID (generated model quality) for 30 talking face videos with a resolution of 256×256 when the number of key frames is 1.

[0136] Table 2 compares the bit rate savings of this method with other five methods in terms of Rate-DISTS, Rate-LPIPS and Rate-FID for 30 talking face videos with a resolution of 256×256 when the number of key frames is 2.

[0137] Table 3 compares the quality indicators of our method and different methods at similar bit rate consumption when the number of key frames is 1.

[0138] Table 4 compares the quality indicators of our method and different methods at similar bit rate consumption when the number of key frames is 1.

[0139] Table 5 compares the objective quality of the proposed method in two reconstructed sequences (sequence 7 and sequence 22) with other methods under the same configuration.

[0140] Table 1 Average bit rate savings for 30 talking face videos at a resolution of 256×256 (n=1):

[0141]

[0142] Table 2 shows the average bit rate savings for 30 talking face videos at a resolution of 256×256 (n=2):

[0143]

[0144] Table 3 DISTS quality comparison at similar bit rate consumption (n=1):

[0145]

[0146] Table 4 DISTS quality comparison at similar bit rate consumption (n=2):

[0147]

[0148] Table 5 Objective quality of two reconstructed sequences (sequence 7 and sequence 22) under the same configuration:

[0149] ;

[0150] According to the experimental results in Tables 1 to 5, it is proved that the proposed face video compression prediction method has strong robustness and flexibility in achieving a balance between high visual quality and efficient compression, making it a strong candidate for various video compression applications.

[0151] Reference Figure 2 As shown, the second aspect of the present invention provides a system for a face video compression method based on key frame prediction, including: a face video quality analysis module, a face video key frame extraction module, a face video compression module, a face compressed video decompression and reconstruction module, and a video compression database.

[0152] The face video quality analysis module is connected to the face video key frame extraction module, the face video key frame extraction module is connected to the face video compression module, the face video compression module is connected to the face compressed video decompression and reconstruction module, and the face video quality analysis module, the face video key frame extraction module and the face compressed video decompression and reconstruction module are all connected to the video compression database.

[0153] The face video quality analysis module is used to obtain and analyze the feature data of the initial face video, obtain the video quality index, and compare it with the video quality index limit value preset in the video compression database. If the video quality index is greater than or equal to the video quality index limit value preset in the video compression database, the initial face video is marked as a face video, and the frame rate feature is directly extracted from the face video through a preset frame rate division method. Conversely, if the video quality index is less than the video quality index limit value preset in the video compression database, the initial face video is preprocessed, the preprocessed initial face video is marked as a face video, and the frame rate feature is extracted from the face video through a preset frame rate division method.

[0154] The face video key frame extraction module is used to obtain and analyze the feature data of each extracted frame based on the process of frame rate feature extraction of the face video through a preset frame rate division method, obtain the feature index of each extracted frame, and divide each extracted frame into key frames and feature frames by data processing of the feature index of each extracted frame.

[0155] The face video compression module is used to perform compression encoding operations on each key frame and each feature frame in the face video, thereby comprehensively obtaining a face compressed video.

[0156] The face compressed video decompression and reconstruction module is used to perform decompression and reconstruction operations on the face compressed video, and record the face compressed video after the decompression and reconstruction operation as the second face video, obtain feature data of the second face video, and perform comprehensive analysis on the face video and the extracted frame feature index to obtain a feature accuracy index of the second face video, which is compared with the feature accuracy threshold of the second face video preset in the video compression database, and whether to issue an early warning prompt for the frame rate feature extraction and compression encoding of the face video based on the comparison result.

[0157] By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames. The specific processing process is as follows:

[0158] Perform difference processing on the feature index of each extracted frame and the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compare it with the extraction frame feature deviation index threshold preset in the video compression database; if the feature deviation index of a certain extracted frame is greater than or equal to the extraction frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames; conversely, if the feature deviation index of a certain extracted frame is less than the extraction frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames; the video compression database is used to store a preset video quality index boundary value, a preset first The feature accuracy threshold of the two-person face video, the quality impact index corresponding to the preset resolution unit value, the quality impact index corresponding to the frame number unit value preset in the database, the preset reference transmission bit rate, the preset reference brightness contrast, the quality impact index corresponding to the preset average signal-to-noise ratio unit value, the feature impact index corresponding to the preset face area unit value, the feature impact index corresponding to the preset face tilt angle unit value, the feature impact index corresponding to the preset video quality index unit value, the accuracy index corresponding to the preset extracted frame feature index unit value, the preset compression allowable deviation time, the preset compression resolution allowable deviation value, the preset reference brightness contrast and the preset reference signal-to-noise ratio, etc.

[0159] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined in this specification, they should all fall within the protection scope of the present invention.

Claims

1. A face video compression method based on key frame prediction, characterized in that: include: S1. Face video quality analysis: Acquire and analyze the initial face video feature data to obtain the video quality index, and compare it with the video quality index limit value preset in the video compression database to obtain a comparison result, and finally determine whether the initial face video needs to be preprocessed according to the comparison result; S2. Extraction of key frames of face videos: Based on the process of extracting frame rate features of face videos by a preset frame rate division method, feature data of each extracted frame is obtained and analyzed to obtain feature indexes of each extracted frame, and each extracted frame is divided into key frames and feature frames by data processing of feature indexes of each extracted frame; S3. Face video compression: each key frame and each feature frame in the face video are compressed and encoded respectively, thereby obtaining a face compressed video; S4. Decompression and reconstruction of compressed face video: decompression and reconstruction operation is performed on the compressed face video, and the compressed face video after the decompression and reconstruction operation is recorded as the second face video, feature data of the second face video is obtained, and comprehensive analysis is performed on the face video and each extracted frame feature index to obtain the feature accuracy index of the second face video, and the feature accuracy threshold of the second face video preset in the video compression database is compared to obtain and determine whether to issue an early warning prompt for the frame rate feature extraction and compression encoding of the face video based on the comparison result; By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames. The specific processing process is as follows: The feature index of each extracted frame is differenced with the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compared with the extracted frame feature deviation index threshold preset in the video compression database. If the feature deviation index of an extracted frame is greater than or equal to the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames. Conversely, if the feature deviation index of an extracted frame is less than the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames.

2. The face video compression method based on key frame prediction according to claim 1, characterized in that: The initial face video feature data specifically includes the resolution, frame number, transmission bit rate, maximum brightness, minimum brightness, average signal power and average noise power of the initial face video; The maximum brightness and the minimum brightness of the initial face video are processed by ratio, so as to obtain the brightness contrast of the initial face video.

3. The face video compression method based on key frame prediction according to claim 2, characterized in that: The specific analysis process of the video quality index is as follows: Perform ratio processing on the average signal power and the average noise power of the initial face video to obtain the average signal-to-noise ratio of the initial face video; The resolution, frame number, transmission bit rate, brightness contrast and average signal-to-noise ratio of the initial face video are comprehensively analyzed to obtain the video quality index.

4. The face video compression method based on key frame prediction according to claim 3 is characterized in that: The process of determining whether to pre-process the initial face video is as follows: If the video quality index is greater than or equal to the video quality index threshold value preset in the video compression database, the comparison result is recorded as the first video quality comparison result; If the video quality index is less than the video quality index threshold value preset in the video compression database, the comparison result is recorded as the second video quality comparison result; If the comparison result is the first video quality comparison result, the initial face video is marked as a face video, and frame rate features are directly extracted from the face video by a preset frame rate division method; If the comparison result is the second video quality comparison result, the initial face video is preprocessed, the preprocessed initial face video is marked as a face video, and frame rate features are extracted from the face video by a preset frame rate division method; The initial face video is preprocessed, and the specific preprocessing includes noise removal, contrast enhancement, resolution improvement, face detection and face alignment.

5. The face video compression method based on key frame prediction according to claim 1, characterized in that: The feature data of each extracted frame specifically includes the brightness contrast, the area occupied by the face, and the tilt angle of the face of each extracted frame; The brightness contrast of each extracted frame is subjected to difference processing with the reference brightness contrast preset in the video compression database to obtain the brightness deviation value of each extracted frame.

6. The face video compression method based on key frame prediction according to claim 1, characterized in that: The specific analysis process of each extracted frame feature index is as follows: The brightness deviation value, face area, face tilt angle and video quality index of each extracted frame are comprehensively analyzed to obtain the feature index of each extracted frame. The specific analysis method is as follows: ； In the formula, is the feature index of the nth extracted frame, e is a natural constant, n is the number of each extracted frame, n=1, 2, 3, ..., m, m is the total number of extracted frames, is the brightness deviation value of the nth extracted frame, The brightness tolerance value preset for the video compression database, is the area occupied by the face of the nth extracted frame, The feature impact index corresponding to the unit value of the face area preset in the video compression database, is the face tilt angle of the nth extracted frame, The feature impact index corresponding to the unit value of the face tilt angle preset in the video compression database, is the video quality index, The characteristic impact indicator corresponding to the unit value of the video quality index preset for the video compression database.

7. The face video compression method based on key frame prediction according to claim 1, characterized in that: The second face video feature data specifically includes the video length, video resolution, video brightness contrast, video average signal power, and video average noise power of the second face video; The video signal power and the video noise power of the second face video are analyzed in ratio to obtain an average signal-to-noise ratio of the second face video.

8. The face video compression method based on key frame prediction according to claim 7, characterized in that: The feature accuracy index of the second face video is specifically analyzed as follows: Get the video duration and resolution of the face video; Performing difference processing on the video duration of the second face video and the video duration of the face video to obtain the compressed deviation duration of the two face videos; Performing difference processing on the video resolution of the second face video and the video resolution of the face video to obtain a compressed resolution deviation value of the two face videos; The compression deviation duration, compression resolution deviation value, video brightness contrast, average signal-to-noise ratio and each extracted frame feature index of the second face video are comprehensively analyzed to obtain the feature accuracy index of the second face video.

9. The face video compression method based on key frame prediction according to claim 1, characterized in that: The specific judgment process of whether to perform an early warning prompt for the frame rate feature extraction and compression coding of the face video is as follows: If the feature accuracy index of the second face video is greater than or equal to the feature accuracy threshold of the second face video preset in the video compression database, the comparison result is recorded as the first comparison result; If the feature accuracy index of the second face video is less than the feature accuracy threshold of the second face video preset in the video compression database, the comparison result is recorded as a second comparison result; If the comparison result is the second comparison result, it is necessary to give an early warning prompt for frame rate feature extraction and compression encoding of the face video.

10. A system using the face video compression method based on key frame prediction as claimed in any one of claims 1 to 9, characterized in that: include: A face video quality analysis module is used to obtain and analyze the feature data of the initial face video, obtain a video quality index, and compare it with the video quality index limit value preset in the video compression database. If the video quality index is greater than or equal to the video quality index limit value preset in the video compression database, the initial face video is marked as a face video, and the frame rate feature extraction is directly performed on the face video through a preset frame rate division method. On the contrary, if the video quality index is less than the video quality index limit value preset in the video compression database, the initial face video is preprocessed, the preprocessed initial face video is marked as a face video, and the frame rate feature extraction is performed on the face video through a preset frame rate division method. A face video key frame extraction module is used to obtain and analyze the feature data of each extracted frame based on the process of extracting frame rate features of the face video by a preset frame rate division method, obtain the feature index of each extracted frame, and divide each extracted frame into key frames and feature frames by processing the data of the feature index of each extracted frame; The face video compression module is used to compress and encode each key frame and each feature frame in the face video, thereby obtaining a face compressed video; The face compressed video decompression and reconstruction module is used to perform decompression and reconstruction operations on the face compressed video, and record the face compressed video after the decompression and reconstruction operation as the second face video, obtain feature data of the second face video, and perform comprehensive analysis on the face video and each extracted frame feature index to obtain a feature accuracy index of the second face video, compare it with the feature accuracy threshold of the second face video preset in the video compression database, and determine whether to issue an early warning prompt for the frame rate feature extraction and compression encoding of the face video based on the comparison result; By processing the data of the feature index of each extracted frame, each extracted frame is divided into key frames and feature frames. The specific processing process is as follows: The feature index of each extracted frame is differenced with the feature index corresponding to the adjacent subsequent extracted frame to obtain the feature deviation index of each extracted frame, and compared with the extracted frame feature deviation index threshold preset in the video compression database. If the feature deviation index of an extracted frame is greater than or equal to the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as key frames. Conversely, if the feature deviation index of an extracted frame is less than the extracted frame feature deviation index threshold preset in the video compression database, the extracted frame corresponding to the extracted frame feature and the subsequent extracted frame are recorded as feature frames.

Images