A video frame rate processing method, device, equipment and storage medium

Abstract

The application discloses a video frame rate processing method, device and equipment and a storage medium. The method comprises the following steps: acquiring original continuous video frames to be compressed, wherein the original continuous video frames comprise at least three continuous frames; performing video correlation feature extraction on the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames; and constructing target compressed video frames after video compression according to the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frames comprise compressed video frames and feature Gaussian distribution, so that redundant information of the video frames can be efficiently removed, the video file size can be significantly reduced without sacrificing the visual quality, and the compression efficiency, the bandwidth usage and the storage cost can be improved and optimized.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a video frame rate processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of multimedia technology and the internet, the ways of consuming video content have become more diversified. Users expect to be able to watch videos seamlessly on different types of devices, including traditional televisions, computers, tablets, and smartphones. However, these devices may support different frame rate standards, which leads to the need for video frame rate conversion.

[0003] In the field of digital video processing and playback, frame rate (FPS) refers to the number of still images (frames) displayed per second. While current video frame rate extraction and compression technologies are advanced, they still have some significant drawbacks. First, efficient compression often comes with a loss of visual quality, especially at high compression ratios, which can lead to blockiness, blurring, or color distortion. Second, complex algorithms increase computational complexity and processing time, posing latency challenges for real-time applications and exhibiting inconsistent performance across different types of video content, resulting in insufficient adaptability. Furthermore, inter-frame dependencies can lead to error propagation, affecting the quality of multiple frames, and dynamically adjusting the bitrate to maintain smooth playback also presents difficulties, hindering practical applications. Summary of the Invention

[0004] This invention provides a video frame rate processing method, apparatus, device, and storage medium to efficiently remove redundant information from video frames, significantly reduce video file size without sacrificing visual quality, thereby improving compression efficiency, optimizing bandwidth usage, and reducing storage costs.

[0005] According to one aspect of the present invention, a video frame rate processing method is provided. The method includes:

[0006] Obtain the original continuous video frames to be compressed, wherein the original continuous video frames include at least three consecutive frames.

[0007] Video correlation features are extracted from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include the component correlation features corresponding to adjacent video frames in the original continuous video frames.

[0008] Based on the original continuous video frames and the comprehensive correlation features, a target compressed video frame after video compression processing is constructed, wherein the target compressed video frame includes the compressed video frame corresponding to the original continuous video frames and the feature Gaussian distribution corresponding to the comprehensive correlation features.

[0009] According to another aspect of the present invention, a video frame rate processing apparatus is provided. The apparatus includes:

[0010] A continuous video frame acquisition module is used to acquire the original continuous video frames to be compressed, wherein the original continuous video frames include at least three consecutive frames.

[0011] The correlation feature determination module is used to extract video correlation features from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include the component correlation features corresponding to adjacent video frames in the original continuous video frames.

[0012] The compressed video frame determination module is used to construct a target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frame includes the compressed video frame corresponding to the original continuous video frames and the feature Gaussian distribution corresponding to the comprehensive correlation features.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video frame rate processing method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the video frame rate processing method according to any embodiment of the present invention.

[0018] The technical solution of this invention involves acquiring the original continuous video frames to be compressed. Video correlation features are extracted from the original continuous video frames to obtain comprehensive correlation features corresponding to them. Based on the original continuous video frames and the comprehensive correlation features, a target compressed video frame is constructed. This efficiently removes redundant information from the video frames, significantly reduces the video file size without significantly sacrificing visual quality, thereby improving compression efficiency, optimizing bandwidth usage, and reducing storage costs. Simultaneously, it enhances the accuracy of inter-frame prediction, accelerates encoding and decoding speeds, and provides strong support for intelligent video processing tasks, making it suitable for various application scenarios from consumer to professional levels.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a video frame rate processing method provided in Embodiment 1 of the present invention;

[0022] Figure 2 This is a flowchart of a video frame rate processing method provided in Embodiment 2 of the present invention;

[0023] Figure 3 This is a structural diagram of a video frame rate processing device according to Embodiment 3 of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the video frame rate processing method of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Example 1

[0028] Figure 1 This is a flowchart of a video frame rate processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where video is compressed and transmitted. The method can be executed by a video frame rate processing device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0029] S101. Obtain the original continuous video frames to be compressed.

[0030] Here, "original continuous video frames" can refer to the video frame data to be compressed. It should be noted that "original continuous video frames" includes at least three consecutive frames.

[0031] Specifically, at least three consecutive video frames need to be extracted from the original video stream as the processing target, i.e., the original consecutive video frames. These original consecutive video frames are preferably directly adjacent frames, depending on the video content and the requirements of the compression algorithm.

[0032] S102. Extract video correlation features from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames.

[0033] The comprehensive correlation feature can refer to the correlation features between original consecutive video frames. For example, the comprehensive correlation feature includes the component correlation features corresponding to adjacent video frames in the original consecutive video frames. It should be noted that the component correlation features can refer to the correlation features between adjacent video frames in the original consecutive video frames, such as the correlation features between the first frame and the second frame, or the correlation features between the second frame and the third frame.

[0034] Specifically, for each frame in the original continuous video, the similarity and changes between frames are analyzed, such as calculating the differences between adjacent frames (e.g., pixel-level changes) and detecting motion vectors and color distribution. This analysis yields features describing the relationships between frames, such as motion compensation parameters and texture similarity. By combining all these correlation features, one or more feature vectors that comprehensively describe the relationships between consecutive frames are formed, and these can be identified as comprehensive correlation features. Comprehensive correlation features can not only reflect simple differences between adjacent frames but also include more complex patterns, such as long-term dependencies or periodic motion.

[0035] S103. Based on the original continuous video frames and the comprehensive correlation features, construct the target compressed video frame after video compression processing.

[0036] The target compressed video frame includes the compressed video frame corresponding to the original continuous video frames and the Gaussian distribution corresponding to the comprehensive correlation features. It should be noted that the Gaussian distribution refers to modeling certain feature values ​​(such as brightness, color, edge intensity, etc.) as a probability density function following a normal distribution, which helps identify which parts can be simplified or discarded during compression without significantly affecting the overall quality.

[0037] Specifically, by combining the obtained comprehensive correlation features with the data from the original video frames, compressed video frames can be created using a pre-trained encoding strategy. The goal here is to minimize data redundancy while maintaining the highest possible visual quality. Statistical models, such as Gaussian distributions, can also be used to represent the distribution of features. A Gaussian distribution of features can help predict and encode information that frequently appears in the original video frames but can be simplified using mathematical models. In practical applications, Gaussian mixture models (GMMs) or other more complex probabilistic models may also be used to better capture different types of feature distributions.

[0038] For example, constructing the target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features includes:

[0039] Based on a pre-selection filtering method, retained video frames are determined from the original continuous video frames. An affine transformation is then performed on the comprehensive correlation features and the retained video frames to obtain output correlation features and output video frames. Finally, an affine transformation is performed on the output correlation features and the output video frames.

[0040] If the affine transformation termination condition is met, the target compressed video frame after video compression is constructed based on the output correlation characteristics and the output video frame.

[0041] The pre-selection filtering method includes at least an interval frame filtering method, which further includes an odd-numbered frame filtering method and an even-numbered frame filtering method. Preferably, the present invention determines the odd-numbered frame filtering method as the interval frame filtering method.

[0042] Specifically, the retained video frames and comprehensive correlation features obtained through pre-selection and screening are subjected to affine transformation processing to obtain output correlation features and output video frames.

[0043] For example, the output after affine transformation can be represented by the following formula:

[0044]

[0045] in, It could refer to preserving video frames. It can refer to comprehensive correlation characteristics. It could refer to output video frames. This can refer to the output correlation features, η and It can be any function and does not need to have an inverse function, thus it can be a complex convolutional network. It should be noted that in the technical solution of this embodiment, the number of affine transformation processes can be determined according to the actual situation; therefore, l can refer to the current number of affine transformation processes.

[0046] Furthermore, in the technical solution of this invention, in order to enhance the transformation capability of the model, a coupling layer structure similar to that in RealNVP is adopted, and branches are added to further optimize the preservation of video frames and the integration of correlation features, as shown below:

[0047]

[0048] in, It could refer to preserving video frames. It can refer to comprehensive correlation characteristics. It could refer to output video frames. This can refer to the output correlation feature, ρ(·). Both ρ and ψ(·) are transformation functions, and both employ lightweight dense blocks. Additionally, a sigmoid function is used after ρ(·) to prevent gradient explosion caused by the exp operation.

[0049] If the affine transformation termination condition is met, a target compressed video frame is constructed based on the output correlation features and the output video frame. The affine transformation termination condition includes at least reaching a preset number of affine transformation iterations. If the affine transformation termination condition is not met, the output correlation features and the output video frame can be used for another affine transformation to obtain the next output correlation features and the next output video frame, until the affine transformation termination condition is met.

[0050] For example, constructing the target compressed video frame after video compression based on the output correlation features and the output video frame includes: splicing the output video frames to obtain a compressed video frame; performing distribution mapping on the output correlation features to obtain a Gaussian distribution; and integrating the compressed video frame and the Gaussian distribution to obtain the target compressed video frame after video compression.

[0051] Specifically, multiple output video frames are merged into a compressed video frame through effective encoding methods, reducing redundant information and improving the compression ratio. The extracted relevance features are transformed into a simple probability distribution (such as an isotropic Gaussian distribution) to facilitate reconstruction during subsequent processing and decoding. For example, a normalized flow model (such as RealNVP, Glow, etc.) can be used to map complex relevance features to a simple target distribution (such as a standard normal distribution). This involves a series of reversible transformations to ensure accurate calculation of the log-likelihood and generation of new samples from the target distribution. Combining the compressed video frame and the information from the feature Gaussian distribution, a comprehensive target compressed video frame is generated, containing both visual content and preserving temporal dynamic information.

[0052] The above technical solution can effectively compress video frames and integrate them with the Gaussian distribution of correlation features to form a comprehensive target compressed video frame. This not only improves compression efficiency but also preserves important spatiotemporal information, providing a solid foundation for subsequent applications (such as video transmission, storage, and analysis).

[0053] The technical solution of this invention involves acquiring the original continuous video frames to be compressed. Video correlation features are extracted from the original continuous video frames to obtain comprehensive correlation features corresponding to them. Based on the original continuous video frames and the comprehensive correlation features, a target compressed video frame is constructed. This efficiently removes redundant information from the video frames, significantly reduces the video file size without significantly sacrificing visual quality, thereby improving compression efficiency, optimizing bandwidth usage, and reducing storage costs. Simultaneously, it enhances the accuracy of inter-frame prediction, accelerates encoding and decoding speeds, and provides strong support for intelligent video processing tasks, making it suitable for various application scenarios from consumer to professional levels.

[0054] Based on the above embodiments, after constructing the target compressed video frame after video compression processing, the method further includes: decomposing the target compressed video frame to obtain the compressed video and the characteristic Gaussian distribution; and using inverse transform processing to reconstruct the original continuous video frame before compression processing based on the compressed video and the characteristic Gaussian distribution.

[0055] In other words, based on the compressed video and the characteristic Gaussian distribution obtained from the decomposition, an inverse transform is performed to reconstruct the original continuous video frames before compression. The specific process is as follows:

[0056]

[0057] in, It could refer to preserving video frames. It can refer to comprehensive correlation characteristics. It could refer to output video frames. This can refer to the output correlation feature, ρ(·). Both ρ and ψ(·) are transformation functions, and both employ lightweight dense blocks. Additionally, a sigmoid function is used after ρ(·) to prevent gradient explosion caused by the exp operation.

[0058] After obtaining the comprehensive correlation features and retaining the video frames, the correlation module reconstructs the original intermediate frames from the retained video frames based on the correlation features. That is, the comprehensive correlation features are separated into forward features and backward features along the channel dimension, and then the intermediate frames are reconstructed by combining them with the feature reconstruction module.

[0059] This invention reconstructs lost intermediate frames by encoding inter-frame information using a Gaussian distribution, effectively improving computational efficiency and making it suitable for real-time processing applications. Compared to traditional frame interpolation methods, it maintains high-precision reconstruction results while reducing computational resource consumption. Compared to existing large-scale deep learning frame interpolation algorithms, this method significantly reduces the number of parameters and substantially improves processing speed, making it particularly suitable for real-time video processing scenarios.

[0060] Example 2

[0061] Figure 2 This is a flowchart of a video frame rate processing method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further refines the extraction of video correlation features from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames. For example... Figure 2 As shown, the method includes:

[0062] S201. Obtain the original continuous video frames to be compressed.

[0063] S202. Decompose the original continuous video frames to obtain multiple sets of original adjacent video frames.

[0064] Among them, the original adjacent video frames can refer to the video frames that are adjacent to each other in the original continuous video frames. For example, if the number of frames in the original continuous video frames is three, the original adjacent video frames can be the first and second frames, or the second and third frames.

[0065] Specifically, the original continuous video frames are decomposed and recombined to obtain multiple sets of original adjacent video frames.

[0066] S203. For each group of original adjacent video frames, determine the component correlation characteristics between the original adjacent video frames.

[0067] Among them, component correlation features can refer to the correlation features between two adjacent video frames in the original adjacent video frames.

[0068] Specifically, for each group of original adjacent video frames, the correlation between the original adjacent video frames is analyzed and summarized to form component correlation characteristics.

[0069] For example, determining the component correlation features between the original adjacent video frames includes:

[0070] The preceding and following video frames from the original adjacent video frames are input into the convolutional layer of the densely connected network, and the component correlation features between the original adjacent video frames are determined based on the convolutional layer.

[0071] Specifically, spatial feature images of preceding and following frames in the original adjacent video frames can be extracted using convolutional layers in DenseNet. Based on the extracted feature images, component correlation features between the preceding and following frames are calculated. Cross-correlation, Euclidean distance, or other similarity metrics can be used to quantify the differences and similarities between the two frames. Alternatively, additional convolutional or fully connected layers can be introduced to extract higher-level component correlation features from the fused feature map. By integrating the extracted component correlation features, subsequent applications such as video compression and frame interpolation can be facilitated, capturing the temporal dependencies between frames. Furthermore, the structural advantages of DenseNet can be fully utilized to improve feature representation capabilities and model performance.

[0072] S204. Perform feature fusion processing on the correlation features of each component to obtain the comprehensive correlation features corresponding to the original continuous video frame.

[0073] Specifically, by connecting the correlation features of each component along the channel dimension, a comprehensive correlation feature can be obtained.

[0074] Furthermore, the comprehensive correlation characteristics can be obtained using the following formula:

[0075] f = Concat(D b (I1,I2),D f (I2,I3));

[0076] Where Concat represents the concatenation of features along the channel dimension, D b D represents a forward feature extraction block composed of Dense Blocks. f denoted as the backward feature extraction block, f refers to the comprehensive correlation feature, and (I1,I2) and (I2,I3) refer to each group of original adjacent video frames in the original adjacent video frames.

[0077] On the other hand, during the inverse transform process, the compressed intermediate frames can be recovered from the reconstructed comprehensive correlation features. Taking three original adjacent video frames as an example, the compressed intermediate frames can be recovered using the following formula:

[0078]

[0079] Where Avg represents taking the average. This indicates a module rebuilt from the back end. This represents the forward reconstruction module, where I1 and I3 refer to the retained video frames, f, and f, respectively. I It refers to the comprehensive correlation characteristics after reconstruction.

[0080] S205. Based on the original continuous video frames and the comprehensive correlation features, construct the target compressed video frame after video compression processing.

[0081] The technical solution of this invention decomposes the original continuous video frames into multiple groups of original adjacent video frames, and extracts component correlation features for each group. Through feature fusion processing, a comprehensive correlation feature is obtained, which not only captures the temporal dependencies between frames but also provides rich contextual information for subsequent applications (such as video compression, frame interpolation, and action recognition). Furthermore, the technical solution of this invention not only improves feature representation capabilities but also fully utilizes the advantages of DenseNet networks to ensure the effectiveness of information flow and gradient propagation. In practical applications, the network configuration and feature extraction strategy can be adjusted according to the specific task requirements to achieve optimal results.

[0082] Example 3

[0083] Figure 3 This is a schematic diagram of a video frame rate processing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0084] The continuous video frame acquisition module 301 is used to acquire the original continuous video frames to be compressed, wherein the original continuous video frames include at least three consecutive frames.

[0085] The correlation feature determination module 302 is used to extract video correlation features from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include the component correlation features corresponding to adjacent video frames in the original continuous video frames.

[0086] The compressed video frame determination module 303 is used to construct a target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frame includes the compressed video frame corresponding to the original continuous video frames and the feature Gaussian distribution corresponding to the comprehensive correlation features.

[0087] The technical solution of this invention involves acquiring the original continuous video frames to be compressed. Video correlation features are extracted from the original continuous video frames to obtain comprehensive correlation features corresponding to them. Based on the original continuous video frames and the comprehensive correlation features, a target compressed video frame is constructed. This efficiently removes redundant information from the video frames, significantly reduces the video file size without significantly sacrificing visual quality, thereby improving compression efficiency, optimizing bandwidth usage, and reducing storage costs. Simultaneously, it enhances the accuracy of inter-frame prediction, accelerates encoding and decoding speeds, and provides strong support for intelligent video processing tasks, making it suitable for various application scenarios from consumer to professional levels.

[0088] Optionally, the relevance feature determination module 302 includes:

[0089] The adjacent video frame determination unit is used to decompose the original continuous video frames to obtain multiple sets of original adjacent video frames.

[0090] The first correlation feature determination unit is used to determine the component correlation features between the original adjacent video frames for each group of the original adjacent video frames.

[0091] The second correlation feature determination unit is used to perform feature fusion processing on the correlation features of each component to obtain the comprehensive correlation features corresponding to the original continuous video frame.

[0092] Optionally, the first relevance feature determination unit is specifically used for:

[0093] The preceding and following video frames from the original adjacent video frames are input into the convolutional layer of the densely connected network, and the component correlation features between the original adjacent video frames are determined based on the convolutional layer.

[0094] Optionally, the compressed video frame determination module 303 includes:

[0095] The video frame retention determination unit is used to determine the video frames to be retained from the original continuous video frames based on a pre-selection filtering method, wherein the pre-selection filtering method includes at least an interval frame filtering method, and the interval frame filtering method includes an odd-numbered frame filtering method and an even-numbered frame filtering method.

[0096] An affine transformation processing unit is used to perform affine transformation processing on the comprehensive correlation features and the retained video frames to obtain output correlation features and output video frames, and to perform affine transformation processing on the output correlation features and the output video frames.

[0097] The compressed video frame determination unit is used to construct the target compressed video frame after video compression processing based on the output correlation characteristics and the output video frame, provided that the affine transformation termination condition is met.

[0098] Optionally, the affine transformation termination condition includes at least the number of affine transformations reaching a preset number.

[0099] Optionally, the compressed video frame determination unit is specifically used for:

[0100] The output video frames are spliced ​​together to obtain compressed video frames;

[0101] The output correlation features are distributed and mapped to obtain a Gaussian distribution of the features.

[0102] The compressed video frame and the characteristic Gaussian distribution are integrated to obtain the target compressed video frame after video compression.

[0103] Optionally, the device further includes a continuous video frame restoration module.

[0104] The continuous video frame restoration module is specifically used for:

[0105] The target compressed video frame is decomposed to obtain the compressed video and the feature Gaussian distribution;

[0106] An inverse transform process is used to reconstruct the original continuous video frames before compression, based on the compressed video and the characteristic Gaussian distribution.

[0107] The video frame rate processing apparatus provided in this embodiment of the invention can execute the video frame rate processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0108] Example 4

[0109] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0110] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as video frame rate processing methods.

[0113] In some embodiments, the video frame rate processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the video frame rate processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the video frame rate processing method by any other suitable means (e.g., by means of firmware).

[0114] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0118] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0119] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0120] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0121] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video frame rate processing method, characterized in that, include: Obtain the original continuous video frames to be compressed, wherein the original continuous video frames include at least three consecutive frames. Video correlation features are extracted from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames. The comprehensive correlation features include component correlation features corresponding to adjacent video frames in the original continuous video frames; and are composed of multiple feature vectors describing the relationship between continuous frames. Based on the original continuous video frames and the comprehensive correlation features, a target compressed video frame after video compression processing is constructed, wherein the target compressed video frame includes the compressed video frame corresponding to the original continuous video frames and the feature Gaussian distribution corresponding to the comprehensive correlation features; The step of constructing the target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features includes: Based on a pre-selection filtering method, retained video frames are determined from the original continuous video frames. The pre-selection filtering method includes at least an interval frame filtering method, which includes an odd-numbered frame filtering method and an even-numbered frame filtering method. The integrated correlation features and the retained video frames are subjected to affine transformation to obtain output correlation features and output video frames, and the output correlation features and output video frames are subjected to affine transformation. If the affine transformation termination condition is met, the target compressed video frame after video compression is constructed based on the output correlation characteristics and the output video frame. Specifically, the affine transformation processing involves processing the integrated correlation features and the retained video frames using a coupled layer structure with added branches. The step of constructing the target compressed video frame after video compression processing based on the output correlation features and the output video frame includes: The output video frames are spliced ​​together to obtain compressed video frames; The output correlation features are distributed and mapped to obtain a Gaussian distribution of the features. The compressed video frame and the characteristic Gaussian distribution are integrated to obtain the target compressed video frame after video compression.

2. The method according to claim 1, characterized in that, The step of extracting video correlation features from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames includes: The original continuous video frames are decomposed to obtain multiple sets of original adjacent video frames. For each group of the original adjacent video frames, determine the component correlation characteristics between the original adjacent video frames; The correlation features of each component are fused to obtain the comprehensive correlation features corresponding to the original continuous video frames.

3. The method according to claim 2, characterized in that, Determining the component correlation features between the original adjacent video frames includes: The preceding and following video frames from the original adjacent video frames are input into the convolutional layer of the densely connected network, and the component correlation features between the original adjacent video frames are determined based on the convolutional layer.

4. The method according to claim 1, characterized in that, The termination condition for the affine transformation includes at least the number of affine transformations reaching a preset number.

5. The method according to claim 1, characterized in that, After constructing the target compressed video frame after video compression processing, the method further includes: The target compressed video frame is decomposed to obtain the compressed video frame and the feature Gaussian distribution; An inverse transform process is used to reconstruct the original continuous video frames before compression based on the compressed video frames and the characteristic Gaussian distribution.

6. A video frame rate processing device, characterized in that, include: A continuous video frame acquisition module is used to acquire the original continuous video frames to be compressed, wherein the original continuous video frames include at least three consecutive frames; and are composed of multiple feature vectors describing the relationship between the consecutive frames. The correlation feature determination module is used to extract video correlation features from the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include the component correlation features corresponding to adjacent video frames in the original continuous video frames. The compressed video frame determination module is used to construct a target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frame includes the compressed video frame corresponding to the original continuous video frames and the feature Gaussian distribution corresponding to the comprehensive correlation features; The compressed video frame determination module includes: The video frame retention determination unit is used to determine the video frames to be retained from the original continuous video frames based on a pre-selection filtering method, wherein the pre-selection filtering method includes at least an interval frame filtering method, and the interval frame filtering method includes an odd-numbered frame filtering method and an even-numbered frame filtering method. An affine transformation processing unit is used to perform affine transformation processing on the comprehensive correlation features and the retained video frames to obtain output correlation features and output video frames, and to perform affine transformation processing on the output correlation features and the output video frames. The compressed video frame determination unit is used to construct the target compressed video frame after video compression processing based on the output correlation characteristics and the output video frame, provided that the affine transformation termination condition is met. Specifically, the affine transformation processing involves processing the integrated correlation features and the retained video frames using a coupled layer structure with added branches. The compressed video frame determination unit is specifically used for: The output video frames are spliced ​​together to obtain compressed video frames; The output correlation features are distributed and mapped to obtain a Gaussian distribution of the features. The compressed video frame and the characteristic Gaussian distribution are integrated to obtain the target compressed video frame after video compression.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video frame rate processing method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the video frame rate processing method according to any one of claims 1-5.

Images