Video frame rate processing method and device, equipment and storage medium

Abstract

The invention discloses a video frame rate processing method and device, equipment and a storage medium. The method comprises the steps that original continuous video frames to be subjected to video compression processing are acquired, and the original continuous video frames at least comprise three continuous frames; and performing video correlation feature extraction on the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames. According to the original continuous video frames and the comprehensive correlation features, a target compressed video frame after video compression processing is constructed, the target compressed video frame comprises a compressed video frame and feature Gaussian distribution, so that redundant information of the video frame can be efficiently removed, and the video quality is improved. On the premise of not sacrificing visual quality, the size of the video file is obviously reduced, so that the compression efficiency is improved, the bandwidth use is optimized, and the storage cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video frame rate processing method, device, equipment and storage medium. Background Art

[0002] With the development of multimedia technology and the Internet, the consumption methods of video content have become diversified, and users expect to be able to watch videos seamlessly on different types of devices, including traditional TVs, computers, tablets, smartphones, etc. However, these devices may support different frame rate standards, which leads to the need for video frame rate conversion.

[0003] In the field of digital video processing and playback, video frame rate (Frames Per Second, FPS) refers to the number of static images (frames) displayed per second. Although the current video frame rate extraction and compression technology is advanced, it still has some significant shortcomings. First, efficient compression is often accompanied by a loss of visual quality, especially at high compression ratios, problems such as block effects, blur or color distortion may occur. Secondly, complex algorithms increase computational complexity and processing time, causing latency challenges for real-time applications, and perform differently on different types of video content and lack adaptability. In addition, inter-frame dependencies may cause error propagation and affect multi-frame quality, and dynamically adjusting the bit rate to maintain smooth playback is also difficult, which is not conducive to practical applications. Summary of the invention

[0004] The present invention provides a video frame rate processing method, device, equipment and storage medium to efficiently remove redundant information of video frames and significantly reduce the size of video files without sacrificing visual quality, thereby improving compression efficiency, optimizing bandwidth usage and reducing storage costs.

[0005] According to one aspect of the present invention, a video frame rate processing method is provided. The method comprises:

[0006] Acquire original continuous video frames to be subjected to video compression processing, wherein the original continuous video frames include at least three continuous frames;

[0007] Extracting video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include component correlation features corresponding to adjacent video frames in the original continuous video frames;

[0008] According to the original continuous video frames and the comprehensive correlation features, a target compressed video frame after video compression processing is constructed, wherein the target compressed video frame includes a compressed video frame corresponding to the original continuous video frame and a characteristic Gaussian distribution corresponding to the comprehensive correlation features.

[0009] According to another aspect of the present invention, a video frame rate processing device is provided. The device comprises:

[0010] A continuous video frame acquisition module, used to acquire original continuous video frames to be subjected to video compression processing, wherein the original continuous video frames include at least three continuous frames;

[0011] A correlation feature determination module, used for extracting video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include component correlation features corresponding to adjacent video frames in the original continuous video frames;

[0012] The compressed video frame determination module is used to construct a target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frame includes a compressed video frame corresponding to the original continuous video frame and a characteristic Gaussian distribution corresponding to the comprehensive correlation features.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video frame rate processing method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video frame rate processing method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention is to obtain the original continuous video frames to be subjected to video compression processing. The original continuous video frames are subjected to video correlation feature extraction to obtain the comprehensive correlation features corresponding to the original continuous video frames. According to the original continuous video frames and the comprehensive correlation features, the target compressed video frames after the video compression processing are constructed, which can efficiently remove the redundant information of the video frames and significantly reduce the size of the video file without significantly sacrificing the visual quality, thereby improving the compression efficiency, optimizing the bandwidth usage and reducing the storage cost. At the same time, it can also enhance the accuracy of inter-frame prediction, speed up the encoding and decoding speed, and provide strong support for intelligent video processing tasks, which is suitable for various application scenarios from consumer level to professional level.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flowchart of a video frame rate processing method provided according to Embodiment 1 of the present invention;

[0022] Figure 2 is a flowchart of a video frame rate processing method provided according to Embodiment 2 of the present invention;

[0023] Figure 3 is a structural diagram of a video frame rate processing device provided according to Embodiment 3 of the present invention;

[0024] Figure 4 It is a structural schematic diagram of an electronic device for implementing the video frame rate processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1 This is a flowchart of a video frame rate processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case where video is compressed and transmitted. The method can be executed by a video frame rate processing device. The video frame rate processing device can be implemented in the form of hardware and / or software. The video frame rate processing device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0029] S101: Acquire original continuous video frames to be compressed.

[0030] The original continuous video frames may refer to video frame data to be compressed. It should be noted that the original continuous video frames include at least three continuous frames.

[0031] Specifically, at least three consecutive video frames need to be extracted from the original video stream as processing objects, ie, original consecutive video frames. The original consecutive video frames are preferably directly adjacent frames, depending on the content of the video and the requirements of the compression algorithm.

[0032] S102: extract video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames.

[0033] The comprehensive correlation feature may refer to the correlation feature between the original continuous video frames. Exemplarily, the comprehensive correlation feature includes component correlation features corresponding to adjacent video frames in the original continuous video frames. It should be noted that the component correlation feature may refer to the correlation feature between adjacent video frames in the original continuous video frames, such as the correlation feature between the first frame and the second frame or the correlation feature between the second frame and the third frame.

[0034] Specifically, for each frame in the original continuous video frames, the similarities and changes between frames are analyzed, such as calculating the differences between adjacent frames (such as changes at the pixel level), and detecting motion vectors, color distribution, etc. The features describing the relationship between frames, such as motion compensation parameters and texture similarity, can be obtained through analysis. Combining all the above correlation features, one or more feature vectors that can fully describe the relationship between consecutive frames are formed, and they can be determined as comprehensive correlation features. Comprehensive correlation features can not only reflect simple differences between adjacent frames, but also contain more complex patterns, such as long-term dependencies or periodic motions.

[0035] S103: construct a target compressed video frame after video compression processing according to the original continuous video frames and the comprehensive correlation features.

[0036] The target compressed video frame includes the compressed video frame corresponding to the original continuous video frame and the characteristic Gaussian distribution corresponding to the comprehensive correlation feature. It should be noted that the characteristic Gaussian distribution refers to modeling certain characteristic values ​​(such as brightness, color, edge strength, etc.) as a probability density function that follows a normal distribution, which helps to identify which parts can be simplified or discarded without significantly affecting the overall quality during the compression process.

[0037] Specifically, using the obtained comprehensive correlation features combined with the data of the original video frame, a compressed video frame can be created through a pre-trained encoding strategy. The goal here is to minimize data redundancy while maintaining the highest possible visual quality. In this process, statistical models such as Gaussian distribution can also be used to represent the distribution of features. Feature Gaussian distribution can help predict and encode information that appears frequently in the original video frame but can be simplified by a mathematical model. In practical applications, a mixed Gaussian model (GMM) or other more complex probabilistic models may also be used to better capture different types of feature distributions.

[0038] Exemplarily, constructing a target compressed video frame after video compression processing according to the original continuous video frames and the comprehensive correlation features includes:

[0039] Based on the pre-selected screening method, a retained video frame is determined from the original continuous video frames. The comprehensive correlation feature and the retained video frame are affine transformed to obtain an output correlation feature and an output video frame, and the output correlation feature and the output video frame are affine transformed;

[0040] When the affine transformation end condition is met, a target compressed video frame after video compression processing is constructed according to the output correlation feature and the output video frame.

[0041] The preselected screening method at least includes an interval frame screening method, and the interval frame screening method includes an odd frame screening method and an even frame screening method. Preferably, the present invention determines the odd frame screening method as the interval frame screening method.

[0042] Specifically, the retained video frames and comprehensive correlation features obtained by the pre-selection screening method are respectively subjected to affine transformation processing, so that the output correlation features and the output video frames can be obtained.

[0043] For example, the output after affine transformation can be expressed by the following formula:

[0044]

[0045] in, It can mean to keep the video frame. It can refer to the comprehensive correlation characteristics. can refer to the output video frame, can refer to the output correlation characteristics, η and It can be any function and does not need to have an inverse function, so it can be a complex convolutional network. It should be noted that in the technical solution of the embodiment of the present invention, the number of times the affine transformation process is performed can be determined according to the actual situation, so l can refer to the number of times the current affine transformation process is performed.

[0046] Furthermore, in the technical solution of the present invention, in order to enhance the conversion capability of the model, a coupling layer structure similar to that in RealNVP is selected, and branches are added to further optimize the retention of video frames and comprehensive correlation features, as shown below:

[0047]

[0048] in, It can mean to keep the video frame. It can refer to the comprehensive correlation characteristics. can refer to the output video frame, It can refer to the output correlation characteristics, ρ(·), and ψ(·) are both transformation functions, and both use lightweight Dense Block. At the same time, the Sigmoid function is used after ρ(·) to prevent the gradient explosion caused by the exp operation.

[0049] When the affine transformation end condition is met, a target compressed video frame after video compression processing is constructed according to the output correlation feature and the output video frame. The affine transformation end condition at least includes that the number of affine transformations reaches a preset number of times. If the affine transformation end condition is not met, the output correlation feature and the output video frame can continue to be processed by the next affine transformation, so as to obtain the next output correlation feature and the output video frame, until the affine transformation end condition is met.

[0050] Exemplarily, constructing a target compressed video frame after video compression processing based on the output correlation features and the output video frames includes: splicing the output video frames to obtain compressed video frames; distribution mapping the output correlation features to obtain characteristic Gaussian distribution; integrating the compressed video frames and the characteristic Gaussian distribution to obtain the target compressed video frame after video compression processing.

[0051] Specifically, multiple output video frames are merged into a compressed video frame through an effective encoding method to reduce redundant information and improve the compression rate. The extracted correlation features are converted into a simple probability distribution (such as an isotropic Gaussian distribution) to facilitate subsequent processing and reconstruction during decoding. Exemplarily, a normalization flow model (such as RealNVP, Glow, etc.) can be used to map complex correlation features to a simple target distribution (such as a standard normal distribution). This involves a series of reversible transformations to ensure that the log-likelihood can be accurately calculated and new samples can be generated from the target distribution. Combining the information of the compressed video frame and the characteristic Gaussian distribution, a comprehensive target compressed video frame is generated, which contains both visual content and retains temporal dynamic information.

[0052] Through the above technical solution, the video frames can be effectively compressed and integrated with the Gaussian distribution of correlation characteristics to form a comprehensive target compressed video frame, which not only improves the compression efficiency but also retains important spatiotemporal information, providing a solid foundation for subsequent applications (such as video transmission, storage, analysis, etc.).

[0053] The technical solution of the embodiment of the present invention is to obtain the original continuous video frames to be subjected to video compression processing. The original continuous video frames are subjected to video correlation feature extraction to obtain the comprehensive correlation features corresponding to the original continuous video frames. According to the original continuous video frames and the comprehensive correlation features, the target compressed video frames after the video compression processing are constructed, which can efficiently remove the redundant information of the video frames and significantly reduce the size of the video file without significantly sacrificing the visual quality, thereby improving the compression efficiency, optimizing the bandwidth usage and reducing the storage cost. At the same time, it can also enhance the accuracy of inter-frame prediction, speed up the encoding and decoding speed, and provide strong support for intelligent video processing tasks, which is suitable for various application scenarios from consumer level to professional level.

[0054] On the basis of the above embodiments, after constructing the target compressed video frame after video compression processing, it also includes: decomposing the target compressed video frame to obtain the compressed video and the characteristic Gaussian distribution; using inverse transformation processing to reconstruct the original continuous video frame before compression processing according to the compressed video and the characteristic Gaussian distribution.

[0055] That is, according to the compressed video and characteristic Gaussian distribution obtained by decomposition, an inverse transformation process is performed, so that the original continuous video frames before the compression process can be reconstructed. The specific process can be as follows:

[0056]

[0057] in, It can mean to keep the video frame. It can refer to the comprehensive correlation characteristics. can refer to the output video frame, It can refer to the output correlation characteristics, ρ(·), and ψ(·) are both transformation functions, and both use lightweight Dense Block. At the same time, the Sigmoid function is used after ρ(·) to prevent the gradient explosion caused by the exp operation.

[0058] After obtaining the comprehensive correlation features and the retained video frames, the correlation module restores the original intermediate frames from the retained video frames according to the correlation features. That is, the comprehensive correlation features are separated into forward features and backward features along the channel dimension, and then the intermediate frames are reconstructed by combining the feature restoration modules.

[0059] The present invention uses Gaussian distribution to encode inter-frame information and reconstruct lost intermediate frames, which effectively improves computational efficiency and is suitable for real-time processing applications. Compared with traditional interpolation methods, it can reduce computing resource consumption while maintaining high-precision reconstruction effects. Compared with existing large-scale deep learning interpolation algorithms, this method greatly reduces the number of parameters and significantly improves processing speed, making it particularly suitable for real-time video processing scenarios.

[0060] Embodiment 2

[0061] Figure 2 This is a flowchart of a video frame rate processing method provided in the second embodiment of the present invention. Based on the above embodiments, this embodiment further refines the video correlation feature extraction of the original continuous video frames to obtain the comprehensive correlation features corresponding to the original continuous video frames. Figure 2 As shown, the method includes:

[0062] S201: Acquire original continuous video frames to be compressed.

[0063] S202: Decompose the original continuous video frames to obtain multiple groups of original adjacent video frames.

[0064] The original adjacent video frames may refer to mutually adjacent video frames in the original continuous video frames. For example, if the number of original continuous video frames is three, the original adjacent video frames may be the first frame and the second frame, or the second frame and the third frame.

[0065] Specifically, the original continuous video frames are decomposed and reassembled to obtain multiple groups of original adjacent video frames.

[0066] S203: For each group of the original adjacent video frames, determine a component correlation feature between the original adjacent video frames.

[0067] The component correlation feature may refer to a correlation feature between two adjacent video frames in original adjacent video frames.

[0068] Specifically, for each group of original adjacent video frames, the correlation between the original adjacent video frames is analyzed and summarized to form component correlation features.

[0069] Exemplarily, determining the component correlation feature between the original adjacent video frames includes:

[0070] The preceding video frame and the succeeding video frame in the original adjacent video frames are input into a convolutional layer in a densely connected network, and component correlation features between the original adjacent video frames are determined based on the convolutional layer.

[0071] Specifically, the convolutional layers in DenseNet can be used to extract spatial feature images of the previous and next frames in the original adjacent video frames. Based on the extracted feature images, the component correlation features between the previous and next frames are calculated, and cross-correlation, Euclidean distance or other similarity metrics can be used to quantify the differences and similarities between the two frames. Alternatively, additional convolutional layers or fully connected layers can be introduced to extract higher-level component correlation features from the fused feature maps. By integrating the extracted component correlation features, it is convenient for subsequent applications such as video compression and frame interpolation, and the temporal dependency between frames can be captured. It can also make full use of the structural advantages of DenseNet to improve feature expression capabilities and model performance.

[0072] S204: Perform feature fusion processing on each of the component correlation features to obtain comprehensive correlation features corresponding to the original continuous video frames.

[0073] Specifically, the correlation features of each component are connected along the channel dimension, and then the comprehensive correlation features can be obtained.

[0074] Furthermore, the comprehensive correlation feature can be obtained by the following formula:

[0075] f=Concat(D b (I1,I2),D f (I2,I3));

[0076] Among them, Concat means that the features are connected along the channel dimension, D b Denotes the forward feature extraction block composed of Dense Block, D f represents the backward feature extraction block, f refers to the comprehensive correlation feature, and (I1, I2) and (I2, I3) refer to the groups of original adjacent video frames.

[0077] On the other hand, in the inverse transform process, the compressed intermediate frame can be restored from the reconstructed comprehensive correlation features. Taking the original adjacent video frames as three frames as an example, the compressed intermediate frame can be restored by the following formula:

[0078]

[0079] Among them, Avg means taking the average, Represents the backward reconstruction module, represents the forward reconstruction module, I1 and I3 refer to the retained video frames, f I Refers to the comprehensive correlation characteristics after reconstruction.

[0080] S205: Construct a target compressed video frame after video compression processing according to the original continuous video frames and the comprehensive correlation features.

[0081] The technical solution of the embodiment of the present invention can effectively decompose the original continuous video frames into multiple groups of original adjacent video frames by decomposing the original continuous video frames, and extract component correlation features for each group of original adjacent video frames. Through feature fusion processing, a comprehensive comprehensive correlation feature is obtained, which can not only capture the temporal dependency between frames, but also provide rich context information for subsequent applications (such as video compression, frame interpolation, action recognition, etc.). On the other hand, the technical solution of the embodiment of the present invention not only improves the ability of feature expression, but also can make full use of the advantages of densely connected networks (DenseNet) to ensure the effectiveness of information flow and gradient propagation. In practical applications, the network configuration and feature extraction strategy can be adjusted according to the needs of specific tasks to achieve the best results.

[0082] Embodiment 3

[0083] Figure 3 FIG. 1 is a schematic diagram of a video frame rate processing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0084] The continuous video frame acquisition module 301 is used to acquire original continuous video frames to be subjected to video compression processing, wherein the original continuous video frames include at least three continuous frames;

[0085] A correlation feature determination module 302 is used to extract video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include component correlation features corresponding to adjacent video frames in the original continuous video frames;

[0086] The compressed video frame determination module 303 is used to construct a target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frame includes a compressed video frame corresponding to the original continuous video frame and a characteristic Gaussian distribution corresponding to the comprehensive correlation features.

[0087] The technical solution of the embodiment of the present invention is to obtain the original continuous video frames to be subjected to video compression processing. The original continuous video frames are subjected to video correlation feature extraction to obtain the comprehensive correlation features corresponding to the original continuous video frames. According to the original continuous video frames and the comprehensive correlation features, the target compressed video frames after the video compression processing are constructed, which can efficiently remove the redundant information of the video frames and significantly reduce the size of the video file without significantly sacrificing the visual quality, thereby improving the compression efficiency, optimizing the bandwidth usage and reducing the storage cost. At the same time, it can also enhance the accuracy of inter-frame prediction, speed up the encoding and decoding speed, and provide strong support for intelligent video processing tasks, which is suitable for various application scenarios from consumer level to professional level.

[0088] Optionally, the correlation feature determination module 302 includes:

[0089] An adjacent video frame determining unit, configured to decompose the original continuous video frames to obtain a plurality of groups of original adjacent video frames;

[0090] A first correlation feature determining unit, configured to determine, for each group of the original adjacent video frames, a component correlation feature between the original adjacent video frames;

[0091] The second correlation feature determination unit is used to perform feature fusion processing on each of the component correlation features to obtain the comprehensive correlation features corresponding to the original continuous video frames.

[0092] Optionally, the first correlation feature determining unit is specifically configured to:

[0093] The preceding video frame and the succeeding video frame in the original adjacent video frames are input into a convolutional layer in a densely connected network, and component correlation features between the original adjacent video frames are determined based on the convolutional layer.

[0094] Optionally, the compressed video frame determination module 303 includes:

[0095] A reserved video frame determination unit, configured to determine the reserved video frames from the original continuous video frames based on a preselected screening method, wherein the preselected screening method at least includes an interval frame screening method, and the interval frame screening method includes an odd frame screening method and an even frame screening method;

[0096] an affine transformation processing unit, configured to perform affine transformation processing on the comprehensive correlation feature and the retained video frame to obtain an output correlation feature and an output video frame, and perform affine transformation processing on the output correlation feature and the output video frame;

[0097] The compressed video frame determination unit is used to construct a target compressed video frame after video compression processing according to the output correlation feature and the output video frame when the affine transformation end condition is met.

[0098] Optionally, the affine transformation termination condition at least includes that the number of affine transformations reaches a preset number.

[0099] Optionally, the compressed video frame determination unit is specifically configured to:

[0100] Splicing the output video frames to obtain compressed video frames;

[0101] Performing distribution mapping on the output correlation features to obtain characteristic Gaussian distribution;

[0102] The compressed video frame and the characteristic Gaussian distribution are integrated to obtain a target compressed video frame after video compression processing.

[0103] Optionally, the device further comprises a continuous video frame restoration module.

[0104] Continuous video frame restoration module, specifically used for:

[0105] Decomposing the target compressed video frame to obtain the compressed video and the characteristic Gaussian distribution;

[0106] The original continuous video frames before compression are reconstructed based on the compressed video and the characteristic Gaussian distribution by using an inverse transformation process.

[0107] The video frame rate processing device provided in the embodiment of the present invention can execute the video frame rate processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0108] Embodiment 4

[0109] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0110] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0112] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a video frame rate processing method.

[0113] In some embodiments, the video frame rate processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the video frame rate processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the video frame rate processing method in any other appropriate manner (e.g., by means of firmware).

[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0116] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0119] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0121] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video frame rate processing method, characterized in that: include: Acquire original continuous video frames to be subjected to video compression processing, wherein the original continuous video frames include at least three continuous frames; Extracting video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include component correlation features corresponding to adjacent video frames in the original continuous video frames; According to the original continuous video frames and the comprehensive correlation features, a target compressed video frame after video compression processing is constructed, wherein the target compressed video frame includes a compressed video frame corresponding to the original continuous video frame and a characteristic Gaussian distribution corresponding to the comprehensive correlation features.

2. The method according to claim 1, characterized in that The extracting video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames includes: Decomposing the original continuous video frames to obtain multiple groups of original adjacent video frames; For each group of the original adjacent video frames, determining a component correlation feature between the original adjacent video frames; The component correlation features are subjected to feature fusion processing to obtain comprehensive correlation features corresponding to the original continuous video frames.

3. The method according to claim 2, characterized in that The determining of the component correlation feature between the original adjacent video frames comprises: The preceding video frame and the succeeding video frame in the original adjacent video frames are input into a convolutional layer in a densely connected network, and component correlation features between the original adjacent video frames are determined based on the convolutional layer.

4. The method according to claim 1, characterized in that: The step of constructing a target compressed video frame after video compression processing according to the original continuous video frames and the comprehensive correlation features comprises: Determine the retained video frames from the original continuous video frames based on a preselected screening method, wherein the preselected screening method at least includes an interval frame screening method, and the interval frame screening method includes an odd frame screening method and an even frame screening method; Performing affine transformation processing on the comprehensive correlation feature and the retained video frame to obtain output correlation feature and output video frame, and performing affine transformation processing on the output correlation feature and the output video frame; When the affine transformation end condition is met, a target compressed video frame after video compression processing is constructed according to the output correlation feature and the output video frame.

5. The method according to claim 4, characterized in that The affine transformation termination condition at least includes that the number of affine transformations reaches a preset number.

6. The method according to claim 4, characterized in that The step of constructing a target compressed video frame after video compression processing according to the output correlation feature and the output video frame comprises: Splicing the output video frames to obtain compressed video frames; Performing distribution mapping on the output correlation features to obtain characteristic Gaussian distribution; The compressed video frame and the characteristic Gaussian distribution are integrated to obtain a target compressed video frame after video compression processing.

7. The method according to claim 1, characterized in that After constructing the target compressed video frame after the video compression process, the method further includes: Decomposing the target compressed video frame to obtain the compressed video and the characteristic Gaussian distribution; The original continuous video frames before compression are reconstructed based on the compressed video and the characteristic Gaussian distribution by using an inverse transformation process.

8. A video frame rate processing device, characterized in that: include: A continuous video frame acquisition module, used to acquire original continuous video frames to be subjected to video compression processing, wherein the original continuous video frames include at least three continuous frames; A correlation feature determination module, used for extracting video correlation features from the original continuous video frames to obtain comprehensive correlation features corresponding to the original continuous video frames, wherein the comprehensive correlation features include component correlation features corresponding to adjacent video frames in the original continuous video frames; The compressed video frame determination module is used to construct a target compressed video frame after video compression processing based on the original continuous video frames and the comprehensive correlation features, wherein the target compressed video frame includes a compressed video frame corresponding to the original continuous video frame and a characteristic Gaussian distribution corresponding to the comprehensive correlation features.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video frame rate processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video frame rate processing method according to any one of claims 1 to 7 when executed.

Images