Video encoding and decoding method, device, equipment, storage medium and computer program

Abstract

The invention discloses a video encoding and decoding method and device, equipment, a storage medium and a computer program, and belongs to the technical field of data compression. The method includes determining a motion vector of a current block in a current video frame, determining a global motion vector of the current video frame, and encoding the current block based on the motion vector of the current block and the global motion vector of the current video frame. The global motion vector of the current video frame can indicate the jitter condition of the current video frame relative to the static frame, so that the current block is coded by determining the global motion vector of the current video frame and based on the global motion vector of the current video frame and the motion vector of the current block, and therefore, the jitter condition of the current video frame relative to the static frame can be indicated. The video jitter compensation can be realized while the hardware cost is not increased.

Description

Technical Field

[0001] The present application relates to the field of data compression technology, and in particular to a video encoding and decoding method, device, equipment, storage medium and computer program. Background Art

[0002] In recent years, with the advancement and development of science and technology, many electronic devices support video shooting. When shooting a video, if the electronic device shakes, it will cause shaking in the video screen, that is, the video screen will appear blurred, diffused or smeared. This situation will not only affect the clarity of the video, but also make the viewer feel dizzy and difficult to observe effective information.

[0003] In the related art, in order to eliminate the problems caused by the shaking of electronic devices, special sensors (such as gyroscopes and accelerometers) are usually used to detect the shaking of the electronic devices, and then the video is processed according to the detection results to achieve optimization of the video picture.

[0004] However, the above method requires adding additional sensors to detect jitter, which greatly increases the hardware cost. Summary of the invention

[0005] The present application provides a video encoding and decoding method, apparatus, device, storage medium and computer program, which can solve the problem of high hardware cost in related technologies. The technical solution is as follows:

[0006] In a first aspect, a video encoding method is provided, the method comprising: determining a global motion vector of a current video frame, the current video frame being a video frame to be encoded in a video signal, the global motion vector indicating a jitter condition of the current video frame relative to a static video frame in the video signal, the static video frame being a video frame in which no jitter occurs in the video screen, and encoding the current block based on the global motion vector of the current video frame.

[0007] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static frame, the present application can determine the global motion vector of the current video frame, and encode the current block based on the global motion vector of the current video frame and the motion vector of the current block. In this way, compensation for video jitter can be achieved without increasing hardware costs.

[0008] Since the current video frame may be a video frame using inter-frame prediction coding or a video frame using intra-frame block copy coding. In the case where the current video frame is coded using inter-frame prediction coding or intra-frame block copy coding, the motion vector of the current block in the current video frame may be determined, the motion vector indicating the positional relationship between the current block and a reference block corresponding to the current block, the current video frame being a video frame to be encoded in the video signal, the global motion vector of the current video frame is determined, the global motion vector indicating the jitter of the current video frame relative to a still video frame in the video signal, the still video frame being a video frame where no jitter occurs in the video screen, and the current block is encoded based on the motion vector of the current block and the global motion vector of the current video frame.

[0009] When the current video frame adopts inter-frame prediction coding, a reference global motion vector can be determined, where the reference global motion vector indicates the jitter of the current video frame relative to the reference video frame corresponding to the current video frame; based on the reference global motion vector, the global motion vector of the current video frame is determined.

[0010] Optionally, if the acceleration vector of the previous video frame is not stored, the step of determining the reference global motion vector is performed, the acceleration vector indicating the jitter acceleration of the previous video frame, and the previous video frame is a previous video frame adjacent to the current video frame in the video signal.

[0011] The present application can determine the reference global motion vector only when the acceleration vector of the previous video frame is not stored, which can reduce the amount of calculation and effectively improve the efficiency of determining the global motion vector.

[0012] In the case where the current video frame adopts inter-frame prediction coding, if the acceleration vector of the previous video frame is stored, the jitter of the current video frame relative to the static video frame is predicted based on the acceleration vector and the global motion vector of the previous video frame to obtain a first predicted global motion vector, where the previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the acceleration vector indicates the jitter acceleration of the previous video frame; based on the first predicted global motion vector, the global motion vector of the current video frame is determined.

[0013] Since the jitter acceleration of the current video frame may still be the jitter acceleration of the previous frame, the present application can predict the global motion vector of the current video frame based on the global motion vector of the previous frame and the acceleration vector of the previous video frame, that is, take the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame as the first predicted global motion vector, and then determine the global motion vector of the current video frame based on the first predicted global motion vector. This can reduce the amount of calculation and effectively improve the efficiency of determining the global motion vector.

[0014] Optionally, based on the motion vector of some blocks with static objects in the current video frame, the jitter of the current video frame relative to the static video frame is predicted to obtain a second predicted global motion vector; determining the global motion vector of the current video frame based on the first predicted global motion vector includes: if the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to a difference threshold, determining a reference global motion vector, the reference global motion vector indicating the jitter of the current video frame relative to a reference video frame corresponding to the current video frame; determining the global motion vector of the current video frame based on the reference global motion vector.

[0015] When the difference between the first predicted global motion vector and the second predicted global motion vector is less than the difference threshold, the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame is used as the global motion vector of the current video frame. In this way, the amount of calculation can be reduced, the efficiency of determining the global motion vector can be effectively improved, and the accuracy of the global motion vector of the current video frame that is finally determined can be guaranteed.

[0016] Optionally, determining the reference global motion vector includes: determining the reference global motion vector based on motion vectors of each block having a static object in the current video frame.

[0017] Since the change in the position of a static object in the current video frame relative to the position of the static object in the reference video frame can indicate the jitter of the current video frame relative to the reference video frame, the reference global motion vector determined based on the motion vectors of each block with static objects in the current video frame is more accurate.

[0018] Optionally, after determining the global motion vector of the current video frame, the method further includes: determining and storing the acceleration vector of the current video frame based on the global motion vector of the current video frame and the global motion vector of a previous video frame, wherein the previous video frame is a previous video frame adjacent to the current video frame in the video signal.

[0019] After each global motion vector of a video frame is determined, the acceleration vector of the current video frame may be determined, thereby updating the acceleration vector and ensuring the real-time performance of the acceleration vector, so as to predict the global motion vector of subsequent video frames.

[0020] In the case where the current video frame adopts intra-frame block copy encoding, the global motion vector of the current video frame can be determined based on the global motion vector of the previous video frame and the global motion vector of the next video frame, wherein the previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the next video frame is the next video frame adjacent to the current video frame in the video signal.

[0021] Optionally, encoding the current block based on the motion vector of the current block and the global motion vector of the current video frame includes: encoding the global motion vector of the current video frame, and the motion vector and prediction residual of the current block into a bitstream, wherein the prediction residual indicates the difference between the current block and the reference block.

[0022] In the case where the current video frame adopts inter-frame prediction coding, jitter compensation can be performed on the motion vector and prediction residual of the current block based on the global motion vector of the current video frame to obtain a corrected motion vector and a corrected prediction residual of the current block, wherein the prediction residual indicates the difference between the current block and the reference block; and the corrected motion vector and the corrected prediction residual of the current block are encoded into the bitstream.

[0023] Encoding the corrected motion vector and the corrected prediction residual of the current block into the bitstream can ensure accurate compensation of video jitter without adding additional information to the bitstream.

[0024] Optionally, the performing jitter compensation on the motion vector and prediction residual of the current block based on the global motion vector of the current video frame to obtain a corrected motion vector and a corrected prediction residual of the current block includes: performing jitter compensation on the motion vector of the current block based on the global motion vector of the current video frame to obtain a corrected motion vector of the current block; re-determining a reference block corresponding to the current block based on the corrected motion vector of the current block; and determining a difference between the current block and the re-determined reference block to obtain a corrected prediction residual of the current block.

[0025] The current video frame may also be a video frame that adopts ordinary intra-frame prediction coding. In this case, the prediction mode of the current block in the current video frame can be determined, and the prediction mode is used to generate a prediction block corresponding to the current block. The current video frame is a video frame to be encoded in the video signal; the global motion vector of the current video frame is determined, and the global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal, and the static video frame is a video frame in which the video screen does not jitter; based on the indication information of the prediction mode of the current block and the global motion vector of the current video frame, the current block is encoded.

[0026] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static frame, for the current video frame that is intra-coded through a prediction mode, the present application can also determine the global motion vector of the current video frame, and encode the current block based on the global motion vector of the current video frame and the prediction mode of the current block. In this way, compensation for video jitter can be achieved without increasing hardware costs.

[0027] Optionally, encoding the current block based on the indication information of the prediction mode of the current block and the global motion vector of the current video frame includes: encoding the indication information of the prediction mode of the current block in the current video frame, the global motion vector of the current video frame and the prediction residual of the current block into the bitstream, the prediction residual indicating the difference between the current block and the prediction block corresponding to the current block, and the prediction block is determined based on the prediction mode of the current block.

[0028] In a second aspect, a video decoding method is provided, the method comprising: parsing a bit stream to obtain a global motion vector of a current video frame, and a motion vector and a prediction residual of a current block; wherein the current video frame is a video frame to be decoded in a video signal, the global motion vector indicates a jitter condition of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where no jitter occurs in the video screen, the motion vector indicates a positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates a difference between the current block and the reference block; based on the global motion vector of the current video frame, and the motion vector and the prediction residual of the current block, reconstructing the current block, wherein the reconstructed current block does not have jitter.

[0029] Optionally, reconstructing the current block based on the global motion vector of the current video frame, and the motion vector and prediction residual of the current block includes: determining an initial current block based on the motion vector of the current block and the prediction residual; and performing jitter compensation on the initial current block based on the global motion vector of the current video frame to obtain a reconstructed current block.

[0030] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, during the decoding process of the current block, the jitter of the current block can be compensated according to the global motion vector of the current video frame, so that the current block after compensation has no jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0031] In a third aspect, a video decoding method is provided, the method comprising: parsing a bit stream to obtain a corrected motion vector and a corrected prediction residual of a current block; wherein the corrected motion vector and the corrected prediction residual are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, the current video frame is a video frame to be decoded in a video signal, the global motion vector indicates the jitter of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector indicates the positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block; based on the corrected motion vector and the corrected prediction residual, the current block is reconstructed, wherein the reconstructed current block does not have jitter.

[0032] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, and the corrected motion vector and the corrected prediction residual are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, during the decoding process of the current block, the current block reconstructed according to the corrected motion vector and the corrected prediction residual of the current block does not have jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0033] In a fourth aspect, a video decoding method is provided, the method comprising: parsing a bit stream to obtain indication information of a prediction mode of a current block in a current video frame, a global motion vector and a prediction residual of the current video frame, wherein the current video frame is a video frame to be decoded in a video signal, the global motion vector indicates the jitter of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the prediction mode of the current block is used to determine a prediction block corresponding to the current block, and the prediction residual indicates the difference between the current block and the prediction block corresponding to the current block. Based on the indication information of the prediction mode of the current block in the current video frame, the global motion vector and the prediction residual of the current video frame, the current block is reconstructed, wherein the reconstructed current block does not have jitter.

[0034] Optionally, based on the prediction mode and prediction residual of the current block, an initial current block is determined, and then based on the global motion vector of the current video frame, jitter compensation is performed on the initial current block to obtain a reconstructed current block.

[0035] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, during the decoding process of the current block, the jitter of the current block can be compensated according to the global motion vector of the current video frame, so that the current block after compensation has no jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0036] In a fifth aspect, a video encoding device is provided, wherein the video encoding device has the function of implementing the video encoding method in the first aspect. The video encoding device includes at least one module, and the at least one module is used to implement the video encoding method provided in the first aspect.

[0037] In a sixth aspect, a video decoding device is provided, wherein the video decoding device has the function of implementing the video decoding method in the second aspect. The video decoding device includes at least one module, and the at least one module is used to implement the video decoding method provided in the second aspect.

[0038] In a seventh aspect, a video decoding device is provided, wherein the video decoding device has the function of implementing the video decoding method in the third aspect. The video decoding device includes at least one module, and the at least one module is used to implement the video decoding method provided in the third aspect.

[0039] In an eighth aspect, a video decoding device is provided, wherein the video decoding device has the function of implementing the video decoding method in the fourth aspect. The video decoding device includes at least one module, and the at least one module is used to implement the video decoding method provided in the fourth aspect.

[0040] In a ninth aspect, a video encoding device is provided, the video encoding device comprising a processor and a memory, the memory being used to store a computer program for executing the video encoding method provided in the first aspect. The processor is configured to execute the computer program stored in the memory to implement the video encoding method described in the first aspect.

[0041] Optionally, the video encoding device may further include a communication bus, and the communication bus is used to establish a connection between the processor and the memory.

[0042] In a tenth aspect, a video decoding device is provided, the video decoding device comprising a processor and a memory, the memory being used to store a computer program for executing the video decoding method provided in the second aspect, or a computer program for executing the video decoding method provided in the third aspect. The processor is configured to execute the computer program stored in the memory to implement the video decoding method described in the second aspect, or the video decoding method described in the third aspect, or the video decoding method described in the fourth aspect.

[0043] Optionally, the video decoding device may further include a communication bus, and the communication bus is used to establish a connection between the processor and the memory.

[0044] In the eleventh aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the steps of the video encoding method described in the first aspect, or executes the steps of the video decoding method described in the second aspect, or executes the steps of the video decoding method described in the third aspect, or executes the steps of the video decoding method described in the fourth aspect.

[0045] In the twelfth aspect, a computer program product comprising instructions is provided, and when the instructions are executed on a computer, the computer executes the steps of the video encoding method described in the first aspect, or executes the steps of the video decoding method described in the second aspect. In other words, a computer program is provided, and when the computer program is executed on a computer, the computer executes the steps of the video encoding method described in the first aspect, or executes the steps of the video decoding method described in the second aspect, or executes the steps of the video decoding method described in the third aspect, or executes the steps of the video decoding method described in the fourth aspect.

[0046] In the thirteenth aspect, a video coding and decoding system is provided, which includes a video coding device and a video decoding device, the video coding device is used to implement the steps of the video coding method described in the first aspect above, and the video decoding device is used to implement the steps of the video decoding method described in the second aspect above, or implement the steps of the video decoding method described in the third aspect above, or implement the steps of the video decoding method described in the fourth aspect above.

[0047] The technical effects obtained in the above-mentioned second to thirteenth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0049] Figure 2 is a flowchart of a video encoding method provided by an embodiment of the present application;

[0050] Figure 3 is a schematic diagram of determining a global motion vector provided by an embodiment of the present application;

[0051] Figure 4 is a schematic diagram of determining a global motion vector provided by an embodiment of the present application;

[0052] Figure 5 is a schematic diagram of a code stream provided in an embodiment of the present application;

[0053] Figure 6 is a flowchart of another video encoding method provided by an embodiment of the present application;

[0054] Figure 7 is a flowchart of a video decoding method provided by an embodiment of the present application;

[0055] Figure 8 is a flowchart of a video decoding method provided by an embodiment of the present application;

[0056] Fig. 9 is a structural diagram of a video encoding device provided in an embodiment of the present application;

[0057] Fig.10 is a structural diagram of a video decoding device provided in an embodiment of the present application;

[0058] Fig.11 It is a structural schematic diagram of a video decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0060] For ease of understanding, before explaining in detail the video encoding and decoding method provided in the embodiment of the present application, the nouns, application scenarios and implementation environment involved in the embodiment of the present application are first introduced.

[0061] Bit rate: In image compression, it refers to the code length required to encode a unit pixel. The higher the bit rate, the better the image reconstruction quality.

[0062] Raw image: It is the original data of the light source signal captured by the image sensor converted into a digital signal, which is lossless. The Raw image contains the original color information of the object, etc. The image sensor can be a complementary metal oxide semiconductor (CMOS) image sensor, or a charge-coupled device (CCD) image sensor, etc., which is not limited in the embodiments of the present application.

[0063] H.26x: A commonly used video compression format, such as H.264 / H.265, is a video compression standard based on a block-based video encoding / compression structure. H.265 is also known as high efficiency video coding (HEVC).

[0064] Block-based video coding / compression structure: It combines prediction, transform, quantization, block partition, in-loop filter, entropy coding and other technical means to compress the video to reduce the redundant signals in the video. The encoder divides the input video frame into coding tree units (CTUs) of similar size. For any CTU, it predicts the CTU and obtains the prediction residual corresponding to the CTU. The prediction residual is transformed to obtain the frequency domain transform coefficient, and then the frequency domain transform coefficient is lossily compressed by quantization. The quantized frequency domain transform coefficient is encoded into the bitstream through entropy coding. Then, based on the bitstream, the CTU is reconstructed through inverse quantization, inverse transformation and in-loop filter, so as to obtain the reconstructed value of the video frame, and other CTUs can be encoded based on the reconstructed value.

[0065] Among them, prediction and transformation refer to reducing redundancy in video signals by removing the correlation of pixels within each CTU in the time domain, spatial domain, and frequency domain. Prediction includes inter-frame prediction and intra-frame prediction. Quantization refers to reducing the number of bits representing the transform coefficient by reducing the precision of the transform coefficient. The purpose of quantization is to remove imperceptible details in the video frame, thereby effectively reducing the irrelevance of the data. In order to address the need for efficient encoding of screen content videos, there are also many screen content coding (SCC) tools. For example, HEVC SCC is a coding standard that includes SCC functions. HEVC SCC includes coding tools such as intra block copy (IBC), which can improve the compression efficiency of screen content by 30% to 40%.

[0066] Inter-frame prediction: Inter-frame prediction includes motion estimation (ME) unit and motion compensation (MC) unit. The basic idea of ​​motion estimation is to divide the video frame into many non-overlapping macroblocks, and then find the block that is most similar to the current block, namely the reference block / matching block, for each macroblock according to certain matching criteria. The relative displacement between the reference block and the current block is the motion vector (MV). The difference between the pixel value of the current block and the pixel value of the reference block is called the prediction residual between the current block and the reference block. The process of determining the MV is called motion estimation. Motion compensation refers to determining the reference block corresponding to the current block based on the MV, and then restoring the current block based on the prediction residual between the current block and the reference block.

[0067] Intra block copy (IBC): IBC is an important tool for screen content coding in HEVC SCC. It is a block-based prediction technology, and its mechanism is similar to inter-frame prediction or motion compensation. Motion compensation means that for the current block, the best matching block is found as the reference block corresponding to the current block through the motion search algorithm in the encoded block of the current video frame, and the matching relationship is indicated by the motion vector. The difference between the pixel value of the current block and the pixel value of the reference block is used as the prediction residual between the current block and the reference block. The prediction residual is processed by the transformation, quantization, entropy coding and other modules and then output to the bitstream.

[0068] Intra-frame prediction: The encoder can determine the prediction mode of the current block, and generate a prediction block corresponding to the current block according to the prediction mode, and then use the difference between the pixel value of the current block and the pixel value of the prediction block as the prediction residual between the current block and the prediction block. The prediction residual is processed by transformation, quantization, entropy coding and other modules and output to the code stream. It should be noted that the encoder can perform intra-frame prediction according to different prediction modes. For example, HEVC specifies 35 intra-frame prediction modes, including Planar mode, DC mode and 33 angle modes. For another example, H.265, i.e. versatile video coding (VVC), provides 67 intra-frame prediction modes, including 65 intra-frame angle prediction modes, Planar mode, and DC mode. In some cases, the intra-frame prediction encoded by the prediction mode is also called ordinary intra-frame prediction / non-intra-frame block copy, and the ordinary intra-frame prediction and the above-mentioned intra-frame block copy are collectively referred to as intra-frame prediction. For the convenience of description, the intra prediction encoded by the prediction mode will be uniformly referred to as normal intra prediction hereinafter. In this case, the intra prediction includes the normal intra prediction and intra block copy.

[0069] Group of pictures (GOP): The video bitstream consists of multiple GOPs. GOP is a group of continuous pictures and is the basic unit accessed by the video encoder and decoder. Each GOP consists of I frames and P frames.

[0070] I frame: A video frame encoded with intra-frame prediction, also known as a key frame. I frames are compressed and generated without reference to other frames. I frames describe the details of the image background and moving subjects. When decoding, only the data of I frames can be used to reconstruct the complete image. I frames are usually the first frame of each GOP.

[0071] P frame: A video frame using inter-frame prediction coding, also known as a forward prediction frame (forward reference frame). A P frame represents the difference between this frame and the previous I frame (or P frame). A P frame uses motion compensation to transmit the prediction residual and motion vector between it and the previous I or P frame.

[0072] In recent years, with the advancement and development of science and technology, many electronic devices support video shooting. When shooting a video, if the electronic device shakes, it will cause shaking in the video screen, that is, the video screen will be blurred, diffused or smeared. This situation will not only affect the clarity of the video, but also make the viewer feel dizzy and difficult to observe effective information. Taking the security monitoring scene as an example, video acquisition equipment such as cameras and pan-tilt heads may vibrate due to external reasons such as unstable support frames and weather, which will cause the video captured by the video acquisition equipment to shake. Shaking video will not only affect the viewer's experience, but also affect the accuracy of artificial intelligence (AI) tasks such as post-image processing and intelligent recognition.

[0073] At present, mechanical stabilization methods, optical stabilization methods or digital stabilization methods are used to eliminate the problems caused by the shaking of electronic devices. Among them, the mechanical stabilization method refers to the use of special sensors (for example, gyroscopes and accelerometers) to detect the shaking of electronic devices, and then process the video according to the detection results to achieve the optimization of the video picture. The optical stabilization method refers to the use of a movable lens combination. When the light passes through the lens, the path length of the light is changed to achieve the optimization of the video picture. The digital stabilization method refers to the use of feature point matching to process the video through a post-algorithm to achieve the optimization of the video picture.

[0074] However, the above-mentioned mechanical stabilization method requires the addition of additional sensors to detect jitter, and the optical stabilization method requires the setting of a movable lens combination, which will greatly increase the hardware cost. The feature point matching algorithm used in the above-mentioned digital stabilization method is highly complex and usually introduces matching errors, resulting in poor optimization of the video picture. Based on this, an embodiment of the present application proposes a video encoding and decoding method, which can determine the jitter of the current video frame relative to the still frame through the motion vector originally existing in the video encoding and decoding, that is, determine the global motion vector of the current video frame, and then encode the current block based on the global motion vector of the current video frame and the motion vector of the current block. In this way, compensation for video jitter can be achieved without increasing hardware costs. Among them, the motion vector of the current block indicates the positional relationship between the current block and the reference block corresponding to the current block.

[0075] Next, the implementation environment involved in the embodiments of the present application is introduced.

[0076] Please refer to Figure 1 , Figure 1is a schematic diagram of an implementation environment provided by an embodiment of the present application. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. Among them, the source device 10 can generate an encoded video, that is, a code stream. Therefore, the source device 10 can also be called an encoding device. The destination device 20 can decode the code stream generated by the source device 10. Therefore, the destination device 20 can also be called a decoding device. The link 30 can receive the encoded video generated by the source device 10, and can transmit the encoded video to the destination device 20. The storage device 40 can receive the encoded video generated by the source device 10, and can store the encoded video. Under such conditions, the destination device 20 can directly obtain the encoded video from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded video generated by the source device 10. Under such conditions, the destination device 20 can stream or download the encoded video stored in the storage device 40.

[0077] The source device 10 and the destination device 20 may each include one or more processors and a memory coupled to the one or more processors, the memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, any other medium that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, etc. For example, the source device 10 and the destination device 20 may each include a mobile phone, a smart phone, a personal digital assistant (PDA), a wearable device, a pocket PC (PPC), a tablet computer, a smart car, a smart TV, a smart speaker, a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video game console, a car computer, or the like.

[0078] The link 30 may include one or more media or devices capable of transmitting the encoded video from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media that enable the source device 10 to send the encoded video directly to the destination device 20 in real time. In an embodiment of the present application, the source device 10 may modulate the encoded video based on a communication standard, which may be a wireless communication protocol, etc., and may send the modulated video to the destination device 20. The one or more communication media may include wireless and / or wired communication media, for example, the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, which may be a local area network, a wide area network, or a global network (e.g., the Internet), etc. The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, etc., and the embodiment of the present application does not specifically limit this.

[0079] In a possible implementation, the storage device 40 may store the received encoded video sent by the source device 10, and the destination device 20 may directly obtain the encoded video from the storage device 40. Under such conditions, the storage device 40 may include any of a variety of distributed or locally accessible data storage media, for example, any of the multiple distributed or locally accessible data storage media may be a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing code streams.

[0080] In one possible implementation, the storage device 40 may correspond to a file server or another intermediate storage device that can store the code stream generated by the source device 10, and the destination device 20 may stream or download the image stored in the storage device 40. The file server may be any type of server that can store the encoded video and send the encoded video to the destination device 20. In one possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. The destination device 20 may obtain the encoded image through any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two suitable for obtaining the encoded video stored on the file server. The transmission of the encoded video from the storage device 40 may be streaming transmission, download transmission, or a combination of the two.

[0081] Figure 1 The implementation environment shown is only one possible implementation method, and the technology of the embodiment of the present application can be applied not only to Figure 1 The source device 10 that can encode images and the destination device 20 that can decode encoded videos can also be applicable to other devices that can encode videos and decode bitstreams, and the embodiments of the present application do not specifically limit this.

[0082] exist Figure 1 In the illustrated implementation, source device 10 includes data source 120, encoder 100, and output interface 140. In some embodiments, output interface 140 may include a modulator / demodulator (modem) and / or a transmitter, where the transmitter may also be referred to as a transmitter. Data source 120 may include a video capture device (e.g., a camera, etc.), an archive containing previously captured video, a feed interface for receiving video from a video content provider, and / or a computer graphics system for generating video, or a combination of these sources of video.

[0083] The data source 120 may send a video to the encoder 100, and the encoder 100 may encode the video received from the data source 120 to obtain an encoded video. The encoder may send the encoded video to the output interface. In some embodiments, the source device 10 directly sends the encoded video to the destination device 20 via the output interface 140. In other embodiments, the encoded video may also be stored on the storage device 40 for the destination device 20 to obtain and use for decoding and / or display later.

[0084] exist Figure 1 In the illustrated implementation environment, the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive the encoded video via the link 30 and / or from the storage device 40, and then send it to the decoder 200. The decoder 200 may decode the received encoded video to obtain a decoded video. The decoder may send the decoded video to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. In general, the display device 220 displays the decoded video. The display device 220 may be any of a variety of types of display devices, for example, the display device 220 may be a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0085] although Figure 1 1 and 2. Although not shown in the drawings, in some aspects, encoder 100 and decoder 200 may be integrated with an encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or in separate data streams. In some embodiments, if applicable, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

[0086] The encoder 100 and the decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (digital signal processing, DSP), application specific integrated circuits (application specific integrated circuits, ASIC), field programmable gate arrays (field-programmable gate array, FPGA), discrete logic, hardware or any combination thereof. If the technology of the embodiment of the present application is partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware to implement the technology of the embodiment of the present application. Any of the foregoing contents (including hardware, software, a combination of hardware and software, etc.) may be regarded as one or more processors. Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or decoders may be integrated as part of a combined encoder / decoder (encoder-decoder) in a corresponding device.

[0087] The embodiments of the present application may generally refer to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The term "signaling" or "sending" may generally refer to the transmission of syntax elements and / or other data used to decode the compressed video. This transmission may occur in real time or near real time. Alternatively, this communication may occur over a period of time, such as when the syntax elements are stored in the encoded bitstream to a computer-readable storage medium at the time of encoding, and the decoding device may then retrieve the syntax elements at any time after the syntax elements are stored to this medium.

[0088] Next, the video encoding and decoding method provided by the embodiment of the present application will be described in detail through multiple embodiments. Figure 1 In the illustrated implementation environment, any of the encoding methods described below may be executed by the encoder 100 in the source device 10 . Any of the decoding methods described below may be executed by the decoder 200 in the destination device 20 .

[0089] Figure 2 This is a flowchart of a video encoding method provided by an embodiment of the present application. Figure 2 , the method comprises the following steps.

[0090] Step 201: Determine a motion vector of a current block in a current video frame, where the motion vector of the current block indicates a positional relationship between the current block and a reference block corresponding to the current block. The current video frame is a video frame to be encoded in a video signal.

[0091] Since the current video frame may be a video frame using inter-frame prediction coding or a video frame using intra-frame block copy coding, when the current video frame uses inter-frame prediction coding or intra-frame block copy coding, the reference block corresponding to the current block may be determined according to a relevant algorithm, and the relative displacement between the current block and the reference block may be determined as a motion vector of the current block.

[0092] It should be noted that, when the current video frame adopts inter-frame prediction coding, there are multiple methods for expressing the motion vector of the current block. In some embodiments, the motion vector of the current block can be expressed as: a two-dimensional vector obtained by subtracting the coordinates of the reference block in the reference video frame from the coordinates of the current block in the current video frame. In other embodiments, the motion vector of the current block can also be expressed as: a two-dimensional vector obtained by subtracting the coordinates of the current block in the current video frame from the coordinates of the reference block in the reference video frame. For ease of description, the motion vector of the current block is taken as an example to be introduced as a two-dimensional vector obtained by subtracting the coordinates of the current block in the current video frame from the coordinates of the reference block corresponding to the current block in the reference video frame. In this case, the motion vector of any block can be expressed as: a two-dimensional vector obtained by subtracting the coordinates of the block in the video frame where the block is located from the coordinates of the reference block corresponding to the block in the reference video frame.

[0093] For example, the coordinates of the current block in the current video frame are (1, 2), and the coordinates of the reference block corresponding to the current block in the reference video frame are (1, 1), then the motion vector of the current block can be expressed as (0, -1).

[0094] In some embodiments, the current video frame may also be a video frame using ordinary intra-frame prediction coding. In this case, it is necessary to determine the prediction mode of the current block in the current video frame. For example, the encoder can determine the prediction mode of the current block according to a relevant algorithm, and then determine the prediction block corresponding to the current block based on the prediction mode of the current block.

[0095] Step 202: Determine a global motion vector of the current video frame, where the global motion vector of the current video frame indicates a jitter condition of the current video frame relative to a static video frame in a video signal, where the static video frame is a video frame where no jitter occurs.

[0096] Since the global motion vector can reflect the jitter of the entire video frame, the global motion vector can also be called a frame-level motion vector.

[0097] Similar to the above, the current video frame may be a video frame using inter-frame prediction coding, a video frame using intra-frame block copy coding, or a video frame using ordinary intra-frame prediction coding. When the coding methods used by the current video frame are different, the methods for determining the global motion vector of the current video frame are different. They will be introduced below.

[0098] In the case where the current video frame adopts inter-frame prediction coding, there are multiple implementation methods for determining the global motion vector of the current video frame, and two of the implementation methods are introduced below.

[0099] In a first implementation, a reference global motion vector is determined, where the reference global motion vector indicates the jitter of the current video frame relative to a reference video frame corresponding to the current video frame, and based on the reference global motion vector, a global motion vector of the current video frame is determined.

[0100] Since the position of the static object in the current video frame changes relative to the position of the static object in the reference video frame, it can indicate the jitter of the current video frame relative to the reference video frame. In some embodiments, there are multiple methods for representing the change of the position of the static object in the current video frame relative to the position of the static object in the reference video frame, that is, there are multiple methods for representing the reference global motion vector of the current video frame. In some embodiments, the reference global motion vector can be represented as: a two-dimensional vector obtained by subtracting the coordinates of the static object in the current video frame from the coordinates of the static object in the reference video frame. In other embodiments, the reference global motion vector can be represented as: a two-dimensional vector obtained by subtracting the coordinates of the static object in the reference video frame from the coordinates of the static object in the current video frame. For ease of description, the reference motion vector of the current video frame is taken as an example to be introduced as a two-dimensional vector obtained by subtracting the coordinates of the static object in the reference video frame from the coordinates of the static object in the current video frame.

[0101] For example, the coordinates of the static object in the current video frame are (1, 2), and the coordinates of the static object in the reference video frame are (1, 1), then the reference motion vector of the current video frame can be expressed as (0, -1).

[0102] Based on the above description, for any block in the current video frame, the motion vector of the block can be expressed as a two-dimensional vector obtained by subtracting the coordinates of the block in the current video frame from the coordinates of the reference block corresponding to the block in the reference video frame, and the global motion vector of the current video frame is a two-dimensional vector obtained by subtracting the coordinates of the static object in the current video frame from the coordinates of the static object in the reference video frame. If the block is a block where a static object is located in the current video frame, the motion vector of the block can indicate the jitter of the current video frame relative to the reference video frame. Based on this, in some embodiments, the reference global motion vector can be determined based on the motion vectors of each block with a static object in the current video frame.

[0103] Optionally, the encoder may determine an image region where the static object is located, and determine each encoding block in the image region where the static object is located as each block having the static object in the current video frame.

[0104] In some embodiments, the current video frame may be input into a static object recognition model to obtain a static object output by the static object recognition model and an image region where the static object is located. In other embodiments, static object region information may be obtained, the static object region information indicating the actual position of the static object in the current video frame, and the image region where the static object is located is determined based on the static object region information.

[0105] Optionally, the static object region information may be obtained by identifying a historical video frame using a static object recognition model. In this case, the historical video frame is a video frame encoded before the current video frame. Of course, in practical applications, the static object region information may also be set in advance, which is not limited in the embodiments of the present application.

[0106] It should be noted that the above-mentioned static objects can be objects such as trees, roads, buildings, etc. that do not move on their own. The static object area information can be the coordinates of the pixel points, or other information that can indicate the position of the pixel points. For example, the static object area information can be (10, 20), (10, 21), (11, 20) and (11, 21), that is, the area composed of four pixels with coordinates of (10, 20), (10, 21), (11, 20) and (11, 21) is the image area where the static object is located. For another example, the static object area information can be all the pixels with a horizontal coordinate of 5 in the current video frame. In this case, the area composed of all the pixels with a horizontal coordinate of 5 in the current video frame is the image area where the static object is located.

[0107] In summary, the current video frame can be directly identified by the static object recognition model to obtain the image area where the static object is located in the current video frame, or the image area where the static object is located in the historical video frame can be used, that is, the image area where the static object is located in the historical video frame is determined as the image area where the static object is located in the current video frame, or the image area where the static object is located in the current video frame can be set in advance, and then the coding blocks in the image area where the static object is located are determined as the blocks with static objects in the current video frame. Of course, in actual applications, the blocks with static objects in the current video frame can also be determined by other methods, which are not limited by the comparison of the embodiments of the present application.

[0108] The implementation process of determining the reference global motion vector based on the motion vectors of each block with static objects in the current video frame includes: determining at least one group of motion vectors for the motion vectors of each block with static objects in the current video frame according to the target window size, the target window shape and the target step size, each group of motion vectors includes at least one motion vector, and the at least one motion vector belongs to the same window, performing initial filtering on the at least one group of motion vectors to obtain first motion vectors corresponding to the at least one group of motion vectors, and determining the reference global motion vector based on the at least one first motion vector.

[0109] It should be noted that the target window size, the shape of the target window and the target step length are set in advance, and can be adjusted according to different requirements in different situations. For example, the target step length can include the step length in the horizontal direction and the step length in the vertical direction, and the step length in the horizontal direction and the step length in the vertical direction can both be set to 3. The target window can be a 3×3 rectangular window, that is, the shape of the target window is rectangular and the size is 3×3. In this case, there are 9 blocks in the target window, and the motion vectors of these 9 blocks are a group of motion vectors. When the target window slides, it can move 3 pixels in the horizontal direction at a time. If the target window has slid to the boundary of the current video frame, it can move 3 pixels in the vertical direction, and then continue to move in the horizontal direction according to the step length in the horizontal direction. Of course, in actual applications, the shape of the target window can also be linear, circular, cross-shaped, annular, etc., and the sliding mode of the target window can also be other forms besides the above examples, which are not limited in the embodiments of the present application.

[0110] For one group of motion vectors in the at least one group of motion vectors, the median of the group of motion vectors can be used as the first motion vector corresponding to the group of motion vectors. In the same way, each group of motion vectors in the at least one group of motion vectors is processed to obtain the first motion vectors corresponding to the at least one group of motion vectors. Of course, in practical applications, for one group of motion vectors in the at least one group of motion vectors, the average value, minimum value or maximum value of the group of motion vectors can also be used as the first motion vector corresponding to the group of motion vectors, and the embodiments of the present application do not limit this.

[0111] In some embodiments, the median of the at least one first motion vector may be determined as the reference global motion vector. Of course, the average value, minimum value or maximum value of the at least one first motion vector may also be used as the reference global motion vector, which is not limited in the embodiments of the present application.

[0112] In practical applications, there is no need to perform initial filtering, and the median, average, minimum or maximum value of the motion vectors of each block with static objects in the current video frame can be directly used as a reference global motion vector. This embodiment of the present application does not limit this.

[0113] The above is to determine the reference global motion vector based on the motion vectors of each block with static objects in the current video frame. Of course, it is also possible to determine the reference global motion vector directly based on the motion vectors of all blocks in the current video frame without determining each block with static objects in the current video frame. The implementation process of determining the reference global motion vector based on the motion vectors of all blocks in the current video frame is similar to the implementation process of determining the reference global motion vector based on the motion vectors of each block with static objects in the current video frame mentioned above. The detailed implementation process refers to the relevant content in the above text and will not be repeated here.

[0114] The implementation process of determining the global motion vector of the current video frame based on the reference global motion vector includes: determining the global motion vector of the reference video frame, and taking the sum of the global motion vector of the reference video frame and the reference motion vector as the global motion vector of the current video frame.

[0115] In some embodiments, the encoder stores the global motion vector of the reference video frame. In this case, the global motion vector of the reference video frame corresponding to the previous video frame can be directly determined. Of course, in practical applications, the global motion vector of the reference video frame can also be determined by other methods, which is not limited in the embodiments of the present application.

[0116] It should be noted that, in practical applications, if the reference video frame has no global motion vector, or the global motion vector corresponding to the reference video frame is zero, the reference motion vector can be directly used as the global motion vector of the current video frame.

[0117] The second implementation method is that if the acceleration vector of the previous video frame is stored, the jitter of the current video frame relative to the static video frame is predicted based on the acceleration vector of the previous video frame and the global motion vector of the previous video frame to obtain a first predicted global motion vector. The previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the acceleration vector indicates the jitter acceleration of the previous video frame. Based on the first predicted global motion vector, the global motion vector of the current video frame is determined.

[0118] In some embodiments, the encoder stores an acceleration vector of a previous video frame. In this case, the sum of the acceleration vector of the previous video frame and the global motion vector of the previous video frame may be used as the first predicted global motion vector.

[0119] Since the jitter acceleration of the current video frame may still be the jitter acceleration of the previous frame, the embodiment of the present application can predict the global motion vector of the current video frame based on the global motion vector of the previous frame and the acceleration vector of the previous video frame, that is, the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame is used as the first predicted global motion vector, and then the global motion vector of the current video frame is determined based on the first predicted global motion vector. This can reduce the amount of calculation and effectively improve the efficiency of determining the global motion vector.

[0120] As an example, the first predicted global motion vector may be directly used as the global motion vector of the current frame.

[0121] For example, see Figure 3 , if the current video frame is the n+1th frame in the video signal, the previous frame is the nth frame, the acceleration vector of the nth frame is gmva, and the global motion vector of the nth frame is gmv', then the global motion vector of the n+1th frame is gmv=gmv'+gmva.

[0122] As another example, the jitter of the current video frame relative to the static video frame can also be predicted based on the motion vectors of some blocks with static objects in the current video frame to obtain a second predicted global motion vector. In this case, the implementation process of determining the global motion vector of the current video frame based on the first predicted global motion vector includes: if the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, then determining the reference global motion vector of the current video frame, the reference global motion vector of the current video frame indicating the jitter of the current video frame relative to the reference video frame corresponding to the current video frame, and determining the global motion vector of the current video frame based on the reference global motion vector of the current video frame.

[0123] Based on the motion vectors of some blocks with static objects in the current video frame, the jitter of the current video frame relative to the static video frame is predicted, and the implementation process of obtaining the second predicted global motion vector includes: based on the motion vectors of some blocks with static objects in the current video frame, a predicted reference global motion vector is determined, and based on the predicted reference global motion vector, a second predicted global motion vector is determined.

[0124] Optionally, the encoder may determine an image region where the static object is located, and based on the image region where the static object is located, determine motion vectors of blocks in the current video frame that partially have the static object.

[0125] In some embodiments, coding blocks having a number threshold value can be randomly selected from coding blocks in the image region where the static object is located as blocks that partially have static objects in the current video frame. In other embodiments, a target image region having a target size can be randomly selected from the image region where the static object is located, and coding blocks in the target image region can be used as blocks that partially have static objects in the current video frame. Of course, in practical applications, blocks that partially have static objects in the current video frame can also be determined by other methods, which is not limited in the embodiments of the present application.

[0126] Since the number of motion vectors of some blocks with static objects in the current video frame is small, the median, average, minimum or maximum value of the motion vectors of some blocks with static objects in the current video frame can be directly used as the prediction reference global motion vector.

[0127] The number threshold is set in advance, for example, the number threshold can be set to 9, and can be adjusted according to different requirements in different situations. The target size is set in advance, for example, the target size can be set to 9×9, that is, the target image area includes 9×9=81 pixels, and can be adjusted according to different requirements in different situations.

[0128] The implementation process of determining the second predicted global motion vector based on the predicted reference global motion vector is similar to the implementation process of determining the global motion vector of the current video frame based on the reference global motion vector. For the detailed implementation process, please refer to the relevant content above, which will not be repeated here.

[0129] Since the second predicted global motion vector is determined based on the motion vector of some blocks with static objects in the current video frame, the second predicted global motion vector can represent the real global motion vector of the current video frame. If the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, it means that the jitter acceleration of the current video frame is different from the jitter acceleration of the previous frame, and the first predicted global motion vector determined by the acceleration vector of the previous video frame is quite different from the real global motion vector of the current video frame. Therefore, it is necessary to determine the reference global motion vector, and determine the global motion vector of the current video frame based on the reference global motion vector of the current video frame. Of course, in actual applications, if the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, the second predicted global motion vector can also be directly used as the global motion vector of the current frame, and the embodiment of the present application does not limit this.

[0130] In some embodiments, if the difference between the first predicted global motion vector and the second predicted global motion vector is less than a difference threshold, the first predicted global motion vector is used as the global motion vector of the current frame.

[0131] If the difference between the first predicted global motion vector and the second predicted global motion vector is less than the difference threshold, it means that the jitter acceleration of the current video frame is close to the jitter acceleration of the previous frame, and the first predicted global motion vector determined by the acceleration vector of the previous video frame is slightly different from the actual global motion vector of the current video frame. Therefore, the first predicted global motion vector can be directly used as the global motion vector of the current frame.

[0132] When the difference between the first predicted global motion vector and the second predicted global motion vector is less than the difference threshold, the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame is used as the global motion vector of the current video frame. In this way, the amount of calculation can be reduced, the efficiency of determining the global motion vector can be effectively improved, and the accuracy of the global motion vector of the current video frame that is finally determined can be guaranteed.

[0133] As an example, see Figure 4 , each block with static objects in the current video frame is located at Figure 4 In area A, based on the motion vectors of each block in area A, a second predicted global motion vector, i.e., gmv2, is determined. The first predicted global motion vector is gmv1. If the difference between gmv2 and gmv1 is less than the difference threshold, the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame is used as the global motion vector of the current video frame. Otherwise, a reference global motion vector of the current video frame is determined, and based on the reference global motion vector, the global motion vector of the current video frame is determined.

[0134] The difference threshold is set in advance and can be adjusted according to different requirements in different situations.

[0135] In some embodiments, if the acceleration vector of the previous video frame is not stored, a reference global motion vector of the current video frame is determined, and based on the reference global motion vector, a global motion vector of the current video frame is determined.

[0136] When the current video frame adopts ordinary intra-frame prediction coding or intra-frame block copy prediction coding, the implementation process of determining the global motion vector of the current video frame includes: determining the global motion vector of the current video frame based on the global motion vector of the previous video frame and the global motion vector of the next video frame, the previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the next video frame is the next video frame adjacent to the current video frame in the video signal.

[0137] In some embodiments, the average value of the global motion vector of the previous video frame and the global motion vector of the next video frame can be used as the global motion vector of the current video frame. Of course, in practical applications, the maximum or minimum value of the global motion vector of the previous video frame and the global motion vector of the next video frame can also be used as the global motion vector of the current video frame, and this embodiment of the application does not limit this.

[0138] After determining the global motion vector of the current video frame, the acceleration vector of the current video frame can also be determined and stored based on the global motion vector of the current video frame and the global motion vector of the previous video frame. For example, the acceleration vector of the current video frame is obtained by subtracting the global motion vector of the previous video frame from the global motion vector of the current video frame.

[0139] That is to say, after each global motion vector of a video frame is determined, the acceleration vector of the current video frame can be determined, thereby updating the acceleration vector and ensuring the real-time performance of the acceleration vector, so as to predict the global motion vector of subsequent video frames.

[0140] Since prediction cannot be made when the acceleration vector of the previous video frame is not stored, computing resources are needed to determine the reference global motion vector, and then the global motion vector of the video frame can be determined based on the reference global motion vector. In addition, when the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, it means that the jitter acceleration of the current video frame is different from the jitter acceleration of the previous frame. At this time, the acceleration vector of the previous frame cannot accurately describe the jitter trend of subsequent video frames. Therefore, in practical applications, the above-mentioned operation of determining and storing the acceleration vector of the current video frame based on the global motion vector of the current video frame and the global motion vector of the previous video frame can be performed when the acceleration vector of the previous video frame is not stored, or when the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold. In this way, not only can the acceleration vector be updated to ensure the accuracy of the global motion vector of subsequent video frames, but also the amount of calculation can be effectively reduced.

[0141] Step 203: Encode the current block based on the motion vector of the current block and the global motion vector of the current video frame.

[0142] In the case where the current video frame adopts inter-frame prediction coding or intra-frame block copy coding, the global motion vector of the current video frame, and the motion vector and prediction residual of the current block can be encoded into the bitstream, and the prediction residual indicates the difference between the current block and the reference block corresponding to the current block. In practical applications, the opposite number of the global motion vector of the current video frame, the motion vector and prediction residual of the current block can also be encoded into the bitstream, which is not limited in the embodiments of the present application.

[0143] As an example, the coding syntax elements for encoding the global motion vector of the current video frame into the bitstream are shown in Table 1 below.

[0144] Table 1

[0145] Encoding parameter definition Descriptors gmv_extension(){ gmv_enable u(1) gmv u(v) gmv[i]->dx f(8) gmv[i]->dy f(8) }

[0146] As shown in Table 1, gmv_enable is a global motion vector enable flag, which is used to indicate whether the global motion vector is enabled for the current video frame. u(1) indicates that the global motion vector enable flag is a one-bit unsigned integer. When the global motion vector enable flag is 0, it indicates that the global motion vector is not enabled for the current video frame. When the global motion vector enable flag is 1, it indicates that the global motion vector is enabled for the current video frame. gmv is a global motion vector information structure data, which indicates the global motion vector of the current video frame. gmv[i]->dx is a global motion vector structure, which indicates the component of the global motion vector of the current video frame in the x direction. f(8) indicates that the component of the global motion vector in the x direction is an 8-bit signed floating point number. gmv[i]->dy is a global motion vector structure, which indicates the component of the global motion vector of the current video frame in the y direction. f(8) indicates that the component of the global motion vector in the y direction is an 8-bit signed floating point number.

[0147] In some embodiments, the code stream includes one or more network abstract layer (NAL) units, each NAL unit includes a NAL unit header and NAL unit body data, wherein the NAL unit header consists of a start code and a NAL unit type, and the NAL unit body data is usually raw byte sequence payload (RBS) data. As an example, the type of NAL unit is shown in Table 2 below:

[0148] Table 2

[0149]

[0150]

[0151] As shown in Table 2, there is no corresponding Nal unit type when the Nal unit type code is 0, 13-23 and 24-31. When the Nal unit type code is 1, it indicates that the NAL unit type is a non-IDR image coding slice. When the Nal unit type code is 2, it indicates that the NAL unit type is an image coding slice data A partition. When the Nal unit type code is 3, it indicates that the NAL unit type is an image coding slice data B partition. When the Nal unit type code is 4, it indicates that the NAL unit type is an image coding slice data C partition. When the Nal unit type code is 5, it indicates that the NAL unit type is an IDR image coding slice. When the Nal unit type code is 6, it indicates that the NAL unit type is SEI. When the Nal unit type code is 7, it indicates that the NAL unit type is SPS. When the Nal unit type code is 8, it indicates that the NAL unit type is PPS. When the Nal unit type code is 9, it indicates that the NAL unit type is a delimiter. When the NAL unit type code is 10, it indicates that the NAL unit type is the end of the sequence. When the NAL unit type code is 11, it indicates that the NAL unit type is the end of the code stream. When the NAL unit type code is 12, it indicates that the NAL unit type is padding.

[0152] In some embodiments, please refer to Figure 5 , the global motion vector enable flag can be incorporated into the NAL unit body data whose Nal unit type is PPS, and the component of the global motion vector in the x direction and the component of the global motion vector in the y direction can be incorporated into the NAL unit body data whose Nal unit type is SEI. In other embodiments, the component of the global motion vector in the x direction and the component of the global motion vector in the y direction can also be incorporated into the NAL unit body data whose Nal unit type code is 0, 13-23 or 24-31.

[0153] In practical applications, for a current video frame encoded by inter-frame prediction, there are multiple implementation methods for encoding the current block, and two of the implementation methods are introduced below.

[0154] The first implementation method is to perform jitter compensation on the motion vector and prediction residual of the current block based on the global motion vector of the current video frame, and obtain the corrected motion vector and corrected prediction residual of the current block. The prediction residual indicates the difference between the current block and the reference block corresponding to the current block, and the corrected motion vector and corrected prediction residual of the current block are encoded into the bitstream.

[0155] In some embodiments, jitter compensation can be performed on the motion vector of the current block based on the global motion vector of the current video frame to obtain a corrected motion vector of the current block. Based on the corrected motion vector of the current block, the reference block corresponding to the current block is re-determined, and the difference between the current block and the re-determined reference block is determined to obtain a corrected prediction residual of the current block.

[0156] Optionally, the vector obtained by subtracting the global motion vector of the current video frame from the motion vector of the current block can be used as the corrected motion vector of the current block, and then based on the corrected motion vector of the current block and in accordance with a relevant algorithm, the reference block corresponding to the current block is re-determined and the difference between the current block and the re-determined reference block is determined to obtain a corrected prediction residual of the current block.

[0157] In a second implementation, the current block is encoded based on the global motion vector of the reference video frame, the reference global motion vector of the current frame, and the motion vector of the current block.

[0158] There are multiple implementation methods for encoding the current block based on the global motion vector of the reference video frame, the reference global motion vector of the current frame and the motion vector of the current block, and two of them are introduced below.

[0159] Method 1: The motion vector of the current block, the global motion vector of the reference video frame, the reference global motion vector of the current video frame, and the prediction residual of the current block are encoded into the bitstream.

[0160] Method 2, based on the global motion vector of the reference video frame, the motion vector of the current block is jitter compensated to obtain the corrected motion vector of the current block, and the corrected motion vector of the current block, the reference global motion vector of the current video frame and the predicted residual of the current block are encoded into the bitstream.

[0161] Since the reference video frame may have jitter relative to the static video frame, there is no jitter after the reference video frame is reconstructed through the embodiment of the present application. However, this will also cause the motion vector of the current block in the current video frame to be unable to indicate the positional relationship between the current block and the original reference block. Therefore, it is necessary to perform jitter compensation on the motion vector of the current block so that the corrected motion vector can re-indicate the positional relationship between the current block and the original reference block.

[0162] Optionally, a vector obtained by subtracting a global motion vector of a reference video frame from a motion vector of the current block may be used as a corrected motion vector of the current block.

[0163] In the case where the current video frame adopts ordinary intra-frame prediction coding, the indication information of the prediction mode of the current block in the current video frame, the global motion vector of the current video frame and the prediction residual of the current block can be encoded into the bitstream, and the prediction residual indicates the difference between the current block and the prediction block corresponding to the current block, and the prediction block is determined based on the prediction mode of the current block. In practical applications, the opposite number of the global motion vector of the current video frame, the indication information of the prediction mode of the current block and the prediction residual of the current block can also be encoded into the bitstream, and the embodiments of the present application do not limit this.

[0164] Next, we will combine Figure 6 A video encoding method provided in an embodiment of the present application is introduced again.

[0165] Based on the motion vectors of each block with static objects in the current video frame, 16 groups of motion vectors are determined ( Figure 6 The 16 groups of motion vectors are schematically represented by 16 squares in the figure), and the 16 groups of motion vectors are initially filtered to obtain the first motion vectors ( Figure 6 The arrow in each square schematically indicates the first motion vector corresponding to the group of motion vectors), a reference global motion vector is determined based on the 16 first motion vectors, and the global motion vector of the current video frame is determined based on the reference global motion vector, and then the opposite number of the global motion vector of the current video frame, the motion vector of the current block and the prediction residual are encoded into the bitstream.

[0166] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static frame, the present application can determine the global motion vector of the current video frame, and encode the current block based on the global motion vector of the current video frame and the motion vector of the current block, so that the compensation for video jitter can be achieved without increasing the hardware cost. Since the jitter acceleration of the current video frame may still be the jitter acceleration of the previous frame, the embodiment of the present application can predict the global motion vector of the current video frame according to the global motion vector of the previous frame and the acceleration vector of the previous video frame, so that the amount of calculation can be reduced and the efficiency of determining the global motion vector can be effectively improved. In addition, the embodiment of the present application can also be used when the difference between the first predicted global motion vector and the second predicted global motion vector is less than the difference threshold, and the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame is used as the global motion vector of the current video frame, so that the accuracy of the global motion vector of the current video frame finally determined can be guaranteed while reducing the amount of calculation and effectively improving the efficiency of determining the global motion vector.

[0167] The embodiment of the present application can also determine the acceleration vector of the current video frame once after the global motion vector of a video frame is determined, thereby updating the acceleration vector and ensuring the real-time performance of the acceleration vector to facilitate the prediction of the global motion vector of subsequent video frames. Since prediction cannot be made when the acceleration vector of the previous video frame is not stored, computing resources are needed to determine the reference global motion vector, and then the global motion vector of the video frame can be determined based on the reference global motion vector. In addition, when the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, it means that the jitter acceleration of the current video frame is different from the jitter acceleration of the previous frame. At this time, the acceleration vector of the previous frame cannot accurately describe the jitter trend of subsequent video frames. Therefore, the embodiment of the present application can also perform the above-mentioned action of determining and storing the acceleration vector of the current video frame based on the global motion vector of the current video frame and the global motion vector of the previous video frame when the acceleration vector of the previous video frame is not stored, or when the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold. In this way, not only can the acceleration vector be updated to ensure the accuracy of the global motion vector of subsequent video frames, but also the amount of calculation can be effectively reduced.

[0168] Figure 7 This is a flowchart of the first decoding method provided in the embodiment of the present application. Please refer to Figure 7 The first decoding method corresponds to the encoding method in which the global motion vector of the current video frame, the motion vector of the current block and the prediction residual are encoded into the bitstream. The decoding method includes the following steps.

[0169] Step 701: Parse the code stream to obtain the global motion vector of the current video frame, and the motion vector and prediction residual of the current block.

[0170] Among them, the current video frame is a video frame currently to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector of the current block indicates the positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block corresponding to the current block.

[0171] Step 702: Reconstruct the current block based on the global motion vector of the current video frame, and the motion vector and prediction residual of the current block, wherein the reconstructed current block does not have jitter.

[0172] In some embodiments, an initial current block is determined based on a motion vector and a prediction residual of the current block, and then jitter compensation is performed on the initial current block based on a global motion vector of the current video frame to obtain a reconstructed current block.

[0173] For example, a reference block corresponding to the current block is determined based on a motion vector of the current block, and then an initial current block is generated according to the reference block corresponding to the current block and a prediction residual.

[0174] Optionally, the position of the initial current block in the current video frame may be moved in the opposite direction and distance indicated by the global motion vector of the current video frame to obtain a reconstructed current block.

[0175] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, during the decoding process of the current block, the jitter of the current block can be compensated according to the global motion vector of the current video frame, so that the current block after compensation has no jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0176] Figure 8 This is a flowchart of the second decoding method provided in the embodiment of the present application. Please refer to Figure 8 The second decoding method corresponds to the encoding method in which the corrected motion vector and the corrected prediction residual of the current block are encoded into the bitstream, and the decoding method includes the following steps.

[0177] Step 801: parse the code stream to obtain the corrected motion vector and corrected prediction residual of the current block.

[0178] Among them, the corrected motion vector and the corrected prediction residual are obtained by jitter compensation of the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, the current video frame is a video frame currently to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector of the current block indicates the positional relationship between the current block and the reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block corresponding to the current block.

[0179] Step 802: Reconstruct the current block based on the modified motion vector and the modified prediction residual, wherein the reconstructed current block does not have jitter.

[0180] In some embodiments, a reference block corresponding to the current block may be determined based on the modified motion vector, and then the current block may be generated according to the reference block corresponding to the current block and the modified prediction residual.

[0181] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, and the corrected motion vector and the corrected prediction residual are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, during the decoding process of the current block, the current block reconstructed according to the corrected motion vector and the corrected prediction residual of the current block does not have jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0182] Next, the third decoding method provided by the embodiment of the present application is introduced. The third decoding method corresponds to the method 1 of encoding the current block based on the global motion vector of the reference video frame and the reference global motion vector of the current frame in the above encoding method, and the decoding method includes the following steps.

[0183] Step 901: Parse the code stream to obtain the global motion vector of the reference video frame corresponding to the current video frame, the reference motion vector of the current video frame, and the motion vector and prediction residual of the current block.

[0184] Among them, the current video frame is a video frame currently to be decoded in the video signal, the global motion vector of the reference video frame indicates the jitter of the reference video frame relative to the static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the reference motion vector of the current video frame indicates the jitter of the current video frame relative to the reference video frame corresponding to the current video frame, the motion vector of the current block indicates the positional relationship between the current block and the reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block corresponding to the current block.

[0185] Step 902: Reconstruct the current block based on the global motion vector of the reference video frame, the reference motion vector of the current video frame, and the motion vector and prediction residual of the current block, wherein the reconstructed current block has no jitter.

[0186] In some embodiments, the motion vector of the current block can be jitter compensated based on the global motion vector of the reference video frame to obtain a corrected motion vector of the current block, and the initial current block can be determined based on the corrected motion vector and the prediction residual of the current block. Then, the initial current block can be jitter compensated based on the reference motion vector of the current video frame to obtain a reconstructed current block.

[0187] Optionally, the position of the initial current block in the current video frame may be moved in the opposite direction and distance indicated by the reference motion vector of the current video frame to obtain a reconstructed current block.

[0188] Since the global motion vector of the reference video frame indicates the jitter of the reference video frame relative to the static video frame in the video signal, and the reference motion vector of the current video frame indicates the jitter of the current video frame relative to the reference video frame corresponding to the current video frame, the global motion vector of the reference video frame and the reference motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal. Therefore, in the process of decoding the current block, the jitter of the current block can be compensated according to the global motion vector based on the reference video frame and the reference motion vector of the current video frame, so that the current block after compensation does not have jitter, that is, the compensation of video jitter is achieved without increasing the hardware cost.

[0189] Next, the fourth decoding method provided by the embodiment of the present application is introduced. The fourth decoding method corresponds to the method 2 of encoding the current block based on the global motion vector of the reference video frame and the reference global motion vector of the current frame in the above encoding method, and the decoding method includes the following steps.

[0190] Step 1001: Parse the bitstream to obtain a modified motion vector of the current block, a reference global motion vector of the current video frame, and a prediction residual.

[0191] Among them, the corrected motion vector of the current block is obtained by jitter compensation of the motion vector of the current block through the global motion vector of the reference video frame corresponding to the current video frame. The current video frame is a video frame currently to be decoded in the video signal. The global motion vector of the reference video frame indicates the jitter of the reference video frame relative to the static video frame in the video signal. The static video frame is a video frame where the video screen does not jitter. The reference motion vector of the current video frame indicates the jitter of the current video frame relative to the reference video frame corresponding to the current video frame. The motion vector of the current block indicates the positional relationship between the current block and the reference block corresponding to the current block. The prediction residual indicates the difference between the current block and the reference block corresponding to the current block.

[0192] Step 1002: Reconstruct the current block based on the corrected motion vector of the current block, the reference global motion vector of the current video frame and the prediction residual, wherein the reconstructed current block does not have jitter.

[0193] In some embodiments, an initial current block may be determined based on a modified motion vector and a prediction residual of the current block, and then jitter compensation may be performed on the initial current block based on a reference motion vector of the current video frame to obtain a reconstructed current block.

[0194] Optionally, the position of the initial current block in the current video frame may be moved in the opposite direction and distance indicated by the reference motion vector of the current video frame to obtain a reconstructed current block.

[0195] Since the global motion vector of the reference video frame indicates the jitter of the reference video frame relative to the still video frame in the video signal, and the corrected motion vector of the current block is obtained by performing jitter compensation on the motion vector of the current block through the global motion vector of the reference video frame, and the reference motion vector of the current video frame can indicate the jitter of the current video frame relative to the still video frame in the video signal, therefore, in the process of decoding the current block, the jitter of the current block can be compensated according to the corrected motion vector of the current block, the reference global motion vector of the current video frame and the prediction residual, so that the current block after compensation does not have jitter, that is, the compensation of video jitter is achieved without increasing the hardware cost.

[0196] Next, the fifth decoding method provided by the embodiment of the present application is introduced. The fifth decoding method corresponds to the way in which the prediction mode of the current block in the current video frame, the global motion vector of the current video frame and the prediction residual are encoded into the bitstream in the above encoding method, and the decoding method includes the following steps.

[0197] Step 1101: Parse the code stream to obtain indication information of the prediction mode of the current block in the current video frame, the global motion vector and the prediction residual of the current video frame.

[0198] Among them, the current video frame is a video frame currently to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the prediction mode of the current block is used to determine the prediction block corresponding to the current block, and the prediction residual indicates the difference between the current block and the prediction block corresponding to the current block.

[0199] Step 1102: reconstruct the current block based on the indication information of the prediction mode of the current block in the current video frame, the global motion vector and the prediction residual of the current video frame, wherein the reconstructed current block does not have jitter.

[0200] In some embodiments, an initial current block is determined based on a prediction mode and a prediction residual of the current block, and then jitter compensation is performed on the initial current block based on a global motion vector of the current video frame to obtain a reconstructed current block.

[0201] For example, a prediction block corresponding to the current block may be generated based on the prediction mode of the current block, and then an initial current block may be generated based on the prediction block corresponding to the current block and a prediction residual.

[0202] Optionally, the position of the initial current block in the current video frame may be moved in the opposite direction and distance indicated by the global motion vector of the current video frame to obtain a reconstructed current block.

[0203] Since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, during the decoding process of the current block, the jitter of the current block can be compensated according to the global motion vector of the current video frame, so that the current block after compensation has no jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0204] Fig. 9 is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. The video encoding device can be implemented as part or all of an encoder by software, hardware, or a combination of both. The encoding device can be Figure 1 The encoder 100 in FIG. Fig. 9 The device includes: a first determination module 901, a second determination module 902 and an encoding module 903.

[0205] The first determination module 901 is used to determine the motion vector of the current block in the current video frame, where the motion vector indicates the positional relationship between the current block and the reference block corresponding to the current block. The current video frame is a video frame to be encoded in the video signal. The detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.

[0206] The second determination module 902 is used to determine the global motion vector of the current video frame, where the global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal, where the static video frame is a video frame where the video screen does not jitter. The detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.

[0207] The encoding module 903 is used to encode the current block based on the motion vector of the current block and the global motion vector of the current video frame. The detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.

[0208] Optionally, the current video frame is a video frame encoded using inter-frame prediction;

[0209] The second determining module 902 is specifically used for:

[0210] Determine a reference global motion vector, where the reference global motion vector indicates a jitter condition of the current video frame relative to a reference video frame corresponding to the current video frame;

[0211] Based on the reference global motion vector, a global motion vector of the current video frame is determined.

[0212] Optionally, the second determining module 902 is specifically configured to:

[0213] If the acceleration vector of the previous video frame is not stored, the step of determining a reference global motion vector is performed, the acceleration vector indicating the jitter acceleration of the previous video frame, the previous video frame being the previous video frame adjacent to the current video frame in the video signal.

[0214] Optionally, the current video frame is a video frame encoded using inter-frame prediction;

[0215] The second determining module 902 is specifically used for:

[0216] If an acceleration vector of a previous video frame is stored, based on the acceleration vector and the global motion vector of the previous video frame, a jitter condition of the current video frame relative to the static video frame is predicted to obtain a first predicted global motion vector, the previous video frame being a previous video frame adjacent to the current video frame in the video signal, and the acceleration vector indicating the jitter acceleration of the previous video frame;

[0217] Based on the first predicted global motion vector, a global motion vector of the current video frame is determined.

[0218] Optionally, the device further comprises:

[0219] A prediction module, configured to predict the jitter of the current video frame relative to the static video frame based on the motion vectors of some blocks with static objects in the current video frame, and obtain a second predicted global motion vector;

[0220] The second determining module 902 is specifically used for:

[0221] If the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, determining a reference global motion vector, the reference global motion vector indicating a jitter condition of the current video frame relative to a reference video frame corresponding to the current video frame;

[0222] Based on the reference global motion vector, a global motion vector of the current video frame is determined.

[0223] Optionally, the second determining module 902 is specifically configured to:

[0224] Based on the motion vectors of each block having a static object in the current video frame, a reference global motion vector is determined.

[0225] Optionally, after determining the global motion vector of the current video frame, the device further includes:

[0226] The third determination module is used to determine and store the acceleration vector of the current video frame based on the global motion vector of the current video frame and the global motion vector of the previous video frame, where the previous video frame is a previous video frame adjacent to the current video frame in the video signal.

[0227] Optionally, the current video frame is a video frame encoded using intra-frame block copying;

[0228] The second determining module 902 is specifically used for:

[0229] The global motion vector of the current video frame is determined based on the global motion vector of the previous video frame and the global motion vector of the next video frame, wherein the previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the next video frame is the next video frame adjacent to the current video frame in the video signal.

[0230] Optionally, the encoding module 903 is specifically used for:

[0231] The global motion vector of the current video frame, as well as the motion vector and prediction residual of the current block are encoded into the bitstream, where the prediction residual indicates the difference between the current block and the reference block.

[0232] Optionally, the current video frame is a video frame encoded using inter-frame prediction;

[0233] The encoding module 903 is specifically used for:

[0234] Based on the global motion vector of the current video frame, jitter compensation is performed on the motion vector and prediction residual of the current block to obtain a corrected motion vector and a corrected prediction residual of the current block, where the prediction residual indicates the difference between the current block and the reference block;

[0235] The modified motion vector and the modified prediction residual of the current block are encoded into the bitstream.

[0236] Optionally, the encoding module 903 is specifically used for:

[0237] Based on the global motion vector of the current video frame, jitter compensation is performed on the motion vector of the current block to obtain a corrected motion vector of the current block;

[0238] Re-determine the reference block corresponding to the current block based on the corrected motion vector of the current block;

[0239] The difference of the current block relative to the re-determined reference block is determined to obtain a modified prediction residual of the current block.

[0240] In the embodiment of the present application, since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static frame, the present application can determine the global motion vector of the current video frame, and encode the current block based on the global motion vector of the current video frame and the motion vector of the current block, so that the compensation for video jitter can be achieved without increasing the hardware cost. Since the jitter acceleration of the current video frame may still be the jitter acceleration of the previous frame, the embodiment of the present application can predict the global motion vector of the current video frame according to the global motion vector of the previous frame and the acceleration vector of the previous video frame, so that the amount of calculation can be reduced and the efficiency of determining the global motion vector can be effectively improved. In addition, the embodiment of the present application can also be used when the difference between the first predicted global motion vector and the second predicted global motion vector is less than the difference threshold, and the sum of the global motion vector of the previous frame and the acceleration vector of the previous video frame is used as the global motion vector of the current video frame, so that the accuracy of the global motion vector of the current video frame finally determined can be guaranteed while reducing the amount of calculation and effectively improving the efficiency of determining the global motion vector.

[0241] The embodiment of the present application can also determine the acceleration vector of the current video frame once after the global motion vector of a video frame is determined, thereby updating the acceleration vector and ensuring the real-time performance of the acceleration vector to facilitate the prediction of the global motion vector of subsequent video frames. Since prediction cannot be made when the acceleration vector of the previous video frame is not stored, computing resources are needed to determine the reference global motion vector, and then the global motion vector of the video frame can be determined based on the reference global motion vector. In addition, when the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold, it means that the jitter acceleration of the current video frame is different from the jitter acceleration of the previous frame. At this time, the acceleration vector of the previous frame cannot accurately describe the jitter trend of subsequent video frames. Therefore, the embodiment of the present application can also perform the above-mentioned action of determining and storing the acceleration vector of the current video frame based on the global motion vector of the current video frame and the global motion vector of the previous video frame when the acceleration vector of the previous video frame is not stored, or when the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to the difference threshold. In this way, not only can the acceleration vector be updated to ensure the accuracy of the global motion vector of subsequent video frames, but also the amount of calculation can be effectively reduced.

[0242] It should be noted that: when the video encoding device provided in the above embodiment performs video encoding, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the video encoding device provided in the above embodiment and the video encoding method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0243] Fig.10 is a schematic diagram of the structure of a video decoding device provided in an embodiment of the present application. The video decoding device can be implemented as part or all of a decoder by software, hardware, or a combination of both. The decoding device can be Figure 1 The decoder 200 is shown. Fig.10 The device includes: a parsing module 1001 and a reconstruction module 1002.

[0244] The parsing module 1001 is used to parse the code stream to obtain the global motion vector of the current video frame, and the motion vector and prediction residual of the current block. The current video frame is a video frame to be decoded in the video signal. The global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal. The static video frame is a video frame where the video screen does not jitter. The motion vector indicates the positional relationship between the current block and the reference block corresponding to the current block. The prediction residual indicates the difference between the current block and the reference block. The detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.

[0245] The reconstruction module 1002 is used to reconstruct the current block based on the global motion vector of the current video frame, the motion vector of the current block and the prediction residual, wherein the reconstructed current block does not have jitter. The detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.

[0246] Optionally, the reconstruction module 1002 is specifically configured to:

[0247] Determine an initial current block based on the motion vector and the prediction residual of the current block;

[0248] Based on the global motion vector of the current video frame, jitter compensation is performed on the initial current block to obtain a reconstructed current block.

[0249] In the embodiment of the present application, since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the static video frame in the video signal, during the decoding process of the current block, the jitter of the current block can be compensated according to the global motion vector of the current video frame, so that the current block after compensation has no jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0250] It should be noted that: when the video decoding device provided in the above embodiment performs video decoding, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the video decoding device provided in the above embodiment and the video decoding method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0251] Fig.11 is a schematic diagram of the structure of a video decoding device provided in an embodiment of the present application. The video decoding device can be implemented as part or all of a decoder by software, hardware, or a combination of both. The video decoding device can be Figure 1 The decoder 200 is shown. Fig.11 The device includes: a parsing module 1101 and a reconstruction module 1102.

[0252] The parsing module 1101 is used to parse the code stream to obtain the corrected motion vector and the corrected prediction residual of the current block, which are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, the current video frame is a video frame to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to the static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector indicates the positional relationship between the current block and the reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block. The detailed implementation process refers to the corresponding content in the above-mentioned embodiments, which will not be repeated here.

[0253] The reconstruction module 1102 is used to reconstruct the current block based on the modified motion vector and the modified prediction residual, wherein the reconstructed current block does not have jitter. The detailed implementation process refers to the corresponding content in the above embodiments, which will not be repeated here.

[0254] In an embodiment of the present application, since the global motion vector of the current video frame can indicate the jitter of the current video frame relative to the stationary video frame in the video signal, and the corrected motion vector and the corrected prediction residual are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block by the global motion vector of the current video frame, therefore, during the decoding process of the current block, the current block reconstructed according to the corrected motion vector and the corrected prediction residual of the current block does not have jitter, that is, compensation for video jitter is achieved without increasing hardware costs.

[0255] It should be noted that: when the video decoding device provided in the above embodiment performs video decoding, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the video decoding device provided in the above embodiment and the video decoding method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0256] An embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the steps of the video encoding method described in the above embodiment, or executes the steps of the video decoding method described in the above embodiment.

[0257] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the video encoding method described in the above embodiment, or the steps of the video decoding method described in the above embodiment. In other words, a computer program is provided, which, when executed on a computer, enables the computer to execute the steps of the video encoding method described in the above embodiment, or the steps of the video decoding method described in the above embodiment.

[0258] An embodiment of the present application also provides a video encoding and decoding system, which includes a video encoding device and a video decoding device. The video encoding device is used to implement the steps of the video encoding method described in the above embodiment, and the video decoding device is used to implement the steps of the video decoding method described in the above embodiment.

[0259] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0260] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0261] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the motion vector of the current block and the global motion vector of the current video frame involved in the embodiments of the present application are obtained with full authorization.

[0262] The above-mentioned embodiments are provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A video encoding method, characterized in that: The method comprises: Determine a motion vector of a current block in a current video frame, wherein the motion vector indicates a positional relationship between the current block and a reference block corresponding to the current block, wherein the current video frame is a video frame to be encoded in a video signal; Determine a global motion vector of the current video frame, wherein the global motion vector indicates a jitter condition of the current video frame relative to a static video frame in the video signal, wherein the static video frame is a video frame in which no jitter occurs in the video picture; The current block is encoded based on the motion vector of the current block and the global motion vector of the current video frame.

2. The method according to claim 1, characterized in that The current video frame is a video frame encoded using inter-frame prediction; The determining of the global motion vector of the current video frame comprises: Determine a reference global motion vector, where the reference global motion vector indicates a jitter condition of the current video frame relative to a reference video frame corresponding to the current video frame; Based on the reference global motion vector, a global motion vector of the current video frame is determined.

3. The method according to claim 2, characterized in that The method further comprises: If the acceleration vector of the previous video frame is not stored, the step of determining the reference global motion vector is performed, wherein the acceleration vector indicates the jitter acceleration of the previous video frame, and the previous video frame is a previous video frame adjacent to the current video frame in the video signal.

4. The method according to claim 1, characterized in that The current video frame is a video frame encoded using inter-frame prediction; The determining of the global motion vector of the current video frame comprises: If an acceleration vector of a previous video frame is stored, predicting the jitter of the current video frame relative to the static video frame based on the acceleration vector and the global motion vector of the previous video frame to obtain a first predicted global motion vector, wherein the previous video frame is a previous video frame adjacent to the current video frame in the video signal, and the acceleration vector indicates the jitter acceleration of the previous video frame; Based on the first predicted global motion vector, a global motion vector of the current video frame is determined.

5. The method according to claim 4, characterized in that The method further comprises: Based on the motion vectors of some blocks with static objects in the current video frame, predict the jitter of the current video frame relative to the static video frame to obtain a second predicted global motion vector; The determining, based on the first predicted global motion vector, the global motion vector of the current video frame comprises: If the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to a difference threshold, determining a reference global motion vector, the reference global motion vector indicating a jitter condition of the current video frame relative to a reference video frame corresponding to the current video frame; Based on the reference global motion vector, a global motion vector of the current video frame is determined.

6. The method according to claim 2, 3 or 5, characterized in that: The determining of the reference global motion vector comprises: The reference global motion vector is determined based on the motion vectors of each block having a static object in the current video frame.

7. The method according to any one of claims 2 to 5, characterized in that: After determining the global motion vector of the current video frame, the method further includes: Based on the global motion vector of the current video frame and the global motion vector of a previous video frame, an acceleration vector of the current video frame is determined and stored, and the previous video frame is a previous video frame adjacent to the current video frame in the video signal.

8. The method according to claim 1, characterized in that The current video frame is a video frame encoded by intra-frame block copying; The determining of the global motion vector of the current video frame comprises: Based on the global motion vector of the previous video frame and the global motion vector of the next video frame, the global motion vector of the current video frame is determined, wherein the previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the next video frame is the next video frame adjacent to the current video frame in the video signal.

9. The method according to claim 1, characterized in that The encoding of the current block based on the motion vector of the current block and the global motion vector of the current video frame includes: The global motion vector of the current video frame, and the motion vector and prediction residual of the current block are encoded into a bitstream, wherein the prediction residual indicates the difference between the current block and the reference block.

10. The method according to claim 1, characterized in that The current video frame is a video frame encoded using inter-frame prediction; The encoding of the current block based on the motion vector of the current block and the global motion vector of the current video frame includes: Based on the global motion vector of the current video frame, jitter compensation is performed on the motion vector and prediction residual of the current block to obtain a corrected motion vector and a corrected prediction residual of the current block, wherein the prediction residual indicates a difference between the current block and the reference block; The modified motion vector and the modified prediction residual of the current block are encoded into a bitstream.

11. The method according to claim 10, characterized in that The step of performing jitter compensation on the motion vector and prediction residual of the current block based on the global motion vector of the current video frame to obtain a corrected motion vector and a corrected prediction residual of the current block includes: Based on the global motion vector of the current video frame, performing jitter compensation on the motion vector of the current block to obtain a corrected motion vector of the current block; Re-determining a reference block corresponding to the current block based on the corrected motion vector of the current block; A difference between the current block and the re-determined reference block is determined to obtain a modified prediction residual of the current block.

12. A video decoding method, characterized in that: The method comprises: Parse the bitstream to obtain the global motion vector of the current video frame, as well as the motion vector and prediction residual of the current block; The current video frame is a video frame to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector indicates the positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block; The current block is reconstructed based on the global motion vector of the current video frame, the motion vector and the prediction residual of the current block, wherein the reconstructed current block does not have jitter.

13. The method according to claim 12, characterized in that The reconstructing the current block based on the global motion vector of the current video frame, and the motion vector and prediction residual of the current block comprises: Determining an initial current block based on the motion vector of the current block and the prediction residual; Based on the global motion vector of the current video frame, jitter compensation is performed on the initial current block to obtain a reconstructed current block.

14. A video decoding method, characterized in that: The method comprises: Parse the bitstream to obtain the corrected motion vector and corrected prediction residual of the current block; The corrected motion vector and the corrected prediction residual are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, the current video frame is a video frame to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector indicates the positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block; The current block is reconstructed based on the modified motion vector and the modified prediction residual, wherein the reconstructed current block does not have jitter.

15. A video encoding device, characterized in that: The device comprises: A first determination module is used to determine a motion vector of a current block in a current video frame, wherein the motion vector indicates a positional relationship between the current block and a reference block corresponding to the current block, and the current video frame is a video frame to be encoded in a video signal; A second determination module is used to determine a global motion vector of the current video frame, wherein the global motion vector indicates a jitter condition of the current video frame relative to a static video frame in the video signal, wherein the static video frame is a video frame in which a video screen does not jitter; The encoding module is used to encode the current block based on the motion vector of the current block and the global motion vector of the current video frame.

16. The device according to claim 15, characterized in that The current video frame is a video frame encoded using inter-frame prediction; The second determining module is specifically used for: Determine a reference global motion vector, where the reference global motion vector indicates a jitter condition of the current video frame relative to a reference video frame corresponding to the current video frame; Based on the reference global motion vector, a global motion vector of the current video frame is determined.

17. The device according to claim 16, characterized in that The second determining module is specifically used for: If the acceleration vector of the previous video frame is not stored, the step of determining the reference global motion vector is performed, wherein the acceleration vector indicates the jitter acceleration of the previous video frame, and the previous video frame is a previous video frame adjacent to the current video frame in the video signal.

18. The device according to claim 15, characterized in that The current video frame is a video frame encoded using inter-frame prediction; The second determining module is specifically used for: If an acceleration vector of a previous video frame is stored, predicting the jitter of the current video frame relative to the static video frame based on the acceleration vector and the global motion vector of the previous video frame to obtain a first predicted global motion vector, wherein the previous video frame is a previous video frame adjacent to the current video frame in the video signal, and the acceleration vector indicates the jitter acceleration of the previous video frame; Based on the first predicted global motion vector, a global motion vector of the current video frame is determined.

19. The device according to claim 18, characterized in that The device also includes: A prediction module, configured to predict the jitter of the current video frame relative to the static video frame based on the motion vectors of some blocks with static objects in the current video frame, to obtain a second predicted global motion vector; The second determining module is specifically used for: If the difference between the first predicted global motion vector and the second predicted global motion vector is greater than or equal to a difference threshold, determining a reference global motion vector, the reference global motion vector indicating a jitter condition of the current video frame relative to a reference video frame corresponding to the current video frame; Based on the reference global motion vector, a global motion vector of the current video frame is determined.

20. The device according to claim 16, 17 or 19, characterized in that The second determining module is specifically used for: The reference global motion vector is determined based on the motion vectors of each block having a static object in the current video frame.

21. The device according to any one of claims 16 to 19, characterized in that After determining the global motion vector of the current video frame, the device further includes: The third determination module is used to determine and store the acceleration vector of the current video frame based on the global motion vector of the current video frame and the global motion vector of the previous video frame, where the previous video frame is the previous video frame adjacent to the current video frame in the video signal.

22. The device according to claim 15, characterized in that The current video frame is a video frame encoded by intra-frame block copying; The second determining module is specifically used for: Based on the global motion vector of the previous video frame and the global motion vector of the next video frame, the global motion vector of the current video frame is determined, wherein the previous video frame is the previous video frame adjacent to the current video frame in the video signal, and the next video frame is the next video frame adjacent to the current video frame in the video signal.

23. The device according to claim 15, characterized in that The encoding module is specifically used for: The global motion vector of the current video frame, and the motion vector and prediction residual of the current block are encoded into a bitstream, wherein the prediction residual indicates the difference between the current block and the reference block.

24. The device according to claim 15, characterized in that The current video frame is a video frame encoded using inter-frame prediction; The encoding module is specifically used for: Based on the global motion vector of the current video frame, jitter compensation is performed on the motion vector and prediction residual of the current block to obtain a corrected motion vector and a corrected prediction residual of the current block, wherein the prediction residual indicates a difference between the current block and the reference block; The modified motion vector and the modified prediction residual of the current block are encoded into a bitstream.

25. The device according to claim 24, characterized in that The encoding module is specifically used for: Based on the global motion vector of the current video frame, performing jitter compensation on the motion vector of the current block to obtain a corrected motion vector of the current block; Re-determining a reference block corresponding to the current block based on the corrected motion vector of the current block; A difference between the current block and the re-determined reference block is determined to obtain a modified prediction residual of the current block.

26. A video decoding device, characterized in that: The device comprises: A parsing module is used to parse the bitstream to obtain the global motion vector of the current video frame, as well as the motion vector and prediction residual of the current block; The current video frame is a video frame to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector indicates the positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block; The reconstruction module is used to reconstruct the current block based on the global motion vector of the current video frame, and the motion vector and prediction residual of the current block, wherein the reconstructed current block does not have jitter.

27. The device according to claim 26, characterized in that The reconstruction module is specifically used for: Determining an initial current block based on the motion vector of the current block and the prediction residual; Based on the global motion vector of the current video frame, jitter compensation is performed on the initial current block to obtain a reconstructed current block.

28. A video decoding device, characterized in that: The device comprises: A parsing module, used for parsing the bitstream to obtain a corrected motion vector and a corrected prediction residual of the current block; The corrected motion vector and the corrected prediction residual are obtained by performing jitter compensation on the motion vector and the prediction residual of the current block through the global motion vector of the current video frame, the current video frame is a video frame to be decoded in the video signal, the global motion vector indicates the jitter of the current video frame relative to a static video frame in the video signal, the static video frame is a video frame where the video screen does not jitter, the motion vector indicates the positional relationship between the current block and a reference block corresponding to the current block, and the prediction residual indicates the difference between the current block and the reference block; The reconstruction module is used to reconstruct the current block based on the modified motion vector and the modified prediction residual, wherein the reconstructed current block does not have jitter.

29. A video encoding device, characterized in that: The video encoding device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program to implement the steps of any method described in claims 1-11.

30. A video decoding device, characterized in that: The video decoding device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program to implement the steps of any one of claims 12-13 or the method of claim 14.

31. A computer-readable storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed on the computer, the computer executes the steps of any one of the methods described in claims 1-14.

32. A computer program product, characterized in that The computer program product stores computer instructions, and when the computer instructions are executed by a processor, the steps of any one of the methods of claims 1-14 are implemented.

33. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores steps of the method according to any one of claims 1 to 14 executed by one or more processors.

Images