Image-based data processing method, apparatus, device, and medium

Abstract

Embodiments of the present application disclose an image-based data processing method, device, equipment and medium. The method comprises: deriving a current block corresponding to a reference block after motion in a current image based on motion vector information of the reference block in a first reference image and a position of the reference block in the first reference image; obtaining a matching relationship between the first reference image and a second reference image, the second reference image being used for inter-frame prediction of the current image; and generating a motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference block. The technical scheme of the present application improves the accuracy of motion vector prediction of the current block, thereby improving the accuracy, efficiency and the like of current block coding, and the reliability of image-based data processing is high.

Description

Technical Field

[0001] This application relates to the field of video processing technology, specifically to an image-based data processing method, an image-based data processing device, an electronic device, and a computer-readable medium. Background Technology

[0002] To accommodate large-scale video data transmission, the original video data is typically encoded at the data sending end to form a compressed data stream. After the data stream is transmitted to the data receiving end, it is then decoded and restored to obtain the predicted and reconstructed video data.

[0003] Understandably, in the predictive coding stage of video encoding and decoding, temporal motion vector prediction (TMVP) can be used for inter-frame prediction. TMVP mainly predicts the motion vector of the current block in the current image by using the motion vector of the co-located block in the reference image. The co-located block is the block in the reference image that is in the same position as the current block in the current image. Usually, the predicted motion vector value of the current block is derived from the motion vector of the blocks near its co-located block.

[0004] However, if the current block and the co-occurring block correspond to different objects, the motion correlation between them is significantly reduced, which leads to lower accuracy in predicting the motion vector of the current block using the motion vector of the co-occurring block. This affects the accuracy and efficiency of the current block encoding, resulting in lower reliability of image-based data processing. Summary of the Invention

[0005] The embodiments of this application provide an image-based data processing method, an image-based data processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which improve the accuracy of current block motion vector prediction, thereby improving the accuracy and efficiency of current block encoding, etc., and the image-based data processing has high reliability.

[0006] In a first aspect, embodiments of this application provide an image-based data processing method, the method comprising: deriving a current block in the current image corresponding to the motion of the reference block based on motion vector information of a reference block in a first reference image and the position of the reference block in the first reference image; obtaining a matching relationship between the first reference image and a second reference image, wherein the second reference image is used to perform inter-frame prediction on the current image; and generating a motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference block.

[0007] Secondly, embodiments of this application provide an image-based data processing apparatus, the apparatus comprising: a derivation module configured to deduce, based on motion vector information of a reference block in a first reference image and the position of the reference block in the first reference image, a current block in the current image corresponding to the motion of the reference block; an acquisition module configured to acquire a matching relationship between the first reference image and a second reference image, the second reference image being used for inter-frame prediction of the current image; and a generation module configured to generate a motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference block.

[0008] Thirdly, embodiments of this application provide an electronic device, including: one or more processors; and a memory for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the image-based data processing method as described above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a computer's processor, cause the computer to perform the image-based data processing method described above.

[0010] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the image-based data processing method described above.

[0011] In the technical solution provided by the embodiments of this application: First, a reference block in a certain reference image is used to deduce the current block in the current image after the reference block moves, thereby establishing a correspondence between the reference block and the current block, wherein the reference block and the current block correspond to the same object; then, the matching relationship between the certain reference image and another reference image used for inter-frame prediction, as well as the motion vector information of the reference block, is used to predict the motion vector of the current block, thereby obtaining the motion vector prediction value of the current block. Since the reference block and the current block correspond to the same object, the motion correlation between the two is high, that is, the motion vector information of the reference block has high reference value, thereby improving the accuracy of the motion vector prediction of the current block, thereby improving the accuracy and efficiency of the current block encoding, and the reliability of image-based data processing is high. Attached Figure Description

[0012] Figure 1 A schematic diagram of a system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0013] Figure 2An exemplary placement of a video encoding device and a video decoding device in a streaming environment is shown;

[0014] Figure 3 A basic flowchart illustrating the encoding process performed by an exemplary video encoder is shown.

[0015] Figure 4 A schematic diagram of an exemplary temporal motion vector prediction is shown;

[0016] Figure 5 A flowchart of an exemplary image-based data processing method is shown;

[0017] Figures 6A to 6D A schematic diagram illustrating the correspondence between a reference block in an exemplary first reference image and a current block in the current image is shown.

[0018] Figure 7 A flowchart of an exemplary image-based data processing method is shown;

[0019] Figure 8 A flowchart of an exemplary image-based data processing method is shown;

[0020] Figure 9 A flowchart of an exemplary image-based data processing method is shown;

[0021] Figures 10A to 10D An exemplary predictive reference image, a co-location reference image, and a schematic diagram of the current image are shown.

[0022] Figure 11 A block diagram of an exemplary image-based data processing apparatus is shown;

[0023] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0027] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0028] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0029] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0030] To facilitate understanding of the technical solutions proposed in the embodiments of this application, the image-based data processing process will be introduced first.

[0031] Video encoding generally refers to the processing of a sequence of images that form a video or video sequence. In the field of video encoding, the terms "picture," "frame," or "image" can be used synonymously. The video encoding used in the embodiments of this application refers to video encoding or video decoding. Video encoding is performed on the source side and typically involves processing (e.g., by compression) the raw video image to reduce the amount of data required to represent the video image, thereby enabling more efficient storage and / or transmission. Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video image. The "encoding" of video frames involved in the embodiments should be understood as involving the "encoding" or "decoding" of a sequence of video images. The combination of encoding and decoding portions is also referred to as encoding and decoding (encoding and decoding).

[0032] Each image in a video sequence is typically segmented into a set of non-overlapping blocks, which are usually encoded at the block level. In other words, the encoder typically processes, i.e., encodes the video at the block (also called image block or video block) level, for example, by generating prediction blocks through spatial (intra-image) and temporal (inter-image) predictions, subtracting the prediction blocks from the current block (the block currently being processed or to be processed) to obtain residual blocks, transforming and quantizing the residual blocks in the transform domain to reduce the amount of data to be transmitted (compressed), while the decoder applies the inverse processing relative to the encoder to the encoded or compressed blocks to reconstruct the current block for representation. Additionally, the encoder replicates the decoder processing loop, such that the encoder and decoder generate the same predictions (e.g., intra-frame and inter-frame predictions) and / or reconstructions for processing, i.e., encoding subsequent blocks.

[0033] The term "block" refers to a portion of an image or frame. In embodiments of this application, the current block refers to the block currently being processed. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded.

[0034] Figure 1 A schematic diagram of a system architecture to which the technical solutions of the embodiments of this application can be applied is shown. For example... Figure 1 As shown, system architecture 100 includes multiple terminal devices that can communicate with each other via, for example, network 150. For instance, system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected via network 150. Figure 1 In one embodiment, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0035] For example, the first terminal device 110 can encode video data (e.g., a video image stream captured by the terminal device 110) to transmit it to the second terminal device 120 via the network 150. The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device 120 can receive the encoded video data from the network 150, decode the encoded video data to recover the video data, and display video images based on the recovered video data.

[0036] In one embodiment of this application, system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded video data, such as during a video conference. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 may encode video data (e.g., a video image stream captured by the terminal device) for transmission over network 150 to the other terminal device. Each of the third terminal device 130 and the fourth terminal device 140 may also receive encoded video data transmitted by the other terminal device, decode the encoded video data to recover the video data, and display the video images on an accessible display device based on the recovered video data.

[0037] exist Figure 1 In the embodiments disclosed herein, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited to these. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 refers to any number of networks that transmit encoded video data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140, including, for example, wired and / or wireless communication networks. Network 150 may exchange data in circuit-switched and / or packet-switched channels. This network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of network 150 may be irrelevant to the operation of this application.

[0038] Figure 2 The placement of video encoding and decoding devices in a streaming environment is illustrated. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital television (TV), and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0039] The streaming system may include an acquisition subsystem 213, which may include a video source 201 such as a digital camera, which creates an uncompressed video image stream 202. In an embodiment, the video image stream 202 includes samples captured by a digital camera. The video image stream 202 is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data 204 (or encoded video bitstream 204). The video image stream 202 may be processed by an electronic device 220, which includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video data 204 (or encoded video bitstream 204) is depicted as a thin line to emphasize the lower data volume of the encoded video data 204 (or encoded video bitstream 204), which may be stored on a streaming server 205 for future use. One or more streaming client subsystems, such as Figure 2 Client subsystems 206 and 208 can access streaming server 205 to retrieve copies 207 and 209 of encoded video data 204. Client subsystem 206 may include, for example, a video decoding device 210 in electronic device 230. Video decoding device 210 decodes the incoming copy 207 of the encoded video data and produces an output video picture stream 211 that can be displayed on display 212 (e.g., a screen) or another presentation device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video stream) may be encoded according to certain video encoding / compression standards.

[0040] It should be noted that electronic devices 220 and 230 may include other components not shown in the figures. For example, electronic device 220 may include a video decoding device, and electronic device 230 may also include a video encoding device.

[0041] In one embodiment of this application, taking the international video coding standards HEVC (High Efficiency Video Coding, H.265) and VVC (Versatile Video Coding, H.266), and the Chinese national video coding standard AVS (Audio Video Coding Standard) as examples, after an input video frame image is received, the video frame image is divided into several non-overlapping processing units according to a block size. Each processing unit will perform a similar compression operation. This processing unit is called a CTU (Coding Tree Unit) or LCU (Largest Coding Unit). The CTU can be further subdivided into one or more basic coding units (CUs), where the CU is the most basic element in a coding process.

[0042] Figure 3 The basic flowchart of the encoding process performed by the video encoder is shown, with intra-frame prediction as an example.

[0043] Wherein, the original image signal s k [x,y] and the predicted image signal Perform the difference operation to obtain the residual signal u. k [x,y], residual signal u k After transformation and quantization, [x,y] is obtained as quantization coefficients. These coefficients are then used to obtain the encoded bitstream through entropy encoding, and to obtain the reconstructed residual signal u′ through inverse quantization and inverse transform. k [x,y], predict image signal With the reconstructed residual signal u′ k [x,y] superimposed to generate image signals Image signal On one hand, the signal is input to the intra-frame mode decision module and the intra-frame prediction module for intra-frame prediction processing; on the other hand, the reconstructed image signal s′ is output through loop filtering. k [x,y], reconstruct the image signal s′ k [x,y] can be used as a reference image for the next frame for motion estimation and motion compensation prediction. Then, based on the motion compensation prediction result s′ r [x+m x ,y+m y ] and intra-frame prediction results Obtain the predicted image signal for the next frame. And continue repeating the above process until the coding is complete.

[0044] The encoding operations for each CU involved in the above video encoding process are detailed below.

[0045] Predictive coding includes intra-frame prediction and inter-frame prediction. The original video signal is predicted from a selected reconstructed video signal to obtain a residual video signal. The encoder needs to determine which predictive coding mode to choose for the current CU and inform the decoder. Intra-frame prediction refers to the predicted signal coming from a region within the same image that has already been encoded and reconstructed; inter-frame prediction refers to the predicted signal coming from another encoded image (called a reference image) that is different from the current image.

[0046] Transform and Quantization: After the residual video signal undergoes transformation operations such as Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT), the signal is transformed into the transform domain, and these are called transform coefficients. The transform coefficients are then subjected to lossy quantization, losing some information to make the quantized signal more suitable for compression. In some video coding standards, there may be more than one transform method to choose from; therefore, the encoder needs to select one of the transform methods for the current CU and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually results in greater distortion and a lower bit rate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually results in less distortion and a higher bit rate.

[0047] Entropy coding, or statistical coding, involves statistically compressing the quantized transform-domain signal based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected coding mode and motion vector data, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless coding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) and Content-Adaptive Binary Arithmetic Coding (CABAC).

[0048] Context-based adaptive binary arithmetic coding mainly involves three steps: binarization, context modeling, and binary arithmetic coding. After binarizing the input syntax elements, the binary data can be encoded using either a regular coding mode or a bypass coding mode. The bypass coding mode does not require assigning a specific probability model to each binary bit; the input binary bit bin value is directly encoded using a simple bypass encoder to speed up the entire encoding and decoding process. Generally, different syntax elements are not completely independent, and the same syntax elements themselves also have a certain degree of memory. Therefore, according to conditional entropy theory, using other encoded syntax elements for conditional coding can further improve coding performance compared to independent coding or memoryless coding. This encoded symbol information used as conditions is called the context. In the regular coding mode, the binary bits of the syntax elements sequentially enter the context modeler. The encoder assigns an appropriate probability model to each input binary bit based on the values ​​of previously encoded syntax elements or binary bits; this process is called context modeling. The context model corresponding to a grammatical element can be located using the context index increment (ctxIdxInc) and the context index start (ctxIdxStart). After the bin value and the assigned probability model are fed into the binary arithmetic encoder for encoding, the context model needs to be updated based on the bin value, which is the adaptive process in encoding.

[0049] Loop Filtering: The transformed and quantized signal undergoes inverse quantization, inverse transform, and prediction compensation to obtain a reconstructed image. Due to the effects of quantization, the reconstructed image differs from the original image in some aspects, resulting in distortion. Therefore, filtering operations can be performed on the reconstructed image, such as deblocking filters (DB), sample adaptive offset (SAO), or adaptive loop filters (ALF), to effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent coded images to predict future image signals, the aforementioned filtering operations are also called loop filtering, i.e., filtering operations within the coding loop.

[0050] Based on the above encoding process, at the decoding end, for each CU, after acquiring the compressed bitstream (i.e., bitstream), entropy decoding is performed to obtain various mode information and quantization coefficients. Then, the quantization coefficients undergo inverse quantization and inverse transform processing to obtain the residual signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to that CU can be obtained. Then, the residual signal and the prediction signal are added together to obtain the reconstructed signal. The reconstructed signal then undergoes loop filtering and other operations to generate the final output signal.

[0051] Understandably, in the predictive coding stage of video encoding and decoding, temporal motion vector prediction can be used for inter-frame prediction. Temporal motion vector prediction primarily predicts the motion vector of the current block in the current image by using the motion vectors of co-located blocks in the reference image. A co-located block refers to a block in the reference image whose position is the same as the current block's position in the current image. Typically, the predicted motion vector value of the current block is derived from the motion vectors of its neighboring co-located blocks; for example... Figure 4 As shown, b is the current image, which includes the current block and some other blocks (A0, A1, B0, B1, B2), and a is the reference image, which includes the co-located block and some other blocks (C0, C1, where the co-located block includes C1). Specifically, the motion vector prediction value of the current block can be derived from the motion vector of C0 or C1.

[0052] However, if the current block and the co-occurring block correspond to different objects, the motion correlation between them is significantly reduced, which leads to lower accuracy in predicting the motion vector of the current block using the motion vector of the co-occurring block, thus affecting the accuracy and efficiency of the current block encoding.

[0053] Therefore, in order to improve the accuracy of motion vector prediction, thereby improving the accuracy and efficiency of current block encoding and ensuring the reliability of image-based data processing, this application provides an image-based data processing scheme, specifically: based on the motion vector information of a reference block in a first reference image and the position of the reference block in the first reference image, the current block in the current image corresponding to the motion of the reference block is derived; the matching relationship between the first reference image and the second reference image is obtained, the second reference image is used to perform inter-frame prediction on the current image; based on the matching relationship and the motion vector information of the reference block, the motion vector prediction value of the current block is generated.

[0054] By implementing the embodiments of this application, a reference block in a reference image is used to deduce the current block in the current image after the reference block moves, thereby establishing a correspondence between the reference block and the current block. The reference block and the current block correspond to the same object, and the motion correlation between the two is high. Accordingly, the motion vector of the current block is predicted by using the matching relationship between the reference image and another reference image used for inter-frame prediction, as well as the motion vector information of the reference block. The motion vector information of the reference block used at this time has high reference value, thereby improving the accuracy of the motion vector prediction of the current block, thus improving the accuracy and efficiency of the current block encoding, and the reliability of image-based data processing is high.

[0055] It should be noted that in the specific implementation of this application, user-related data is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0056] The following details the various implementation details of the technical solutions in the embodiments of this application:

[0057] Figure 5 A flowchart of an image-based data processing method according to one embodiment of this application is shown. This image-based data processing method can be executed by a terminal device or a server that sends video encoded data. This embodiment uses a method executed by a terminal device as an example for illustration. The terminal device may be, for example, a... Figure 2 The video encoding device 203 shown is an example. Figure 5 As shown, this image-based data processing method includes at least S510 to S530, which are described in detail below:

[0058] S510, based on the motion vector information of the reference block in the first reference image and the position of the reference block in the first reference image, the current block in the current image after the reference block moves is derived.

[0059] It is understandable that the current image refers to the image to be encoded, and the reference image refers to other images that have already been encoded and are different from the current image. Usually, there are multiple other images, and for the current image, its corresponding reference image is any one of the multiple other images.

[0060] In the embodiments of this application, the first reference image and the second reference image are essentially both reference images corresponding to the current image, distinguished only by the designation "first" and "second". The first reference image is used as a reference to deduce the motion of a reference block in the first reference image, thereby obtaining the correspondence between the reference block in the first reference image and the current block in the current image. At this time, the object corresponding to the reference block in the first reference image is the same as the object corresponding to the current block in the current image. This object can be a moving object or a non-moving object (i.e., a stationary object). If it is a moving object, the position of the reference block in the first reference image is different from the position of the current block in the current image; if it is a stationary object, the position of the reference block in the first reference image is different from the position of the current block in the current image. The second reference image is used as a reference to perform inter-frame prediction on the current image.

[0061] For easier understanding, please refer to Figure 6A This is a schematic diagram illustrating the correspondence between a reference block in a first reference image and a current block in the current image. For example... Figure 6A As shown, after the reference block R_b in the first reference image moves, it corresponds to the current block C_b in the current image. That is, the objects corresponding to the reference block R_b and the current block C_b are the same, and the object is a moving object.

[0062] Optionally, the first reference image may be an image with the same or closer spatial location as the current image, typically an image that is temporally adjacent to the current image; for example, if the current image is the i-th frame in a video, then the (i-1)-th frame in the video may be the first reference image, or the (i+1)-th frame in the video may be the first reference image. For example, the first reference image may be a co-located reference image corresponding to the current image.

[0063] Optionally, the first reference image may be an image with a different / farther spatial location than the current image, typically an image that is temporally distant from the current image; for example, if the current image is the i-th frame in a video, then the (i-100)-th frame in the video may be the first reference image, or the (i+100)-th frame in the video may be the first reference image. For example, the first reference image is not a corresponding co-located reference image of the current image.

[0064] In practical applications, the first reference image can be flexibly selected according to the specific application scenario.

[0065] In the embodiments of this application, a reference block refers to an image block in a first reference image, and a current block refers to an image block in the current image. Optionally, an image block includes, but is not limited to, at least one of an encoding unit, a luminance encoding unit, a chrominance encoding unit, an encoding block, a luminance encoding block, a chrominance encoding block, a prediction unit, a luminance prediction unit, a chrominance prediction unit, a luminance prediction block, and a chrominance prediction block.

[0066] In this embodiment, the motion vector information refers to the motion vector corresponding to the reference block, which can be calculated using a time-domain motion vector prediction method, i.e., it can be the predicted value of the motion vector corresponding to the reference block. In other embodiments, it can be the actual value of the motion vector corresponding to the reference block. In practical applications, the motion vector information can be flexibly adjusted according to the specific application scenario.

[0067] In one embodiment of this application, the motion vector information includes the motion vector prediction value of a reference block, and the position of the reference block in the first reference image includes position coordinates; for example, the motion vector prediction value of the reference block is represented by MV0, specifically, MV0 = (Δx, Δy), and the position coordinates of the reference block in the first reference image are represented by P, let P = (x, y).

[0068] Accordingly, the process in S510 of deriving the current block in the current image after the reference block has moved, based on the motion vector information of the reference block in the first reference image and the position of the reference block in the first reference image, may include:

[0069] The target position coordinates are obtained by calculating the predicted motion vector value of the reference block and its position coordinates.

[0070] The target block corresponding to the target position coordinates is determined from the current image, and the target block is used as the reference block to move to the current block in the current image.

[0071] That is, in the optional embodiment, the motion vector prediction value of the reference block and the position coordinates are used to calculate the target position coordinates, and then the target block corresponding to the target position coordinates is determined from the current image. At this time, the target block is the current block in the current image corresponding to the reference block after the motion.

[0072] In one optional embodiment, the process of calculating the target position coordinates based on the motion vector prediction value of the reference block and the position coordinates may include: adding the motion vector prediction value to the coordinate values ​​of the same dimension in the position coordinates to obtain the target position coordinates. Optionally, the dimensions include a horizontal dimension and a vertical dimension, where the horizontal dimension corresponds to the X-axis and the vertical dimension corresponds to the Y-axis; for example, continuing from the previous example, if P' represents the target position coordinates, then P' = (x + Δx, y + Δy).

[0073] In an optional embodiment, the target block refers to the image block in the current image located at the target position coordinates; for example, following the previous example, the image block in the current image located at the target position coordinates P' is the target block.

[0074] By implementing this optional embodiment, the current block in the current image corresponding to the motion of the reference block can be obtained easily and accurately, providing strong support for the prediction of the motion vector of the current block.

[0075] S520, obtain the matching relationship between the first reference image and the second reference image, the second reference image is used to perform inter-frame prediction on the current image.

[0076] In the embodiments of this application, the matching relationship refers to whether the first reference image and the second reference image are the same image. Specifically, it can include two cases: case 1 is that the first reference image and the second reference image are the same image, and case 2 is that the first reference image and the second reference image are not the same image (i.e., they are different images).

[0077] S530 generates the predicted motion vector value of the current block based on the matching relationship and the motion vector information of the reference block.

[0078] In this embodiment, the current block in the current image corresponding to the reference block after the reference block moves, and the matching relationship between the first reference image and the second reference image are obtained. Then, the matching relationship between the first reference image and the second reference image, as well as the motion vector information of the reference block, can be used to calculate and obtain the motion vector prediction value of the current block.

[0079] In one embodiment of this application, taking the first reference image as the corresponding co-position reference image of the current image as an example; accordingly, the process of generating the motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference block in S530 can include the following two cases:

[0080] Case 1: If the matching relationship indicates that the second reference image and the co-position reference image are the same image, then the motion vector information of the reference block is used as the motion vector prediction value of the current block.

[0081] That is, in case 1, the second reference image and the co-position reference image are the same image. In this case, the motion vector information of the reference block can be directly used as the motion vector prediction value of the current block.

[0082] Case 2: If the matching relationship indicates that the second reference image and the co-position reference image are different images, then the motion vector information of the reference block is scaled to obtain the motion vector prediction value of the current block.

[0083] That is, in case 2, the second reference image and the co-position reference image are different images. In this case, the motion vector information of the reference block needs to be scaled (shrunk or enlarged) accordingly to obtain the motion vector prediction value of the current block.

[0084] In one embodiment of this application, for case 2, the process of scaling the motion vector information of the reference block to obtain the predicted motion vector value of the current block may include:

[0085] Obtain the distance between the current image and the corresponding reference image, and the distance between the current image and the second reference image;

[0086] Based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, the motion vector information of the reference block is scaled to obtain the motion vector prediction value of the current block.

[0087] That is, in an optional embodiment, the distances between the current image and the co-located reference image and the second reference image are first obtained, and then the motion vector information of the reference block is scaled using the distances between the current image and the co-located reference image, and the distances between the current image and the second reference image, to obtain the motion vector prediction value of the current block.

[0088] By implementing this optional embodiment, the motion vector prediction value of the current block is calculated using the distances between the current image and the corresponding reference image and the second reference image, respectively. This takes into account the relative distance relationship between the current image and the corresponding reference image and the second reference image, thereby improving the accuracy of the motion vector prediction of the current block.

[0089] In an optional embodiment, the process of obtaining the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, may include:

[0090] Obtain the display positions of the co-position reference image, the second reference image, and the current image in the video;

[0091] Based on the display position of the current image and the display position of the peer reference image, determine the distance between the current image and the peer reference image, and based on the display position of the current image and the display position of the second reference image, determine the distance between the current image and the second reference image.

[0092] It is understood that in the optional embodiment, the co-position reference image, the second reference image, and the current image are images in the same video; therefore, in the optional embodiment, the display positions of the co-position reference image, the second reference image, and the current image in the video are first obtained, and then the distance between the current image and the co-position reference image is determined by using the display position of the current image and the display position of the co-position reference image, and the distance between the current image and the second reference image is determined by using the display position of the current image and the display position of the second reference image.

[0093] In one optional embodiment, the display position of the co-position reference image in the video can be the frame number of the co-position reference image in the video; for example, the co-position reference image is the j-th frame image in the video. Similarly, the display position of the second reference image in the video can be the frame number of the second reference image in the video, and the display position of the current image in the video can be the frame number of the current image in the video.

[0094] Accordingly, in optional embodiments, the process of determining the distance between the current image and the corresponding reference image based on the display position of the current image and the display position of the corresponding reference image, and the process of determining the distance between the current image and the second reference image based on the display position of the current image and the display position of the second reference image, may include: performing an absolute value operation on the difference between the frame number corresponding to the current image and the frame number corresponding to the corresponding reference image to obtain the frame number difference between the current image and the corresponding reference image, and using the obtained frame number difference as the distance between the current image and the corresponding reference image; performing an absolute value operation on the difference between the frame number corresponding to the current image and the frame number corresponding to the second reference image to obtain the frame number difference between the current image and the second reference image, and using the obtained frame number difference as the distance between the current image and the second reference image.

[0095] For example, suppose the reference image is the j_r1 frame of the video, the second reference image is the j_r2 frame of the video, and the current image is the j_c frame of the video. Let d0 represent the distance between the current image and the reference image, where d0 = |j_c - j_r1|. At the same time, let d1 represent the distance between the current image and the second reference image, where d1 = |j_c - j_r2|.

[0096] By implementing this optional embodiment, the distances between the current image and the corresponding reference image and the second reference image can be obtained easily and accurately, providing strong support for scaling processing of reference block motion vector information.

[0097] In one optional embodiment, the process of scaling the motion vector information of the reference block based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, to obtain the predicted motion vector value of the current block, may include:

[0098] The distance ratio is obtained by calculating the ratio between the distance between the current image and the corresponding reference image, and between the current image and the second reference image.

[0099] The motion vector prediction value of the current block is obtained by multiplying the distance ratio and the motion vector information of the reference block.

[0100] That is, in the optional embodiment, the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image are obtained. Then, the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image are first compared to obtain the distance ratio. Then, the distance ratio and the motion vector information of the reference block are multiplied to obtain the motion vector prediction value of the current block.

[0101] For example, continuing from the previous example, we obtain the distance d0 between the current image and the co-located reference image, the distance d1 between the current image and the second reference image, and the motion vector prediction value MV0 corresponding to the reference block. MV1 represents the motion vector prediction value of the current block, where MV1 = (d1 / d0) × MV0.

[0102] By implementing this optional embodiment, the prediction of the motion vector of the current block can be achieved simply and accurately using ratio and product operations, thereby obtaining the predicted value of the motion vector of the current block.

[0103] In one optional embodiment, the process of scaling the motion vector information of the reference block based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, to obtain the predicted motion vector value of the current block, may include:

[0104] Based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, the motion vector information of the reference block is scaled to obtain the candidate motion vector prediction value of the current block;

[0105] Based on the display positions of the co-position reference image, the second reference image, and the current image in the video, the direction of the candidate motion vector prediction values ​​is adjusted to obtain the motion vector prediction value of the current block.

[0106] That is, in the optional embodiment, the distance between the current image and the peer reference image, and the distance between the current image and the second reference image are obtained. Then, the motion vector information of the reference block is scaled using the distance between the current image and the peer reference image, and the distance between the current image and the second reference image to obtain the candidate motion vector prediction value of the current block. Then, the direction of the candidate motion vector prediction value is adjusted using the display positions of the peer reference image, the second reference image, and the current image in the video, so as to obtain the motion vector prediction value of the current block.

[0107] In the optional embodiment, the candidate motion vector prediction value refers to the motion vector prediction value obtained by scaling the motion vector information of the reference block using the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image. It may be the final motion vector prediction value corresponding to the current block, or it may not be the final motion vector prediction value corresponding to the current block, so it is called the candidate motion vector prediction value.

[0108] For example, MV1' can be used to represent the candidate motion vector prediction value of the current block, where MV1' = (d1 / d0) × MV0.

[0109] In one of the optional embodiments, the direction of the obtained candidate motion vector prediction values ​​is adjusted by using the display positions of the co-position reference image, the second reference image, and the current image in the video, so as to obtain the final motion vector prediction value corresponding to the current block.

[0110] Optionally, in the optional embodiments, the process of adjusting the direction of the candidate motion vector prediction values ​​based on the display positions of the co-position reference image, the second reference image, and the current image in the video to obtain the motion vector prediction value of the current block may include the following two cases:

[0111] Case 1: If the display position of the co-positioned reference image and the display position of the second reference image are on the same side of the current image display position, then the direction of the candidate motion vector prediction value remains unchanged, and the candidate motion vector prediction value with the unchanged direction is used as the motion vector prediction value of the current block.

[0112] That is, in case 1, the display position of the co-position reference image and the display position of the second reference image are on the same side of the current image display position. In this case, it is necessary to keep the direction of the candidate motion vector prediction value unchanged. Accordingly, the candidate motion vector prediction value with the unchanged direction is the final motion vector prediction value corresponding to the current block.

[0113] Optionally, the display positions of the co-position reference image and the second reference image are on the same side of the current image display position, which may include: both the co-position reference image and the second reference image being on the left side of the current image; or, both the co-position reference image and the second reference image being on the right side of the current image.

[0114] For example, continuing from the previous example, suppose the display position of the co-position reference image and the display position of the second reference image are on the same side of the current image display position. In this case, keep the direction of the candidate motion vector prediction value MV1' unchanged, and the motion vector prediction value MV1 = MV1' = (d1 / d0) × MV0.

[0115] Case 2: If the display position of the co-position reference image and the display position of the second reference image are on different sides of the current image display position, then the direction of the candidate motion vector prediction value is adjusted to the opposite direction, and the candidate motion vector prediction value in the opposite direction is used as the motion vector prediction value of the current block.

[0116] That is, in case 2, the display positions of the co-position reference image and the second reference image are on different sides of the current image display position. In this case, it is necessary to adjust the direction of the candidate motion vector prediction value to the opposite direction. Accordingly, the candidate motion vector prediction value in the opposite direction is the final motion vector prediction value corresponding to the current block.

[0117] Optionally, the display positions of the co-position reference image and the second reference image are located on different sides of the current image display position, which may include: the co-position reference image being on the left side of the current image and the second reference image being on the right side of the current image; or, the co-position reference image being on the right side of the current image and the second reference image being on the left side of the current image.

[0118] For example, continuing from the previous example, suppose the display position of the co-position reference image and the display position of the second reference image are on different sides of the current image display position. In this case, the direction of the candidate motion vector prediction value MV1' is adjusted to the opposite direction, and the motion vector prediction value MV1 = -1MV1' = -1(d1 / d0)×MV0.

[0119] By implementing this optional embodiment, the direction of the candidate motion vector prediction value is adjusted using the display positions of the co-position reference image, the second reference image, and the current image in the video, taking into account the relative directional relationship between the current image and the co-position reference image and the second reference image, thereby improving the accuracy of the current block motion vector prediction.

[0120] This application embodiment uses a reference block in a reference image to deduce the current block in the current image after the reference block moves, thereby establishing a correspondence between the reference block and the current block. The reference block and the current block correspond to the same object, and the motion correlation between the two is high. Accordingly, the motion vector of the current block is predicted by using the matching relationship between the reference image and another reference image used for inter-frame prediction, as well as the motion vector information of the reference block. The motion vector information of the reference block used at this time has high reference value, thereby improving the accuracy of the motion vector prediction of the current block, thus improving the accuracy and efficiency of the current block encoding, and the reliability of image-based data processing is high.

[0121] In one embodiment of this application, another image-based data processing method is provided. For example... Figure 7As shown, the image-based data processing method may include S710 to S720 and S510 to S520.

[0122] In the embodiments of this application, the reference block includes multiple blocks; for example, R_B represents the set of reference blocks and R_b represents the reference block. Specifically, R_B = {R_b1, R_b2, ..., R_b(n-1), R_bn}, where n≥2.

[0123] For easier understanding, please refer to Figure 6B This is a schematic diagram illustrating the correspondence between a reference block in another first reference image and the current block in the current image. For example... Figure 6B As shown, the first reference image includes reference block R_b1 and reference block R_b2, where reference block R_b1 corresponds to the current block C_b1 in the current image, and reference block R_b2 corresponds to the current block C_b2 in the current image. It can be understood that... Figure 6B Different reference blocks correspond to different current blocks; in other words, one current block corresponds to one reference block. Specifically, for... Figure 6B As shown, the motion vector information of the reference block can be clearly obtained because the current block corresponds to only one reference block, and the motion vector information of the reference block corresponding to the current block can be obtained directly.

[0124] For easier understanding, please refer to Figure 6C This is a schematic diagram illustrating the correspondence between a reference block in another first reference image and the current block in the current image. For example... Figure 6C As shown, the first reference image includes reference block R_b1 and reference block R_b2, where reference block R_b1 corresponds to the current block C_b1 in the current image, and reference block R_b2 corresponds to the current block C_b2 in the current image. It can be understood that... Figure 6C Different reference blocks correspond to different current blocks; in other words, one current block corresponds to one reference block. Simultaneously, the position of reference block R_b1 in the first reference image is the same as the position of current block C_b1 in the current image. That is, the object corresponding to reference block R_b1 is the same as the object corresponding to current block C_b1, and this object is a stationary object. Optionally, a correspondence can be established between reference blocks in the first reference image and current blocks in the current image for moving objects, i.e., a correspondence can be established between reference block R_b2 and current block C_b2 to obtain the predicted motion vector value of current block C_b2. For current block C_b1, other temporal prediction methods can be used to predict its motion vector value. Specifically, for... Figure 6C As shown, the motion vector information of the reference block can also be clearly obtained. This is because the current block corresponds to only one reference block, so the motion vector information of the reference block corresponding to the current block can be obtained directly.

[0125] For easier understanding, please refer to Figure 6D This is a schematic diagram illustrating the correspondence between a reference block in another first reference image and the current block in the current image. For example... Figure 6D As shown, the first reference image includes reference block R_b1 and reference block R_b2, where reference block R_b1 corresponds to the current block C_b1 in the current image, and reference block R_b2 also corresponds to the current block C_b1 in the current image. It can be understood that... Figure 6D Different reference blocks in the same block correspond to the same current block; in other words, one current block corresponds to at least two reference blocks. Specifically, for... Figure 6D As shown, since the current block corresponds to at least two reference blocks, the motion vector information of the reference blocks cannot be explicitly obtained, that is, there is a corresponding processing procedure. For details of the processing procedure, please refer to the descriptions in S710 to S720.

[0126] Detailed introductions of S710 to S720 are as follows:

[0127] S710, if different reference blocks have the same current block for the current image after motion, then select the target reference block from the different reference blocks.

[0128] When it appears Figure 6D As shown in the embodiments of this application, a target reference block can be selected from different reference blocks. The target reference block refers to the reference block selected from different reference blocks, which is used to generate the motion vector prediction value of the current block based on the motion vector information corresponding to the selected reference block.

[0129] In one embodiment of this application, the process of selecting a target reference block from different reference blocks in S710 may include:

[0130] Obtain the derivation order of the current block corresponding to the current image after different reference block motions;

[0131] Based on the derivation order corresponding to each reference block, the different reference blocks are sorted to obtain a derivation order sequence;

[0132] The target reference block is selected from different reference blocks based on the derivation sequence.

[0133] It is understandable that, when there are multiple reference blocks, the derivation can be performed sequentially in the optional embodiments, i.e., serial derivation, to ensure the accuracy of the derivation and thus improve the accuracy of generating the predicted motion vector value of the current block; for example, Figure 6D In this process, we can first derive the current block C_b1 corresponding to the reference block R_b1, and then derive the current block C_b1 corresponding to the reference block R_b2. Accordingly, the derivation order of the reference block R_b1 is first, and the derivation order of the reference block R_b2 is later.

[0134] Therefore, in the optional embodiment, when the current block corresponding to different reference blocks is the same after different reference blocks move, the derivation order corresponding to different reference blocks can be obtained first, and then the different reference blocks can be sorted using the derivation order corresponding to different reference blocks to obtain the derivation order sequence. Then, the target reference block can be selected from different reference blocks using the derivation order sequence.

[0135] For example, let different reference blocks be R_b1, R_b2, R_b3, and R_b4. First, the reference block R_b1 is derived, then the reference block R_b2 is derived, then the reference block R_b3 is derived, and finally the reference block R_b4 is derived.

[0136] Optionally, the derivation order can be arranged in reverse order to obtain the derivation sequence L1, i.e., L1 =

[0137] [R_b1, R_b2, R_b3, R_b4], at this point, the derivation sequence L1 can be used to select the target reference block from different references R_b1, R_b2, R_b3, and R_b4.

[0138] Alternatively, the derivation order can be arranged from back to front to obtain the derivation sequence L2, i.e., L2 =

[0139] [R_b4, R_b3, R_b2, R_b1], at this point, the derivation sequence L2 can be used to select the target reference block from different references R_b1, R_b2, R_b3, and R_b4.

[0140] By implementing the optional embodiments, the target reference block can be easily and accurately selected from different reference blocks using the derivation order corresponding to different reference blocks, providing strong support for the prediction of the motion vector of the current block.

[0141] In one of the optional embodiments, the process of selecting a target reference block from different reference blocks based on the derivation order sequence can include the following two methods:

[0142] Method 1: Select the first reference block in the derivation sequence from different reference blocks as the target reference block.

[0143] That is, in an optional embodiment, the first reference block corresponding to the derivation order in the derivation order sequence is selected as the target reference block from different reference blocks. In other words, a reference block is directly selected as the target reference block.

[0144] Optionally, if the derivation sequence is arranged from first to last according to the derivation order, the first derivation sequence selected is the reference block corresponding to the first derivation sequence as the target reference block; for example, in the previous example, reference block R_b1 is selected as the target reference block.

[0145] Optionally, if the derivation sequence is arranged from back to front according to the derivation order, the first derivation sequence selected is the reference block corresponding to the last derivation sequence as the target reference block; for example, in the previous example, reference block R_b4 is selected as the target reference block.

[0146] Method 2: Select a specified number of reference blocks corresponding to the first derivation order from different reference blocks as candidate reference blocks, with the specified number being greater than or equal to 2, and select one candidate reference block as the target reference block from the specified number of candidate reference blocks according to the order of use.

[0147] That is, in the optional embodiment, at least two reference blocks corresponding to the first two derivation sequences in the derivation sequence are selected as candidate reference blocks (i.e., at least two candidate reference blocks are obtained), and then one candidate reference block is selected as the target reference block from the at least two candidate reference blocks according to the usage order.

[0148] Optionally, if the derivation order sequence is arranged from first to last according to the derivation order, then the first two derivation orders are selected as the reference blocks corresponding to the earliest derivation order as the target reference blocks; for example, in the previous example, reference blocks R_b1 and R_b2 are selected as candidate reference blocks.

[0149] Optionally, if the derivation sequence is arranged from back to front according to the derivation order, then the first two derivation sequences are selected as the reference blocks corresponding to the last derivation sequence and used as the target reference blocks; for example, in the previous example, reference blocks R_b4 and R_b3 are selected as candidate reference blocks.

[0150] It is understandable that the usage order is predetermined, and a candidate reference block can be selected as the target reference block from a specified number of candidate reference blocks according to the usage order.

[0151] For example, continuing from the previous example of selecting reference blocks R_b1 and R_b2 as candidate reference blocks, and assuming that in the current round, reference block R_b1 can be selected as the target reference block during the encoding process of the current image in the current round, and reference block R_b2 can be selected as the target reference block during the encoding process of the current image in the next adjacent round.

[0152] Alternatively, following the example of selecting reference blocks R_b4 and R_b3 as candidate reference blocks, and assuming that in the current round, reference block R_b4 can be selected as the target reference block during the encoding process of the current image in the current round, and reference block R_b3 can be selected as the target reference block during the encoding process of the current image in the next adjacent round.

[0153] In other embodiments, a candidate reference block may be selected as the target reference block from a specified number of candidate reference blocks without following the order of use. For example, the motion vector information corresponding to the specified number of candidate reference blocks may be averaged to obtain the average motion vector, and this average motion vector may be used as the motion vector information required to predict the motion vector information of the current block.

[0154] In practical applications, the selection method can be determined according to the agreement between the decoding end and the encoding end, and is not limited to the selection methods exemplified above.

[0155] By implementing optional embodiments, there are multiple selection methods, which allows for flexible selection of the target reference block from different reference blocks, providing high flexibility and applicability to many scenarios.

[0156] It is understood that when there are multiple reference blocks, the alternative embodiments can also derive in parallel to ensure the efficiency of the derivation, thereby improving the efficiency of generating the predicted value of the motion vector of the current block; in practical applications, the derivation method can be flexibly adjusted according to the specific application scenario.

[0157] S720 generates the predicted motion vector value of the current block based on the matching relationship and the motion vector information of the target reference block.

[0158] In this embodiment, a target reference block is selected from different reference blocks. Then, the motion vector information of the target reference block and the matching relationship between the first reference image and the second reference image can be used to calculate the motion vector prediction value of the current block.

[0159] It should be noted that, Figure 7 For detailed information on S510 to S520 shown, please refer to [link / reference]. Figure 5 S510 to S520 shown will not be described again here.

[0160] In this embodiment of the application, when the current block corresponding to the current image is the same after different reference blocks have moved, the target reference block can be selected from different reference blocks using a selection method agreed upon by the encoder and decoder. This allows for the simple and accurate prediction of the motion vector of the current block using the motion vector information of the target reference block, and is applicable to many scenarios.

[0161] In one embodiment of this application, another image-based data processing method is provided. For example... Figure 8 As shown, the image-based data processing method may also include S810 to S830 after S530.

[0162] Detailed introductions of S810 to S830 are as follows:

[0163] S810, acquire the motion vector prediction value of the current block acquired through other time-domain prediction methods.

[0164] In this embodiment of the application, the motion vector prediction value of the current block can also be obtained through other time-domain prediction methods. Correspondingly, the motion vector prediction value of the current block collected through other time-domain prediction methods can be obtained. Optionally, the obtained motion vector prediction value of the current block can be one or more.

[0165] S820: Select the target motion vector prediction value from the acquired motion vector prediction value and the generated motion vector prediction value.

[0166] In this embodiment of the application, the motion vector prediction value of the current block is obtained and the motion vector prediction value of the current block is generated. Then, the target motion vector prediction value can be selected from the obtained motion vector prediction value and the generated motion vector prediction value.

[0167] In the embodiments of this application, the target motion vector prediction value refers to a motion vector prediction value selected from the obtained motion vector prediction values ​​and the generated motion vector prediction values, which is used to encode the current block.

[0168] For example, following the previous example of generating motion vector prediction value MV1, if the obtained motion vector prediction values ​​are MV2, MV3, and MV4, then one of MV1, MV2, MV3, and MV4 is selected as the target motion vector prediction value.

[0169] S830 encodes the current block based on the predicted value of the target motion vector to obtain the corresponding encoded block, and sends the encoded block to the decoding end so that the decoding end can decode according to the encoded block and reconstruct the current image.

[0170] In this embodiment, the predicted value of the target motion vector is obtained. Then, the residual value can be obtained using the predicted value of the target motion vector. Then, the transformation and quantization stage, the entropy coding stage, the loop filtering stage, etc. are entered in sequence to realize the encoding of the current block, that is, to realize the encoding of the current image. The corresponding encoded data is sent to the decoding end. After receiving the corresponding encoded data, the decoding end decodes according to the decoding process to reconstruct the current image.

[0171] It should be noted that, Figure 8 For detailed information on S510 to S530 shown, please refer to [link / reference]. Figure 5 S510 to S530 shown will not be described again here.

[0172] In this embodiment, a target motion vector prediction value is selected from at least two motion vector prediction values ​​corresponding to the current block to predict the motion vector of the current block, thereby realizing the encoding of the current block and improving the accuracy and efficiency of the current block encoding.

[0173] The following provides a detailed description of specific scenarios in the embodiments of this application:

[0174] Please see Figure 9 , Figure 9 This is a flowchart illustrating a data processing method according to an embodiment of this application.

[0175] like Figure 9 As shown, this data processing method includes at least S910 to S980, which are described in detail below:

[0176] S910, obtain the corresponding co-position reference image of the current image, and deduce the current block in the current image after the reference block moves based on the motion vector information of the reference block in the co-position reference image and the position of the reference block in the co-position reference image.

[0177] Optionally, the motion vector information includes the motion vector prediction value of the reference block, and the position of the reference block in the co-located reference image includes position coordinates; accordingly, the target position coordinates are calculated based on the motion vector prediction value of the reference block and the position coordinates, and the target block corresponding to the target position coordinates is determined from the current image, and the target block is used as the current block in the current image corresponding to the motion of the reference block.

[0178] S920: Obtain a reference image (hereinafter referred to as the prediction reference image) for inter-frame prediction of the current image, and determine the matching relationship between the co-position reference image and the prediction reference image.

[0179] S930, if the matching relationship indicates that the predicted reference image and the co-position reference image are the same image, then the motion vector information of the reference block is used as the motion vector prediction value of the current block, and jump to S980.

[0180] For easier understanding, please refer to Figure 10A This is a schematic diagram of a predicted reference image, a co-located reference image, and the current image. For example... Figure 10AAs shown, the prediction reference image and the co-position reference image are the same image. After the reference block in the co-position reference image moves, it corresponds to the current block in the current image. At this time, the motion vector prediction value of the reference block in the co-position reference image is the motion vector prediction value of the current block in the current image.

[0181] S940, if the matching relationship indicates that the predicted reference image and the co-position reference image are different images, then obtain the distance between the current image and the co-position reference image, and the distance between the current image and the predicted reference image.

[0182] For easier understanding, please refer to Figure 10B This is a schematic diagram of another predictive reference image, a co-located reference image, and the current image. For example... Figure 10B As shown, the prediction reference image and the co-position reference image are different images; optionally, the prediction reference image can be a reference image of the co-position reference image, wherein the reference block in the co-position reference image corresponds to the current block in the current image after the reference block moves, and the reference block in the reference image of the co-position reference image also corresponds to the current block in the current image after the reference block moves, while the prediction reference image and the co-position reference image are located on the same side of the current image (i.e., both are on the left side).

[0183] For easier understanding, please refer to Figure 10C This is a schematic diagram of another predictive reference image, a co-located reference image, and the current image. For example... Figure 10C As shown, the prediction reference image and the co-position reference image are different images. Optionally, the prediction reference image can be a reference image of the co-position reference image, wherein the reference block in the co-position reference image corresponds to the current block in the current image after the reference block is moved, and the reference block in the reference image of the co-position reference image corresponds to the reference block in the co-position reference image after the reference block is moved. At the same time, the prediction reference image and the co-position reference image are located on different sides of the current image (i.e., one on the left and one on the right).

[0184] S950: Calculate the ratio between the distance between the current image and the co-located reference image, and between the current image and the predicted reference image, to obtain the distance ratio. Then, multiply the distance ratio with the motion vector information of the reference block to obtain the candidate motion vector prediction value of the current block.

[0185] For example, following the previous... Figure 10B For example, let the distance between the current image and the co-located reference image be d0, the distance between the current image and the predicted reference image be d1, and the motion vector prediction value of the reference block be MV0. Then the candidate motion vector prediction value of the current block is MV1' = (d1 / d0) × MV0.

[0186] Or, following the above Figure 10CFor example, let the distance between the current image and the co-located reference image be d2, the distance between the current image and the predicted reference image be d3, and the motion vector prediction value of the reference block be MV0. Then the candidate motion vector prediction value of the current block is MV1' = (d3 / d2) × MV0.

[0187] S960, if the display position of the co-position reference image and the display position of the prediction reference image are on the same side of the current image display position, then the direction of the candidate motion vector prediction value remains unchanged, and the candidate motion vector prediction value with unchanged direction is used as the motion vector prediction value of the current block.

[0188] For example, following the previous... Figure 10B For example, the predicted motion vector value for the current block is MV1 = MV1' = (d1 / d0) × MV0.

[0189] S970, if the display position of the co-position reference image and the display position of the prediction reference image are on different sides of the current image display position, then the direction of the candidate motion vector prediction value is adjusted to the opposite direction, and the candidate motion vector prediction value in the opposite direction is used as the motion vector prediction value of the current block.

[0190] For example, following the previous... Figure 10C For example, the predicted motion vector value for the current block is MV1 = -1MV1' = -1(d3 / d2)×MV0.

[0191] S980 encodes the current block based on the predicted motion vector values ​​of the current block.

[0192] In other embodiments, there can be multiple reference blocks in the co-position reference image, and correspondingly, there can also be multiple reference blocks in the prediction reference image, and different reference blocks in the co-position reference image can correspond to the same current block in the current image.

[0193] For easier understanding, please refer to Figure 10D This is a schematic diagram of another predictive reference image, a co-located reference image, and the current image. For example... Figure 10D As shown, the prediction reference image and the co-position reference image are different images; optionally, the prediction reference image can be a reference image of the co-position reference image, wherein after the two different reference blocks in the co-position reference image are moved, they correspond to the current block in the current image, and after the two different reference blocks in the reference image of the co-position reference image are moved, they also correspond to the current block in the current image.

[0194] Optionally, at this time, a target reference block can be selected from the co-position reference image according to the specified selection method. Then, the motion vector information of the target reference block and the matching relationship between the co-position reference image and the prediction reference image are used to calculate the motion vector prediction value of the current block. For the specific process, please refer to the foregoing embodiments, which will not be repeated here.

[0195] In this embodiment, a reference block in a reference image is first used to deduce the current block in the current image after the reference block moves, thus establishing a correspondence between the reference block and the current block, where the reference block and the current block correspond to the same object. Then, the motion vector of the current block is predicted using the matching relationship between the reference image and another reference image used for inter-frame prediction, as well as the motion vector information of the reference block, thereby obtaining the motion vector prediction value of the current block. Since the reference block and the current block correspond to the same object, the motion correlation between the two is high, that is, the motion vector information of the reference block has high reference value, thereby improving the accuracy of the motion vector prediction of the current block, thus improving the accuracy and efficiency of the current block encoding, and the reliability of image-based data processing is high.

[0196] Figure 11 This is a block diagram illustrating an image-based data processing apparatus according to one embodiment of this application. Figure 11 As shown, the device includes:

[0197] The derivation module 1101 is configured to derive the current block in the current image after the reference block moves based on the motion vector information of the reference block in the first reference image and the position of the reference block in the first reference image;

[0198] The acquisition module 1102 is configured to acquire the matching relationship between the first reference image and the second reference image, wherein the second reference image is used to perform inter-frame prediction on the current image;

[0199] The generation module 1103 is configured to generate a motion vector prediction value for the current block based on the matching relationship and the motion vector information of the reference block.

[0200] In one embodiment of this application, based on the aforementioned scheme, the first reference image includes a co-position reference image corresponding to the current image; the generation module 1103 is specifically configured as follows: if the matching relationship indicates that the second reference image and the co-position reference image are the same image, then the motion vector information of the reference block is used as the motion vector prediction value of the current block; if the matching relationship indicates that the second reference image and the co-position reference image are different images, then the motion vector information of the reference block is scaled to obtain the motion vector prediction value of the current block.

[0201] In one embodiment of this application, based on the foregoing scheme, the generation module 1103 is further configured to: obtain the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image; based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, scale the motion vector information of the reference block to obtain the motion vector prediction value of the current block.

[0202] In one embodiment of this application, based on the foregoing scheme, the generation module 1103 is further configured to: perform a ratio operation on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, to obtain a distance ratio; and perform a product operation on the distance ratio and the motion vector information of the reference block to obtain the motion vector prediction value of the current block.

[0203] In one embodiment of this application, based on the foregoing scheme, the co-position reference image, the second reference image, and the current image are images in the same video; the generation module 1103 is further specifically configured to: obtain the display positions of the co-position reference image, the second reference image, and the current image in the video; determine the distance between the current image and the co-position reference image based on the display position of the current image and the display position of the co-position reference image, and determine the distance between the current image and the second reference image based on the display position of the current image and the display position of the second reference image.

[0204] In one embodiment of this application, based on the foregoing scheme, the generation module 1103 is further configured to: scale the motion vector information of the reference block based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, to obtain the candidate motion vector prediction value of the current block; and adjust the direction of the candidate motion vector prediction value based on the display positions of the co-located reference image, the second reference image, and the current image in the video, to obtain the motion vector prediction value of the current block.

[0205] In one embodiment of this application, based on the foregoing scheme, the generation module 1103 is further configured as follows: if the display position of the co-position reference image and the display position of the second reference image are on the same side of the current image display position, then the direction of the candidate motion vector prediction value is kept unchanged, and the candidate motion vector prediction value with the unchanged direction is used as the motion vector prediction value of the current block; if the display position of the co-position reference image and the display position of the second reference image are on different sides of the current image display position, then the direction of the candidate motion vector prediction value is adjusted to the opposite direction, and the candidate motion vector prediction value with the opposite direction is used as the motion vector prediction value of the current block.

[0206] In one embodiment of this application, based on the aforementioned scheme, the reference block includes multiple blocks; the generation module 1103 is specifically configured as follows: if different reference blocks have the same current block corresponding to the current image after motion, a target reference block is selected from the different reference blocks; and a motion vector prediction value of the current block is generated based on the matching relationship and the motion vector information of the target reference block.

[0207] In one embodiment of this application, based on the foregoing scheme, the generation module 1103 is further configured to: obtain the derivation order of the current block corresponding to the current image after the different reference blocks are moved; sort the different reference blocks according to the derivation order corresponding to the different reference blocks respectively to obtain a derivation order sequence; and select a target reference block from the different reference blocks according to the derivation order sequence.

[0208] In one embodiment of this application, based on the foregoing scheme, the derivation order sequence is arranged from first to last according to the derivation order, or the derivation order sequence is arranged from last to first according to the derivation order; the generation module 1103 is further specifically configured to: select the first reference block corresponding to the derivation order in the derivation order sequence from the different reference blocks as the target reference block; or select the first specified number of reference blocks corresponding to the derivation order in the derivation order sequence from the different reference blocks as candidate reference blocks, wherein the specified number is greater than or equal to 2, and select one candidate reference block from the specified number of candidate reference blocks as the target reference block according to the usage order.

[0209] In one embodiment of this application, based on the aforementioned scheme, the motion vector information includes the motion vector prediction value of the reference block, and the position of the reference block in the first reference image includes position coordinates; the derivation module 1101 is specifically configured to: calculate the target position coordinates based on the motion vector prediction value of the reference block and the position coordinates; determine the target block corresponding to the target position coordinates from the current image, and use the target block as the current block in the current image corresponding to the motion of the reference block.

[0210] In one embodiment of this application, based on the foregoing scheme, the device further includes an encoding module configured to: acquire motion vector prediction values ​​of the current block obtained through other temporal prediction methods; select a target motion vector prediction value from the acquired motion vector prediction values ​​and the generated motion vector prediction values; encode the current block based on the target motion vector prediction value to obtain an encoded block corresponding to the current block, and send the encoded block to a decoding end so that the decoding end can decode according to the encoded block and reconstruct the current image.

[0211] It should be noted that the image-based data processing apparatus and the image-based data processing method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the image-based data processing apparatus provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0212] Embodiments of this application also provide an electronic device, including: one or more processors; and a memory for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the image-based data processing methods provided in the various embodiments above.

[0213] Figure 12 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the embodiments of this application. It should be noted that... Figure 12 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0214] like Figure 12As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1203. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. An Input / Output (I / O) interface 1205 is also connected to bus 1204.

[0215] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a local area network (LAN) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0216] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0217] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0218] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0219] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0220] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image-based data processing method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0221] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image-based data processing methods provided in the various embodiments described above.

[0222] The above content is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

[0223] It is understood that in the specific implementation of this application, the input data and other related data required to perform the deduction of the local model are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. An image-based data processing method, characterized in that, include: Based on the motion vector information of the reference block in the first reference image and the position of the reference block in the first reference image, the current block in the current image after the reference block moves is obtained; Obtain the matching relationship between the first reference image and the second reference image, wherein the second reference image is used to perform inter-frame prediction on the current image; Based on the matching relationship and the motion vector information of the reference block, a motion vector prediction value for the current block is generated.

2. The method according to claim 1, characterized in that, The first reference image includes the co-position reference image corresponding to the current image; The step of generating the motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference block includes: If the matching relationship indicates that the second reference image and the co-located reference image are the same image, then the motion vector information of the reference block is used as the motion vector prediction value of the current block; If the matching relationship indicates that the second reference image and the co-position reference image are different images, then the motion vector information of the reference block is scaled to obtain the motion vector prediction value of the current block.

3. The method according to claim 2, characterized in that, The step of scaling the motion vector information of the reference block to obtain the predicted motion vector value of the current block includes: Obtain the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image; Based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, the motion vector information of the reference block is scaled to obtain the motion vector prediction value of the current block.

4. The method according to claim 3, characterized in that, The step of scaling the motion vector information of the reference block based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, to obtain the motion vector prediction value of the current block, includes: The distance ratio is obtained by calculating the ratio between the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image. The distance ratio and the motion vector information of the reference block are multiplied to obtain the predicted motion vector value of the current block.

5. The method according to claim 3, characterized in that, The co-location reference image, the second reference image, and the current image are images from the same video; obtaining the distance between the current image and the co-location reference image, and the distance between the current image and the second reference image, includes: Obtain the display positions of the co-position reference image, the second reference image, and the current image in the video; The distance between the current image and the corresponding reference image is determined based on the display position of the current image and the display position of the corresponding reference image. Similarly, the distance between the current image and the second reference image is determined based on the display position of the current image and the display position of the second reference image.

6. The method according to claim 5, characterized in that, The step of scaling the motion vector information of the reference block based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, to obtain the motion vector prediction value of the current block, includes: Based on the distance between the current image and the co-located reference image, and the distance between the current image and the second reference image, the motion vector information of the reference block is scaled to obtain the candidate motion vector prediction value of the current block; Based on the display positions of the co-position reference image, the second reference image, and the current image in the video, the direction of the candidate motion vector prediction value is adjusted to obtain the motion vector prediction value of the current block.

7. The method according to claim 6, characterized in that, The step of adjusting the direction of the candidate motion vector prediction values ​​based on the display positions of the co-position reference image, the second reference image, and the current image in the video to obtain the motion vector prediction value of the current block includes: If the display position of the co-position reference image and the display position of the second reference image are on the same side of the current image display position, then the direction of the candidate motion vector prediction value remains unchanged, and the candidate motion vector prediction value with the unchanged direction is used as the motion vector prediction value of the current block. If the display positions of the co-position reference image and the second reference image are on different sides of the current image display position, the direction of the candidate motion vector prediction value is adjusted to the opposite direction, and the candidate motion vector prediction value in the opposite direction is used as the motion vector prediction value of the current block.

8. The method according to any one of claims 1 to 7, characterized in that, The reference blocks include multiple blocks; generating the motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference blocks includes: If multiple reference blocks have different reference blocks that, after motion, correspond to the same current block for the current image, then the target reference block is selected from the different reference blocks; Based on the matching relationship and the motion vector information of the target reference block, a motion vector prediction value for the current block is generated.

9. The method according to claim 8, characterized in that, Selecting a target reference block from the different reference blocks includes: The derivation order of the current block corresponding to the current image after obtaining the motion of the different reference blocks is obtained; The different reference blocks are sorted according to their respective derivation orders to obtain a derivation order sequence; The target reference block is selected from the different reference blocks based on the derivation sequence.

10. The method according to claim 9, characterized in that, The derivation sequence is arranged from first to last according to the derivation order, or the derivation sequence is arranged from last to first according to the derivation order; The step of selecting a target reference block from the different reference blocks based on the derivation sequence includes: Select the first reference block corresponding to the derivation order in the derivation order sequence from the different reference blocks as the target reference block; or Select a specified number of reference blocks corresponding to the first derivation order from the different reference blocks as candidate reference blocks, wherein the specified number is greater than or equal to 2, and select one candidate reference block as the target reference block from the specified number of candidate reference blocks according to the order of use.

11. The method according to any one of claims 1 to 7, characterized in that, The motion vector information includes the predicted motion vector value of the reference block, and the position of the reference block in the first reference image includes position coordinates; the step of deriving the current block in the current image corresponding to the motion of the reference block based on the motion vector information of the reference block in the first reference image and the position of the reference block in the first reference image includes: The target position coordinates are calculated based on the motion vector prediction value of the reference block and the position coordinates. The target block corresponding to the target position coordinates is determined from the current image, and the target block is used as the current block in the current image after the reference block is moved.

12. The method according to any one of claims 1 to 7, characterized in that, After generating the motion vector prediction value of the current block based on the matching relationship and the motion vector information of the reference block, the method further includes: Obtain the motion vector prediction value of the current block acquired through other time-domain prediction methods; Select the target motion vector prediction value from the obtained motion vector prediction value and the generated motion vector prediction value; The current block is encoded based on the predicted value of the target motion vector to obtain the encoded block corresponding to the current block, and the encoded block is sent to the decoding end so that the decoding end can decode according to the encoded block and reconstruct the current image.

13. An image-based data processing apparatus, characterized in that, include: The derivation module is configured to derive the current block in the current image after the reference block moves based on the motion vector information of the reference block in the first reference image and the position of the reference block in the first reference image; The acquisition module is configured to acquire the matching relationship between the first reference image and the second reference image, wherein the second reference image is used to perform inter-frame prediction on the current image; The generation module is configured to generate a motion vector prediction value for the current block based on the matching relationship and the motion vector information of the reference block.

14. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by the electronic device, cause the electronic device to perform the method of any one of claims 1 to 12.

15. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

16. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1 to 12.

Images