Motion vector processing method and device
By adding motion vectors with high correlation with the current block and new component combinations extracted based on component direction in the candidate motion vector list, the problem of limited size of the candidate motion vector list is solved, the video decoding and encoding performance is improved, and the video decoding and encoding performance is improved. Video fluency.
Patent Information
- Application Number
- CN202411153425.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-21
AI Technical Summary
In the prior art, the size of candidate motion vector lists is limited in video encoding and decoding, which affects video fluency and encoding effect, and needs to be optimized to improve decoding and encoding performance.
By obtaining the number of idle bits and motion vectors of the candidate motion vector list, add priority order, add motion vectors with high correlation with the current block, and extract new components based on the movement range of component direction to combine new motion vectors and add them to the candidate motion. vector list.
The candidate motion vector list is optimized, the probability of selecting better motion vectors is improved, the effect of video decoding and encoding is improved, and the video fluency is enhanced.
Smart Images

Figure CN119031145A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present specification relate to the field of computer technology, and more particularly to a motion vector processing method and device. Background Art
[0002] In video encoding and decoding, building a candidate motion vector list is an important step. The candidate motion vector list is a list of candidate motion vectors, which is constructed based on spatial motion vectors and temporal motion vectors. In inter-frame prediction of video coding, the candidate motion vector list is used to predict the motion vector of the current block of the video to be encoded to reduce the amount of data that needs to be transmitted during the encoding process. In video decoding, the candidate motion vector list is used to reconstruct the current block of the video to be decoded.
[0003] Due to the important role of the candidate motion vector list in video encoding and decoding, in order to make users watch more smoothly, it is urgently necessary to optimize the candidate motion vector list to improve the video decoding and encoding effects. Summary of the invention
[0004] In view of this, the embodiments of this specification provide a motion vector processing method. One or more embodiments of this specification also relate to a motion vector processing device, a computing device, a computer-readable storage medium, and a computer program product to optimize the motion vector list and improve the video decoding and encoding effects.
[0005] According to a first aspect of an embodiment of the present specification, a motion vector processing method is provided, comprising: obtaining a candidate motion vector list of a current block of a video to be decoded; determining a target motion vector to be added according to the number of free bits in the candidate motion vector list and a motion vector adding priority order, wherein the number of target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes an order of adding priorities of a plurality of motion vectors, and among the plurality of motion vectors, a motion vector having a greater correlation with the current block has a higher adding priority; obtaining the target motion vector to be added, and adding the target motion vector to the candidate motion vector list.
[0006] According to the second aspect of the embodiments of this specification, a motion vector processing method is provided, including: obtaining a list of candidate motion vectors of a current block of a video to be encoded; determining the maximum and minimum component values of the candidate motion vectors in the candidate motion vector list in different component directions; determining the moving range of the candidate motion vectors in the candidate motion vector list in different component directions according to the maximum and minimum component values in different component directions; extracting new components in different component directions from the moving range in the different component directions according to a component extraction strategy, and combining the new components in different component directions to obtain a fourth motion vector; and adding the fourth motion vector to the candidate motion vector list.
[0007] According to a third aspect of the embodiments of this specification, a motion vector processing device is provided, comprising: a decoding list acquisition module, configured to acquire a candidate motion vector list of a current block of a video to be decoded; a decoding vector determination module, configured to determine a target motion vector to be added according to the number of free bits in the candidate motion vector list and a motion vector adding priority order, wherein the number of target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes an order of adding priorities of a plurality of motion vectors, and among the plurality of motion vectors, a motion vector having a greater correlation with the current block has a higher adding priority; a decoding vector adding module, configured to obtain a target motion vector to be added, and add the target motion vector to the candidate motion vector list.
[0008] According to a fourth aspect of the embodiments of this specification, a motion vector processing device is provided, including: a coding list acquisition module, configured to obtain a candidate motion vector list of a current block of a video to be encoded; a component range determination module, configured to determine the component maximum value and component minimum value of the candidate motion vector in the candidate motion vector list in different component directions, and determine the moving range of the candidate motion vector in the candidate motion vector list in different component directions according to the component maximum value and component minimum value in different component directions; a component extraction module, configured to extract new components in different component directions from the moving range in different component directions according to a component extraction strategy, and combine the new components in different component directions to obtain a fourth motion vector; a coding vector adding module, configured to add the fourth motion vector to the candidate motion vector list.
[0009] According to the fifth aspect of the embodiments of this specification, a computing device is provided, including: a memory and a processor; the memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method described in any embodiment of this specification.
[0010] According to the sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the method described in any embodiment of this specification are implemented.
[0011] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of this specification.
[0012] An embodiment of the present specification implements a motion vector processing method. Since the method obtains a candidate motion vector list of a current block of a video to be decoded, a target motion vector to be added is determined according to the number of free bits in the candidate motion vector list and a motion vector adding priority order, wherein the number of target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes an order of adding priorities of multiple motion vectors, and among the multiple motion vectors, a target motion vector with a greater correlation with the current block has a higher adding priority. Therefore, the target motion vector to be added can be obtained and the target motion vector can be added to the candidate motion vector list.
[0013] It can be seen that this method targets the situation where the size of the candidate motion vector list is limited due to the video coding standard. Under the constraint of the number of free bits in the candidate motion vector list, based on the priority order of adding motion vectors, priority is given to adding motion vectors that are more relevant to the current block to the candidate motion vector list. That is, priority is given to adding motion vectors that are more relevant to the current block, and then considering adding motion vectors with relatively weaker correlation. This optimizes the candidate motion vector list for video decoding, increases the probability of selecting a better motion vector for decoding, and improves the decoding effect.
[0014] Another embodiment of the present specification implements another motion vector processing method. This method obtains a candidate motion vector list of the current block of the video to be encoded, determines the component maximum value and component minimum value of the candidate motion vectors in the candidate motion vector list in different component directions, and determines the moving range of the candidate motion vectors in the candidate motion vector list in different component directions according to the component maximum value and component minimum value in different component directions, thereby extracting new components in different component directions from the moving range in the different component directions according to the component extraction strategy, and combining the new components in different component directions to obtain a fourth motion vector, and adding the fourth motion vector to the candidate motion vector list.
[0015] It can be seen that this method extracts new components in different component directions based on the moving range of candidate motion vectors in different component directions, combines the new components in different component directions to obtain a fourth motion vector, and adds the fourth motion vector to the candidate motion vector list, which effectively makes up for the shortage of candidate motion vectors in the candidate motion vector list, optimizes the candidate motion vector list for video encoding, increases the probability of selecting a better motion vector for encoding, and improves the encoding effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1a This is a schematic diagram of the positions of the spatial domain neighboring blocks and the temporal domain neighboring blocks of the current block provided by an embodiment of this specification;
[0017] Figure 1b is an architecture diagram of a video transmission system provided by an embodiment of this specification;
[0018] Figure 2 is a flow chart of a motion vector processing method provided by an embodiment of this specification;
[0019] Figure 3 is a flowchart of a processing process of a motion vector processing method provided by an embodiment of the present specification applied in a server transcoding process;
[0020] Figure 4 is a flow chart of a motion vector processing method provided by another embodiment of the present specification;
[0021] Figure 5 is a schematic diagram of inter-frame prediction provided by an embodiment of this specification;
[0022] Figure 6 It is a schematic diagram of equally dividing the moving range provided by one embodiment of this specification;
[0023] Figure 7 is a flowchart of a processing process of a motion vector processing method provided by another embodiment of the present specification applied in a server transcoding process;
[0024] Figure 8 is a schematic diagram of a current frame and a reference frame provided by an embodiment of this specification;
[0025] Fig. 9 is a schematic structural diagram of a motion vector processing device provided by an embodiment of this specification;
[0026] Fig.10 is a structural schematic diagram of a motion vector processing device provided by another embodiment of the present specification;
[0027] Fig.11 It is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION
[0028] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.
[0029] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.
[0030] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0031] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0032] First, the terms involved in one or more embodiments of this specification are explained.
[0033] The candidate motion vector list is a list containing a series of candidate motion vectors, which is constructed based on spatial domain motion vectors and temporal domain motion vectors.
[0034] The spatial motion vector is the motion vector of the encoded adjacent blocks around the current block, which is used to predict the motion vector of the current block, thereby improving the prediction accuracy and reducing the bit rate. Figure 1aAs shown in the schematic diagram of the positions of the spatial neighboring blocks and the temporal neighboring blocks of the current block, the spatial neighboring blocks of the current block include A0, A1, B0, B1 and B2. Therefore, the spatial motion vector of the current block includes: motion vectors of A0, A1, B0, B1 and B2.
[0035] The temporal motion vector describes the relative displacement between the current block and the corresponding matching block in the reference frame and is used to predict the current block. Figure 1a As shown in the schematic diagram of the positions of the spatial-domain neighboring blocks and the temporal-domain neighboring blocks of the current block, the temporal-domain neighboring blocks of the current block include C1 and C0. Therefore, the temporal-domain motion vector of the current block includes: the motion vectors of C1 and C0.
[0036] H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding) are video compression standards designed to provide high standards of compression efficiency.
[0037] The candidate motion vector list is a list of candidate motion vectors, which is constructed based on spatial motion vectors and temporal motion vectors. In inter-frame prediction of video coding, the candidate motion vector list is used to predict the motion vector of the current block of the video to be encoded, so as to reduce the amount of data that needs to be transmitted during the encoding process. In video decoding, the candidate motion vector list is used to reconstruct the current block of the video to be decoded. Due to the important role of the candidate motion vector list in video encoding and decoding, in order to make users watch more smoothly, it is urgent to optimize the candidate motion vector list to improve the video decoding and encoding effects.
[0038] Based on this, in this specification, a motion vector processing method is provided. This specification also involves a motion vector processing device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.
[0039] See also Figure 1b , Figure 1b The architecture diagram of a video transmission system provided by an embodiment of the present specification is shown. The video transmission system may include: a server and multiple clients, such as a first client, a second client, etc.
[0040] The client can be the sender or receiver of the video. For example, the client can be the client corresponding to the user who initiates the live broadcast, the client corresponding to the user who uploads the short video, the client corresponding to the user who watches the live broadcast, the client corresponding to the user who watches the short video, and so on.
[0041] The server can be used to provide video transmission and processing services between clients.
[0042] In some embodiments, such as live broadcast or short video scenarios, a user may send a video stream to a server through a client, and the server transcodes the video stream and pushes it to the client with which the communication is established. In this scenario, the client as the video transmitter can encode the video data (for example, a video stream collected by the client) to transmit it to the server through the network, and after transcoding by the server, it is transmitted to the client as the video receiver for decoding and playback. In order to make the user's online video viewing smoother, the client and / or server as the video transmitter can obtain an optimized candidate motion vector list for the current block of the video to be encoded according to the motion vector processing method provided in the embodiment of this specification, and use the optimized motion vector list to encode the current block. The server and / or the client as the video receiver can obtain an optimized candidate motion vector list for the current block of the video to be decoded according to the motion vector processing method provided in the embodiment of this specification, and use the optimized motion vector list to decode the current block.
[0043] In other embodiments, such as a video service scenario, the server may store a wealth of video resources and provide video resources to users at their request. In this scenario, the server, which serves as a video storage end and a video transmitter, may obtain an optimized candidate motion vector list for the current block of the video to be encoded according to the motion vector processing method provided in the embodiments of this specification, encode the video to be encoded using the optimized motion vector list, and send the encoded video data to multiple clients serving as video receivers. Any one or more of the multiple clients may obtain an optimized candidate motion vector list for the current block of the video to be decoded according to another motion vector processing method provided in the embodiments of this specification, decode and play the video to be decoded using the optimized motion vector list.
[0044] The client and the server are connected via a network. The network provides a medium for the communication link between the client and the server. The network can include various connection types, such as wired or wireless communication links or optical fiber cables. The data transmitted by the client may need to be encoded, transcoded, compressed, etc. before being published to the server.
[0045] The client can be a browser, an APP (Application), or a web application such as an H5 (HyperTextMarkup Language5, Hypertext Markup Language Version 5) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The client can be based on the software development kit (SDK, Software Development Kit) of the corresponding service provided by the server, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The client can be deployed in an electronic device, and needs to rely on the device to run or some APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, mailbox clients, social platform software, etc.
[0046] The server may include servers that provide various services, such as servers that provide communication services to multiple clients, servers that provide background training to support models used on clients, and servers that process data sent by clients. It should be noted that the server can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.
[0047] In some live broadcast scenarios, the video transmission system may also include: an upstream node, a downstream node, and a live broadcast source station. In the live broadcast scenario, the client is used to send the initial live video stream to the upstream node, and the initial live video stream is sent to the live broadcast source station through the upstream node. The live broadcast source station is used to receive the initial live video stream from the upstream node, encode and package it, and then push it to the server. The server, as the entrance or live broadcast center server of the content distribution network, decodes the initial live video stream pushed by the live broadcast source station to obtain the source live video stream, transcodes the source live video stream to obtain the target live video stream, and sends the target live video stream to other clients through the downstream node so that other clients can play the target live video stream. In this process, the client, server and / or live broadcast source station can obtain an optimized candidate motion vector list according to the motion vector processing method provided in the embodiment of this specification, and use the optimized candidate motion vector list for encoding.
[0048] It should be noted that in the video transmission system of the embodiment of this specification, the video encoding standard that can be used is not limited. For example, H.265 / HEVC, H.266 / VVC and other video encoding standards can be used for encoding.
[0049] Exemplarily, in the embodiment of the system, the process from encoding by the encoder at the transmitting end to decoding by the decoder at the receiving end may include:
[0050] The encoder at the transmitting end, for the video to be encoded, first constructs an initial candidate motion vector list based on the time domain and the spatial domain, and then adds new motion vectors to the candidate motion vector list based on the number of free bits in the initial candidate motion vector list and the priority order of adding motion vectors according to a motion vector processing method of an embodiment of the present specification, to obtain a first optimized motion vector list.
[0051] Afterwards, the encoder searches for the best candidate motion vector in the first optimized motion vector list through a motion search algorithm. The best candidate motion vector minimizes the difference between the predicted block and the current block. After determining the index value of the best candidate motion vector, the index value is transmitted to the receiving end through a bit stream (a binary data stream after video compression encoding). Among them, the motion search algorithm can adopt any motion search algorithm such as full search, fast search, hierarchical search, etc., and this specification does not limit this.
[0052] In addition, the encoder also transmits the motion vector difference MVD (Motion Vector Difference) in the bitstream, that is, the difference between the predicted block and the current block. Therefore, the encoder determines the maximum and minimum component values of the candidate motion vectors in the first optimized motion vector list in different component directions according to the motion vector processing method of another embodiment of this specification, and determines the moving range of the candidate motion vectors in the first optimized candidate motion vector list in different component directions according to the maximum and minimum component values in different component directions. According to the component extraction strategy, new components in different component directions are extracted from the moving range in different component directions, and the new components in different component directions are combined to obtain a fourth motion vector, and the fourth motion vector is added to the candidate motion vector list to obtain a second optimized motion vector list. The encoder obtains the best motion vector prediction value from the second optimized motion vector list, takes the best motion vector prediction value as the starting point, and selects the motion vector actually used by the current block through the motion estimation module. Afterwards, the encoder calculates the difference between the motion vector actually used by the current block and the motion vector prediction value selected from the first optimized motion vector list to obtain the MVD, and transmits the MVD to the receiving end through the bitstream.
[0053] After the receiving end receives the bit stream of the video to be decoded, the decoder of the receiving end first constructs an initial candidate motion vector list based on the time domain and the spatial domain, and then adds a new motion vector to the candidate motion vector list based on the number of free bits in the initial candidate motion vector list and the priority order of adding motion vectors according to the motion vector processing method of an embodiment of the present specification, and obtains a first optimized motion vector list. The decoder obtains the index value in the bit stream, obtains the candidate motion vector at the position corresponding to the index value from the first optimized motion vector list as the predicted value of the motion vector of the current block, adds the predicted value of the motion vector of the current block to the MVD, obtains the reconstructed block, and completes the reconstruction of the current block.
[0054] It can be seen from the above process that, on the one hand, the encoder at the transmitting end and the decoder at the receiving end in the embodiment of the system construct the first optimized candidate motion list through the same process, including: for the current block of the video to be decoded, obtaining the candidate motion vector list of the current block, and determining the target motion vector to be added according to the number of free bits in the candidate motion vector list and the priority order of adding the motion vector, so as to obtain the optimized candidate motion vector list based on the priority order of adding the motion vector in view of the situation that the size of the candidate motion vector list is limited due to the video coding standard. Therefore, the index value in the bit stream sent by the encoder is the best candidate motion vector searched out by the motion search algorithm based on the optimized candidate motion list, which improves the probability of searching for a better motion vector for decoding, and the decoder can obtain the best candidate motion vector from the same optimized candidate motion list based on the index value, thereby improving the decoding effect.
[0055] On the other hand, the encoder at the transmitting end of the embodiment of the system obtains the candidate motion vector list (i.e., the first optimized motion vector list) of the current block of the video to be encoded, determines the maximum and minimum component values of the candidate motion vectors in the candidate motion vector list in different component directions, and determines the moving range of the candidate motion vectors in the candidate motion vector list in different component directions according to the maximum and minimum component values of the components in different component directions, thereby extracting new components in different component directions based on the moving range of the candidate motion vectors in different component directions, and combining the new components in different component directions to obtain the first motion vector, and adding the first motion vector to the candidate motion vector list, thereby obtaining an optimized candidate motion vector list (i.e., the second optimized motion vector list). Thus, the encoder can obtain the best motion vector prediction value from the further optimized candidate motion vector list, and use the best motion vector prediction value as the starting point to select the motion vector actually used by the current block through the motion estimation module, thereby, the difference between the motion vector actually used by the current block calculated by the encoder and the motion vector prediction value selected from the first optimized motion vector list can be minimized, effectively improving the encoding effect.
[0056] See also Figure 2 , Figure 2 FIG. 1 is a flowchart of a motion vector processing method provided by an embodiment of the present specification. In one or more embodiments, the motion vector processing method can be applied to an encoder and a decoder of a server or a client. Figure 2 As shown, the method specifically comprises the following steps:
[0057] Step 202: Obtain a candidate motion vector list of a current block of a video to be decoded.
[0058] The video to be decoded refers to the video that has been compressed and encoded, and can be decoded and restored to the original video data by the decoder. For example, the video to be decoded can be a video that needs to be decoded by the server during the transcoding process in a live broadcast scene or a short video scene. Another example is that the video to be decoded can be a video received by the client from the server and needs to be decoded for playback.
[0059] During the video encoding or decoding process, the video frame will be divided into multiple smaller data blocks for processing. For example, in some coding standards, the data block can be represented as a CU (Coding Unit). The current block refers to any data block currently being decoded among the multiple smaller data blocks.
[0060] The candidate motion vector list provides a series of possible candidate motion vectors. During decoding, a suitable candidate motion vector can be selected from these candidate motion vectors to reconstruct the current block.
[0061] Since the candidate motion vector list constructed based on spatial motion vectors and temporal motion vectors may lack sufficient candidate motion vectors to accurately describe the motion of the current block, the method provided in the embodiments of this specification can first obtain an initial candidate motion vector column constructed based on spatial motion vectors and temporal motion vectors, and then add motion vectors to the candidate motion vector list to obtain an optimized candidate motion vector list, thereby increasing the probability of selecting a better candidate motion vector and improving the decoding effect.
[0062] The process of constructing a candidate motion vector list based on spatial motion vectors and temporal motion vectors can be implemented with reference to a general construction process, and this specification does not limit this. For example, a spatial candidate list and a temporal candidate list can be constructed separately, merged, deduplicated and sorted to obtain a candidate motion vector list. Among them, the spatial candidate list includes motion vectors of encoded blocks adjacent to the current block, and the construction process may include: identifying the encoded adjacent blocks to the upper left, above and left of the current block, collecting motion vectors from these adjacent blocks, removing redundancy, and sorting to obtain a spatial candidate list. The temporal candidate list is constructed by the motion vector of the corresponding position of the current block in the previous frame, and the construction process may include: finding the corresponding position of the current block in the reference frame, extracting the motion vector of the corresponding position, and after appropriate adjustments, such as adjustment of the scaling factor, to adapt to the different resolutions or time intervals between the current frame and the reference frame, obtaining a temporal candidate list.
[0063] Step 204: Determine the target motion vector to be added according to the number of free bits in the candidate motion vector list and the priority order of adding the motion vectors.
[0064] Among them, the number of target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes the order of adding priorities of multiple motion vectors, among which the target motion vector with greater correlation with the current block has a higher adding priority.
[0065] There are free bits in the candidate motion vector list because the size of the candidate motion vector list depends on the specific video coding standard and has a certain upper limit. If the upper limit is not reached, there will be a certain amount of free bits. Different video coding standards have different regulations on the size of the candidate motion vector list. In the case of meeting the video coding standard, the candidate motion vector list may have free bits during the construction process due to various reasons. For example, when the motion vector of the adjacent block of the current block is not available, free bits will appear in the candidate list. For example, the same motion vector may be removed due to deduplication, resulting in free bits in the candidate list. Assume that according to a certain video coding standard, the upper limit of the size of the candidate motion vector list is 16 bits. For the current block of the video to be decoded, the constructed candidate motion vector list is actually 12 bits, then the number of free bits in the candidate motion vector list is 4. Based on this, up to 4 motion vectors can be added to the candidate motion vector list later.
[0066] The motion vector addition priority order refers to the order of priority of adding different motion vectors when adding motion vectors to the candidate motion vector list under the constraint of the number of free bits. In the motion vector addition priority order, the order of priority of adding motion vectors available for addition is agreed upon, and this specification does not restrict this. Through the motion vector addition priority order, it is possible to make decisions on which target motion vectors to add when the number of free bits is limited, so that motion vectors with greater relevance to the current block are added to the candidate motion vector list, thereby increasing the probability of selecting a better motion vector for decoding.
[0067] For example, the motion vectors available for addition include the first motion vector and the second motion vector. In this case, the motion vector addition priority order may be expressed as follows: the first motion vector has a higher priority than the second motion vector. It is understandable that the method provided in the embodiments of this specification may set different addition priorities for different motion vectors available for addition based on the difference in the magnitude of their correlation with the current block, and the motion vector with a greater correlation with the current block has a higher priority for addition.
[0068] Exemplarily, the motion vector adding priority order may include: the adding priority of the first motion vector is higher than the adding priority of the second motion vector, the first motion vector is determined based on the pairwise average of the first two candidate motion vectors in the candidate motion vector list, and the second motion vector is determined based on the regrouping and determination of the components of the first two candidate motion vectors.
[0069] Accordingly, determining the target motion vector to be added according to the number of free bits in the candidate motion vector list and the motion vector adding priority order includes:
[0070] If it is determined according to the number of free bits that the free bits in the candidate motion vector list are sufficient to accommodate the first motion vector, but insufficient to accommodate the first motion vector and the second motion vector, then, according to the motion vector adding priority order, it is determined to give priority to adding the first motion vector;
[0071] If it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector and the second motion vector, it is determined to add the first motion vector and the second motion vector, and the second motion vector is located before the first motion vector.
[0072] The paired average is the paired average of the first two candidate motion vectors MV0 and MV1 in the candidate motion vector list {MV0, MV1, MV2, ..., MVn-1} constructed based on the spatial motion vector and the temporal motion vector. The first motion vector obtained by the paired average can be expressed as follows:
[0073]
[0074] Among them, MV0->x represents the component of candidate motion vector MV0 in the horizontal direction x, MV1->x represents the component of candidate motion vector MV1 in the vertical direction y, MV0->y represents the component of candidate motion vector MV0 in the vertical direction y, and MV1->y represents the component of candidate motion vector MV1 in the vertical direction y.
[0075] Assuming that the first two candidate motion vectors in the candidate motion vector list are (x0, y0) and (x1, y1), the components of the two are recombined, and the obtained second motion vectors include: (x0, y1) and (x1, y0).
[0076] Exemplarily, the components of the first two candidate motion vectors MV0 and MV1 in the candidate motion vector list are recombined to generate new second motion vectors (x0, y1) and (x1, y0). When added to the candidate motion vector list, (x0, y1) and (x1, y0) are placed before the paired average. The candidate motion vector list after addition is expressed as: {(x0, y1), (x1, y0), paired average, MV0, MV1, MV2, ..., MVn-1}.
[0077] In this embodiment, since the candidate motion vector list is constructed, the candidate motion vector with higher prediction accuracy is usually placed at the front of the candidate motion vector list, that is, the first two candidate motion vectors in the candidate motion vector list are usually regarded as the first two candidate motion vectors with the highest prediction accuracy. Therefore, considering adding the first motion vector obtained by the paired average of the first two candidate motion vectors and the second motion vector obtained by recombining the components to the candidate motion vector list can effectively improve the optimized candidate motion vector list. Moreover, since the paired average of the first two candidate motion vectors has the advantages of smoothing motion estimation, reducing noise influence, utilizing adjacent block information, improving prediction consistency, reducing prediction error, and adapting to complex motion patterns, its correlation with the current block is greater than the motion vector obtained by recombining the components of the first two candidate motion vectors. Therefore, setting the adding priority of the first motion vector determined based on the paired average to be higher than the adding priority of the second motion vector determined based on the recombining of the components can further improve the prediction accuracy of the candidate motion vector list.
[0078] Exemplarily, assuming that the initially constructed candidate motion vector list has only one free bit, it is determined that only the first motion vector determined by the paired average is added to the candidate motion vector list; assuming that the initially constructed candidate motion vector list has multiple free bits, it is determined that the first motion vector and the recombined second motion vector can be added to the candidate motion vector list, and in the candidate motion vector list, the second motion vector is added at a position before the first motion vector.
[0079] Step 206: Obtain the target motion vector to be added, and add the target motion vector to the candidate motion vector list.
[0080] For example, when it is determined that the first motion vector is to be added, the first motion vector may be generated according to the pairwise average of the first two candidate motion vectors in the candidate motion vector list, and the first motion vector may be added to the first position in the candidate motion vector list.
[0081] For another example, when it is determined that the first motion vector is to be added, it is determined that the second motion vector is also to be added, then the components of the first two candidate motion vectors are recombined to generate a second motion vector, and the second motion vector is added before the first motion vector in the candidate motion vector list.
[0082] It can be seen that this method aims at the situation where the size of the candidate motion vector list is limited due to the video coding standard. Under the constraint of the number of free bits in the candidate motion vector list, based on the priority order of adding motion vectors, priority is given to adding motion vectors that are more relevant to the current block to the candidate motion vector list. That is, priority is given to adding motion vectors that are more relevant to the current block, and then considering adding motion vectors with relatively weak correlation. This optimizes the candidate motion vector list for video decoding, increases the probability of selecting a better motion vector for decoding, and improves the decoding effect.
[0083] For example: based on the priority order of adding motion vectors, priority is given to adding the paired average of the top two candidate motion vectors, and then considering adding the regrouped and obtained motion vector of the components of the top two candidate motion vectors, thereby optimizing the list of candidate motion vectors used for video decoding, increasing the probability of selecting a better motion vector for decoding, and improving the decoding effect.
[0084] In one or more embodiments of the present specification, considering that the zero-value motion vector has the effect of simplifying the decoding process and, in some cases, improving the decoding accuracy, therefore, when the number of free bits in the candidate motion vector list allows, the zero-value motion vector will be added to the candidate motion vector list. The correlation between the zero-value motion vector and the current block is weaker than that of the second motion vector. Therefore, in this embodiment, the priority order of adding the motion vector includes: the priority of adding the second motion vector is higher than the priority of adding the zero-value motion vector. Accordingly, the method can determine whether to add the zero-value motion vector based on the number of free bits in the candidate motion vector list and the priority order of adding the motion vector. Therefore, in the method, the target motion vector to be added is determined based on the number of free bits in the candidate motion vector list and the priority order of adding the motion vector, including:
[0085] If it is determined according to the number of free bits that the free bits in the candidate motion vector list are sufficient to accommodate the first motion vector and the second motion vector, but insufficient to continue to accommodate the zero-valued motion vector, then according to the motion vector adding priority order, it is determined not to add the zero-valued motion vector;
[0086] If it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector, the second motion vector and the zero-valued motion vector, it is determined to add the zero-valued motion vector.
[0087] Among them, the zero-value motion vector is a special motion vector whose value is (0,0), indicating that no movement occurs.
[0088] It can be seen that in this embodiment, according to the number of free bits in the candidate motion vector list, if there are still free bits after adding the first motion vector and the second motion vector, zero-value motion vectors can continue to be added to the candidate motion vector list, thereby utilizing zero-value motion vectors to improve decoding accuracy.
[0089] In practical applications, if the size of the current block is small, it indicates that the texture features of the current block are greatly different from those of the surrounding data blocks, which can also be understood as the current block may contain finer details or unique textures. The use of a zero-valued motion vector can better match the same position block in the reference frame, thereby improving the accuracy of the prediction. Therefore, in one or more embodiments of this specification, a coding block size threshold is set to determine whether the size of the current block is small, and then decide whether to add a zero-valued motion vector. Specifically, before adding the zero-valued motion vector to the candidate motion vector list, the method also includes:
[0090] Determine whether the size of the current block is less than or equal to the coding block size threshold;
[0091] If yes, then entering the step of adding the zero-valued motion vector to the candidate motion vector list;
[0092] If not, and based on the number of free bits, it is determined that the free bits in the candidate motion vector list are sufficient to continue to accommodate new motion vectors, then any two candidate motion vectors are selected from the candidate motion vector list, and a third motion vector is obtained based on the re-combination of the components of the any two candidate motion vectors, and the third motion vector is added to the candidate motion vector list.
[0093] Among them, the coding block size threshold is a threshold used to determine whether the size of the current block is small. Its size can be determined based on experience or historical data statistics, and this specification does not limit this. Size can refer to any one or more metrics such as brightness pixels and chrominance pixels. For example: the coding block size threshold can be set to 256 brightness pixels. When the brightness pixels of the current block are less than or equal to 256 brightness pixels, a zero-valued motion vector can be added. Otherwise, any two candidate motion vectors are selected from the candidate motion vector list, and their components are recombined to obtain a new third motion vector, and the new third motion vector is added to the candidate motion vector list.
[0094] Selecting any two candidate motion vectors from the candidate motion vector list refers to selecting the remaining candidate motion vectors from the candidate motion vector list except the first two candidate motion vectors.
[0095] Exemplarily, from the candidate motion vector list, any two motion vectors MVi and MVj are taken, where 0 ≤ i, j < n. Assume MVi = (xi, yi) and MVj = (xj, yj), and then using these two motion vectors, a new third motion vector is generated by combination, including: (xi, yj) and (xj, yi).
[0096] In the above embodiment, by determining the addition priority order of motion vectors to decide whether to add a zero-valued motion vector, the zero-valued motion vector can be reasonably used to better match the same-position block in the reference frame, improving the prediction accuracy. Moreover, when the zero-valued motion vector is not used, adding the new third motion vector obtained by recombining the components of any two candidate motion vectors to the candidate motion vector list can also improve the prediction accuracy to a certain extent, thus achieving the purpose of optimizing the candidate motion vector list.
[0097] In another embodiment or multiple embodiments of this specification, when the number of idle bits is sufficient, in order to further make full use of the zero-valued motion vector to improve the prediction accuracy, the method further includes:
[0098] When the third motion vector is added to the candidate motion vector list, if there are still idle bits in the candidate motion vector list, the zero-valued motion vector is added to the candidate motion vector list.
[0099] That is, when the size of the current block is greater than the coding block size threshold, it is considered that the correlation between the recombined third motion vector and the current block is greater than that of the zero-valued motion vector. At this time, preferentially adding the recombined third motion vector can improve the prediction accuracy. On this basis, if there are still idle bits, the zero-valued motion vector is added to the candidate motion vector list to further improve the prediction accuracy.
[0100] In order to keep the gap between the newly added motion vector and the existing candidate motion vectors in the candidate motion vector list within a reasonable range and avoid the reduction of prediction efficiency caused by the unreasonableness of the newly added motion vector, in one or more embodiments of this specification, before adding the motion vector obtained by recombination to the candidate motion vector list, the method further includes:
[0101] Determine whether the difference between the motion vector obtained by recombination and the candidate motion vectors in the candidate motion vector list is within a preset difference threshold range;
[0102] If so, determine to add the motion vector obtained by recombination to the candidate motion vector list.
[0103] Among them, the motion vector obtained by recombination may refer to the second motion vector and / or the third motion vector.
[0104] The specific value of the preset difference threshold range can be obtained based on experience or statistics, and this specification does not limit this. Exemplarily, the preset difference threshold range can be expressed as a range between a preset difference lower limit and a preset difference upper limit.
[0105] For example, if the difference between the second motion vector and the candidate motion vector in the candidate motion vector list is less than the lower limit, or greater than the upper limit, it means that the motion vectors are repeated or too irrelevant, which will reduce the prediction efficiency. In this case, the second motion vector is abandoned. The third motion vector is the same, and will not be described here.
[0106] After adding the motion vector to the candidate motion vector list through one or more of the above embodiments, an optimized candidate motion vector list is obtained, and then the optimized candidate motion vector list can be used for decoding to improve the decoding effect. Specifically, after adding the motion vector to the candidate motion vector list, the method further includes:
[0107] The current block is reconstructed using the candidate motion vector list.
[0108] For example: the decoder can select a candidate motion vector from the candidate motion vector list according to the candidate motion vector selection strategy (such as the method of obtaining the index value from the bitstream as described in the aforementioned embodiment) as the predicted motion vector MVP, and use the MVP to reconstruct the current block. The specific process may include, for example: the decoder selects a candidate motion vector from the candidate motion vector list as the predicted motion vector MVP; if the AMVP (Advanced Motion Vector Prediction, an inter-frame prediction mode in video coding) mode is used, the residual MVD needs to be decoded; the MVP is added to the MVD to obtain a reconstructed motion vector, and the reconstructed motion vector is used to find a matching block from the reference frame, and motion compensation is performed to obtain the final reconstructed current block.
[0109] See also Figure 3 , Figure 3 A flowchart showing a processing process of a motion vector processing method provided by an embodiment of the present specification applied in a server transcoding process is shown, which specifically includes the following steps:
[0110] Step 302: The server receives the short video uploaded by the client.
[0111] For example: in a short video scenario, the creator uploads the short video to the server through the client. The server needs to transcode the short video and then send the transcoded video to the user for viewing. In this process, decoding and encoding of the short video are involved. In order to minimize the bandwidth of the transcoded video and make the user's online viewing smoother, in this process, according to the motion vector processing method provided in the embodiment of this specification, during the decoding process, the motion vector can be processed to obtain an optimized candidate motion vector list, and the optimized candidate motion vector list can be used for decoding to improve the decoding effect.
[0112] Correspondingly, the encoder of the client that uploads the short video obtains an optimized candidate motion vector list through the same processing process, searches for the index value of the best candidate motion vector, and transmits the index value and motion vector difference MVD to the server as the receiving end through the bit stream.
[0113] Therefore, in order to improve the decoding effect, in step 302, the server will first receive the short video uploaded by the client, so as to optimize the candidate motion vector list and decode the short video using the optimized candidate motion vector list.
[0114] Step 304: For a current block of the short video to be decoded, obtain a candidate motion vector list of the current block.
[0115] For example, referring to the aforementioned embodiment, based on the spatial motion vector and the temporal motion vector, a candidate motion vector list {MV0, MV1, MV2, ..., MVn-1} of the current block is constructed, and the specific construction process is not repeated here.
[0116] Step 306: If there is only one free bit in the candidate motion vector list, the pairwise average of the first two candidate motion vectors in the candidate motion vector list is added to the first position in the candidate motion vector list.
[0117] For example, the number of candidate motion vectors in the candidate motion vector list may be determined by traversing the candidate motion vector list.
[0118] Exemplarily, assuming that the number of candidate motion vectors in the candidate motion vector list is Cnt, according to the coding standard, the upper limit threshold of the size of the candidate motion vector list is maxMrgNum. If Cnt = maxMrgNum-1, it means that there is only one free bit in the candidate motion vector list. At this time, the paired average is added to the candidate motion vector list.
[0119] Step 308: If there are multiple free bits in the candidate motion vector list, the components of the first two candidate motion vectors in the candidate motion vector list are recombined to obtain a second motion vector.
[0120] Exemplarily, assuming that the number of motion vectors Cnt in the motion vector list is not equal to maxMrgNum-1 and is less than maxMrgNum, it means that there are multiple free bits in the candidate motion vector list. Then, the components of the first two motion vectors MV0(x0, y0) and MV1(x1, y1) in the candidate motion vector list are recombined to generate a new second motion vector, including: (x0, y1) and (x1, y0).
[0121] Step 310: Compare the second motion vector with each original candidate motion vector in the candidate motion vector list to obtain a difference with each candidate motion vector.
[0122] Exemplarily, (x0, y1) and (x1, y0) are compared with each motion vector in the candidate motion vector list {MV0, MV1, MV2, ..., MVn-1} to obtain the difference between each motion vector in {MV0, MV1, MV2, ..., MVn-1}.
[0123] Step 312: If the difference with each candidate motion vector is within a preset difference threshold range, then the second motion vector is added to the candidate motion vector list.
[0124] Exemplarily, the difference between (x0, y1) and (x1, y0) and each motion vector in {MV0, MV1, MV2, ..., MVn-1} is compared with a preset difference threshold range [thr0, thr1], where thr0 represents the lower limit of the threshold range and thr1 represents the upper limit of the threshold range. If the difference is not within the range of [thr0, thr1], it means that the motion vector may be repeated or the difference is too large and should not be added. If the difference is within the range of [thr0, thr1], it means that (x0, y1) and (x1, y0) are different from each candidate motion vector in {MV0, MV1, MV2, ..., MVn-1}, and the difference is reasonable, and (x0, y1) and (x1, y0) can be added to the candidate motion vector list.
[0125] Therefore, if the difference between (x0, y1) and (x1, y0) and any candidate motion vector is not within the preset difference threshold range, adding (x0, y1) and (x1, y0) is abandoned.
[0126] If there are still free spaces in the candidate motion vector list after adding the second motion vector in step 312, or after giving up adding the second motion vector, proceed to step 314; otherwise, obtain the current candidate motion vector list as the optimized candidate motion vector list, proceed to step 328, and use the optimized candidate motion vector list to reconstruct the current block of the short video to be decoded.
[0127] Step 314: If there are still free positions in the candidate motion vector list, add the pairwise average of the first two candidate motion vectors in the original candidate motion vector list to the candidate motion vector list.
[0128] For example: After adding the second motion vector, or after giving up adding the second motion vector, determine whether Cnt is still less than maxMrgNum. If so, add this pairwise average to the candidate motion vector list.
[0129] If there are still free positions in the candidate motion vector list after Step 314, go to Step 316; otherwise, obtain the current candidate motion vector list as the optimized candidate motion vector list, go to Step 328, and use the optimized candidate motion vector list to reconstruct the current block of the short video to be decoded.
[0130] Step 316: If there are still free positions in the candidate motion vector list, determine whether the size of the current block is less than or equal to the coding block size threshold.
[0131] For example: Determine whether the size of the current block is less than or equal to 256 luma pixels.
[0132] Step 318: If the size of the current block is less than or equal to the coding block size threshold, add a zero-valued motion vector to the candidate motion vector list.
[0133] Step 320: If the size of the current block is greater than the coding block size threshold, select any two candidate motion vectors from the candidate motion vectors in the original candidate motion vector list, and recombine the components of the any two candidate motion vectors to obtain a third motion vector.
[0134] Exemplarily, assume that any two candidate motion vectors MVi and MVj are taken from the candidate motion vector list, where 0 ≤ i, j < n. Assume MVi = (xi, yi) and MVj = (xj, yj). Recombine the components of these two candidate motion vectors, then generate new third motion vectors, including: (xi, yj) and (xj, yi).
[0135] Step 322: Compare the third motion vector with each candidate motion vector in the original candidate motion vector list to obtain the difference from each candidate motion vector.
[0136] Exemplarily, compare the third motion vectors (xi, yj) and (xj, yi) with each candidate motion vector in the candidate motion vector list {MV0, MV1, MV2,..., MVn-1} to obtain the differences from each candidate motion vector in {MV0, MV1, MV2,..., MVn-1}.
[0137] Step 324: If the difference with each candidate motion vector is within a preset difference threshold range, then add the third motion vector to the candidate motion vector list.
[0138] For example, if the difference between (xi,yj) and each motion vector in {MV0,MV1,MV2,...,MVn-1} is within the range of [thr0,thr1], then (xi,yj) is added to the candidate motion vector list. (xj,yi) is similar and will not be described here.
[0139] It can be understood that if the difference between the third motion vector and any candidate motion vector is not within the preset difference threshold range, the third motion vector is abandoned from being added.
[0140] After adding the third motion vector, or deciding to give up adding the third motion vector, if there are still free spaces in the candidate motion vector list, proceed to step 326; otherwise, obtain the current candidate motion vector list as the optimized candidate motion vector list, proceed to step 328, and use the optimized candidate motion vector list to reconstruct the current block of the short video to be decoded.
[0141] Step 326: If there are still free spaces in the candidate motion vector list, add the zero-valued motion vector to the candidate motion vector list.
[0142] After step 326, the current candidate motion vector list is obtained as the optimized candidate motion vector list, and the process proceeds to step 328 to reconstruct the current block of the short video to be decoded using the optimized candidate motion vector list.
[0143] Step 328: Reconstruct the current block of the short video to be decoded using the optimized candidate motion vector list.
[0144] Exemplarily, referring to the aforementioned embodiment, the best candidate motion vector may be obtained from the candidate motion vector list based on the index value as the predicted motion vector MVP, and the current block may be reconstructed using the MVP.
[0145] Step 330: After reconstructing the current block of the short video to be decoded, determine the next current block.
[0146] For the next current block, step 304 is continued to be executed to reconstruct the next current block until the short video to be decoded is completely decoded.
[0147] Step 332: The short video to be decoded is decoded to obtain video data of the decoded short video.
[0148] In the above embodiment, when there is only one free bit in the candidate motion vector list, the paired average of the first two candidate motion vectors is added preferentially; when there are multiple free bits, the recombined components of the first two candidate motion vectors and the paired average are added, and the recombined motion vector of the components of the first two candidate motion vectors is placed before the paired average; on this basis, if there are still free bits, it is determined according to the current block size whether to use the components of the remaining candidate motion vectors to recombine and generate a new motion vector to add to the motion vector list, thereby giving priority to adding motion vectors with stronger correlation with the current block, and then adding motion vectors with relatively weak correlation, thereby optimizing the candidate motion vector list for video decoding, increasing the probability of selecting a better motion vector for decoding, and improving the decoding effect.
[0149] See also Figure 4 , Figure 4 FIG. 2 is a flowchart of a motion vector processing method provided by another embodiment of the present specification. In one or more embodiments, the motion vector processing method can be applied to an encoder of a server or a client. Figure 4 As shown, the method specifically comprises the following steps:
[0150] Step 402: Obtain a candidate motion vector list of a current block of the video to be encoded.
[0151] The candidate motion vector list in this step can be based on Figure 3 The processing procedure shown is constructed.
[0152] In the field of video coding, inter-frame prediction is divided into forward prediction and bidirectional prediction according to the prediction direction. Images encoded with forward prediction are called P frames, and images encoded with bidirectional prediction are called B frames. Figure 5 As shown in the inter-frame prediction diagram, the P frame has only one motion vector MV0, and the block pointed to by MV0 is used to generate the final prediction block of the current block; while the B frame has two motion vectors MV0 and MV1, and the blocks pointed to by these two motion vectors are used to generate two prediction blocks of the current block, and the weighted average of these two prediction blocks is used as the final prediction block of the current block. In order to improve the efficiency of video compression, according to some coding standards, a candidate motion vector list {MV0, MV1, MV2,..., MVn-1} is constructed for the current block. From the candidate motion vector list, a suitable candidate motion vector is selected as the motion vector prediction value MVP of the current block, and then encoded based on the MVD between the motion vector MV of the current block and the MVP.
[0153] Therefore, in order to improve the encoding effect, the candidate motion vector list of the current block of the video to be encoded is optimized. First, the candidate motion vector list of the current block of the video to be encoded is obtained.
[0154] Step 404: Determine the maximum value and the minimum value of the components of the candidate motion vectors in the candidate motion vector list in different component directions.
[0155] The candidate motion vector list usually contains a series of candidate motion vectors. Each candidate motion vector includes at least two component directions, for example, usually a horizontal component direction and a vertical component direction. Of course, according to the actual application, there may be other component directions, and this specification does not limit this. On this basis, for any component direction, these candidate motion vectors have components in this component direction, so that a component maximum value and a component minimum value can be found in the components in this component direction of these candidate motion vectors. That is, for a candidate motion vector list, a component maximum value and a component minimum value can be found in each component direction.
[0156] Exemplarily, the candidate motion vector list {MV0, MV1, MV2, ..., MVn-1} has a horizontal component direction and a vertical component direction. Thus, for any candidate motion vector MVi, it has a horizontal component direction, i.e., a component xi in the x direction, and a vertical component direction, i.e., a component yi in the y direction. Thus, based on the candidate motion vector list {MV0, MV1, MV2, ..., MVn-1}, the maximum value x of the component in the x direction of the candidate motion vector list can be determined. max and the minimum value of the component x min , and the maximum value of the component y in the y direction max and the minimum value of the component y min .
[0157] Step 406: Determine the moving range of the candidate motion vectors in the candidate motion vector list in different component directions according to the maximum and minimum component values in different component directions.
[0158] The moving range of the candidate motion vector in a component direction may also be understood as a range between a component maximum value and a component minimum value in the component direction.
[0159] For example, assuming that there is a maximum value x of the component in the x direction of the candidate motion vector list max and the minimum value of the component x min , then the moving range of the candidate motion vector in the candidate motion vector list in the x direction is x min to x max The range of movement of the candidate motion vector in the candidate motion vector list in the y direction is y min to y max range.
[0160] Step 408: according to the component extraction strategy, extract new components in different component directions from the movement ranges in the different component directions, and combine the new components in different component directions to obtain a fourth motion vector.
[0161] The component extraction strategy can be set as needed according to the actual application needs. For example, the new component extracted from the moving range in the x direction can be obtained by x min +P*(x max –x min ), where 0≤P≤1, the new component taken out in the moving range in the y direction can be expressed by y min +P*(y max –y min ) is used to represent the fourth motion vector obtained by the combination: (x min +P*(x max –x min ), y min +P*(y max –y min )), the component extraction strategy can be expressed as: extracting different new components by different values of P. According to different values of P, in the moving range in different component directions, the extracted new components can be one or more, and the fourth motion vector obtained by combining the new components in different component directions can also be one or more.
[0162] For example, a new component x is taken in the x direction min +(x max –x min ) / 2, a new component y is taken out in the y direction min +(y max –y min ) / 2, then the new components are combined to obtain the fourth motion vector: (x min +(x max –x min ) / 2,y min +(y max –y min ) / 2).
[0163] Step 410: Add the fourth motion vector to the candidate motion vector list.
[0164] The method extracts new components in different component directions based on the moving range of candidate motion vectors, combines the new components in different component directions to obtain a fourth motion vector, and adds the fourth motion vector to the candidate motion vector list, effectively making up for the shortage of candidate motion vectors in the candidate motion vector list, and optimizing the candidate motion vector list for video encoding. The encoder can obtain the best motion vector prediction value from the further optimized candidate motion vector list, take the best motion vector prediction value as the starting point, and select the motion vector actually used by the current block through the motion estimation module. Therefore, the difference between the motion vector actually used by the current block calculated by the encoder and the motion vector prediction value selected from the first optimized motion vector list can be minimized, effectively improving the encoding effect.
[0165] The probability of selecting a better motion vector for encoding is increased, thereby improving the encoding effect.
[0166] In one or more embodiments of the present specification, in order to effectively make up for the deficiency of the candidate motion vector list, new components are extracted by equal division, and a new motion vector is combined and added to the candidate motion vector list. Specifically, according to the component extraction strategy, new components in different component directions are extracted from the moving ranges in the different component directions, and the new components in different component directions are combined to obtain a fourth motion vector, including:
[0167] According to the method of equally dividing the moving range, new components corresponding to the equally divided points are taken from the moving range in the horizontal component direction;
[0168] According to the method of equally dividing the moving range, the new components corresponding to the equally divided points are taken from the moving range in the vertical component direction;
[0169] The new component in the horizontal direction is combined with the new component in the vertical direction to obtain a fourth motion vector.
[0170] The moving range may be divided into any number of equal parts, such as 2, 3, 4, etc., which is not limited in this specification. Figure 6 The moving range is divided into equal parts as shown in the figure. Figure 6 In the example, the moving range is divided into four equal parts, and the new components taken out include: x in the x direction min +(x max -x min ) / 4,x min +(x max -x min ) / 2,x min +3(x max -x min ) / 4, and y in the y directionmin +(y max -y min ) / 4, y min +(y max -y min ) / 2, y min +3(y max -y min ) / 4. Combining these three new components in the x - direction and these three new components in the y - direction can obtain nine fourth motion vectors as shown in Figure 6 . Thus, these nine fourth motion vectors are added to the candidate motion vector list.
[0171] In practical applications, the smaller the movement range, the more effectively the fourth motion vectors combined by equally dividing the new components can compensate for the deficiencies of the original candidate motion vectors in the candidate motion vector list. Based on this, in one or more embodiments of this specification, movement range thresholds in different component directions are also set. Only when the movement range is less than the movement range threshold, the new components are taken out by the equal - division method.
[0172] Exemplarily, the movement range threshold in the x - direction may include movement range threshold thr1, and the movement range threshold in the y - direction may include movement range threshold thr2. Thus, when |x max –x min |< thr1 and |y max –y min |< thr2, x min +(x max -x min ) / 4, x min +(x max -x min ) / 2, x min +3(x max -x min ) / 4, and for the y - direction, y min +(y max -y min ) / 4, y min +(y max -y min ) / 2, y min +3(y max -y min ) / 4 are combined to generate new fourth motion vectors and added to the candidate motion vector list.
[0173] Since the new components are taken out in the way of equal division according to the moving range, the gaps between the original candidate motion vectors in the candidate motion vector list can be filled more evenly, so as to optimize the candidate motion vector list more reasonably, improve the probability of selecting a better motion vector for coding, and improve the coding effect.
[0174] In one or more embodiments of this specification, the method further includes:
[0175] Select any two candidate motion vectors from the candidate motion vector list, and obtain a fifth motion vector based on the recombination of the components of the any two candidate motion vectors;
[0176] Add the fifth motion vector to the candidate motion vector list.
[0177] Among them, selecting any two candidate motion vectors means selecting any two candidate motion vectors from the original candidate motion vectors in the candidate motion vector list. In practical applications, according to the accuracy requirements, multiple groups of candidate motion vector pairs can be selected, and the components of each group of candidate motion vector pairs are recombined to obtain multiple fifth motion vectors.
[0178] Exemplarily: Two motion vectors MVi and MVj can be arbitrarily taken from the candidate motion vector list, where 0 ≤ i, j < n. Assume MVi = (xi, yi) and MVj = (xj, yj), and then use these two motion vectors to generate new fifth motion vectors, including: (xi, yj) and (xj, yi).
[0179] In order to make the gap between the newly added motion vector and the existing candidate motion vectors in the candidate motion vector list within a reasonable range and avoid the reduction of prediction efficiency caused by the unreasonableness of the newly added motion vector, in one or more embodiments of this specification, before adding the fifth motion vector to the candidate motion vector list, it further includes:
[0180] Judge whether the difference between the fifth motion vector and the candidate motion vectors in the candidate motion vector list is within a preset difference threshold range;
[0181] If so, determine to add the fifth motion vector to the candidate motion vector list.
[0182] The specific value of the preset difference threshold range can be obtained based on experience or statistics, and this specification does not limit this. Exemplarily, the preset difference threshold range can be expressed as the range between a preset difference lower limit and a preset difference upper limit.
[0183] For example: if the difference between the fifth motion vector and the candidate motion vector in the candidate motion vector list is less than the lower limit, or higher than the upper limit, it means that the motion vectors will be repeated or too irrelevant, which will reduce the prediction efficiency. At this time, give up adding the fifth motion vector.
[0184] In this embodiment, by presetting a difference threshold range, the difference between the newly added motion vector and the existing candidate motion vectors in the candidate motion vector list can be within a reasonable range, thereby avoiding a reduction in prediction efficiency due to an unreasonable newly added motion vector.
[0185] In one or more embodiments of this specification, the method further includes:
[0186] Obtaining a new sixth motion vector based on a pairwise average of first two candidate motion vectors in the candidate motion vector list;
[0187] The sixth motion vector is added to the candidate motion vector list.
[0188] The specific calculation method of the paired average value can be implemented with reference to the above-mentioned embodiment, which will not be described in detail here. It can be understood that since the paired average value has the advantages of smoothing motion estimation, reducing noise influence, utilizing adjacent block information, improving prediction consistency, reducing prediction error, and adapting to complex motion patterns, this embodiment can effectively improve the encoding effect by adding the paired average value to the candidate motion vector list.
[0189] In order to further make up for the deficiencies of the candidate motion vector list, after adding the motion vector of one or more of the above embodiments to the candidate motion vector list, an optimized candidate motion vector list can be obtained, and then the encoding of the current block of the video is completed using the optimized candidate motion vector list. Specifically, after adding the motion vector to the candidate motion vector list, the method further includes:
[0190] Perform motion estimation on the current block using the candidate motion vector list.
[0191] For example, the motion estimation module of the encoder can use SATD (Sum of Absolute Transformed Differences) or SAD (Sum of Absolute Difference) to measure the error between the current block and the reference block, select a suitable motion vector from the list of candidate motion vectors optimized by the method of the embodiment of this specification as the starting point, perform motion estimation, and complete video encoding.
[0192] See also Figure 7 , Figure 7A flowchart showing a processing process of a motion vector processing method provided by another embodiment of the present specification applied in a server transcoding process is shown, which specifically includes the following steps:
[0193] Step 702: The server obtains the decoded short video.
[0194] Continuing with the aforementioned short video scenario, after decoding the short video, the server also needs to encode the decoded short video into other formats as needed so as to send the re-encoded short video to the user.
[0195] Therefore, in order to improve the encoding effect, in step 702, the server will obtain the decoded short video, so as to optimize the candidate motion vector list and encode the short video using the optimized candidate motion vector list.
[0196] Step 704: For a current block of the short video to be encoded, obtain a candidate motion vector list of the current block.
[0197] In this step, the candidate motion vector list {MV0, MV1, MV2, ..., MVn-1} of the current block can be constructed based on the spatial motion vector and the temporal motion vector with reference to the above embodiment, and then based on Figure 2 or Figure 3 The optimized candidate motion vector list obtained by the processing shown is used as the candidate motion vector list obtained in this step, and the specific process is not repeated here.
[0198] Step 706: Select any two candidate motion vectors from the candidate motion vector list, obtain a new motion vector based on the re-combination of the components of the any two candidate motion vectors, and add the new motion vector to the candidate motion vector list if the difference between the new motion vector and the original candidate motion vector in the candidate motion vector list is within a preset difference threshold range.
[0199] Step 708: Add the pairwise average of the first two candidate motion vectors in the candidate motion vector list to the candidate motion vector list.
[0200] Step 710: From the candidate motion vector list, take the maximum component value and the minimum component value in the x direction, and the maximum component value and the minimum component value in the y direction, respectively.
[0201] Step 712: If the moving range corresponding to the maximum and minimum values of the x-component is smaller than the moving range threshold in the x-direction, and the moving range corresponding to the maximum and minimum values of the y-component is smaller than the moving range threshold in the y-direction, then min +(x max -x min ) / 4,xmin +(x max -x min ) / 2,x min +3(x max -x min ) / 4, and y in the y direction min +(y max -y min ) / 4,y min +(y max -y min ) / 2,y min +3(y max -y min ) / 4 are combined to generate a new motion vector and add it to the candidate motion vector list to obtain an optimized candidate motion vector list.
[0202] Step 714: Use the optimized candidate motion vector list to perform motion estimation on the current block of the short video to be encoded.
[0203] For example: Figure 8 As shown in the schematic diagram of the current frame and the reference frame, the motion estimation module of the encoder measures the error between the current block and the reference block, selects a suitable motion vector as a starting point from the optimized candidate motion vector list, performs motion estimation, and completes the motion estimation of the current block.
[0204] Step 716: After completing motion estimation for the current block of the short video to be encoded, determine the next current block.
[0205] For the next current block, step 704 is continued to be executed to perform motion estimation on the next current block until the encoding of the short video to be encoded is completed.
[0206] Step 718: The short video to be encoded is encoded, and video data of the encoded short video is obtained.
[0207] According to the above embodiment, by complementing the candidate motion vector list with the newly added motion vector, the encoder can use the motion vector newly added to the candidate motion vector list as a candidate starting position for motion estimation, thereby improving the accuracy of motion estimation and improving the encoding effect.
[0208] Corresponding to the above method embodiment, this specification also provides a motion vector processing device embodiment, Fig. 9 FIG. 2 shows a schematic diagram of the structure of a motion vector processing device provided by an embodiment of the present specification. Fig. 9 As shown, the device comprises:
[0209] A decoding list acquisition module 902 is configured to acquire a candidate motion vector list of a current block of a video to be decoded;
[0210] The decoding vector determination module 904 is configured to determine the target motion vector to be added according to the number of free bits in the candidate motion vector list and the motion vector adding priority order, wherein the number of the target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes the order of adding priorities of multiple motion vectors, and among the multiple motion vectors, the motion vector with greater correlation with the current block has a higher adding priority;
[0211] The decoding vector adding module 906 is configured to obtain a target motion vector to be added, and add the target motion vector to the candidate motion vector list.
[0212] Optionally, the motion vector adding priority order includes: the adding priority of the first motion vector is higher than the adding priority of the second motion vector, the first motion vector is determined based on the pairwise average of the first two candidate motion vectors in the candidate motion vector list, and the second motion vector is determined based on the regrouping and determination of the components of the first two candidate motion vectors;
[0213] The decoding vector determination module is configured to, if it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector but insufficient to accommodate the first motion vector and the second motion vector, determine to give priority to adding the first motion vector according to the motion vector adding priority order; if it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector and the second motion vector, determine to add the first motion vector and the second motion vector, and the second motion vector is located before the first motion vector.
[0214] Optionally, the motion vector adding priority order includes: the adding priority of the second motion vector is higher than the adding priority of the zero-value motion vector:
[0215] The decoding vector determination module is configured to determine not to add the zero-value motion vector according to the motion vector adding priority order if it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector and the second motion vector, but are insufficient to continue to accommodate the zero-value motion vector; and to determine to add the zero-value motion vector if it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector, the second motion vector and the zero-value motion vector.
[0216] Optionally, the decoding vector adding module is configured to determine whether the size of the current block is less than or equal to the coding block size threshold before adding the zero-value motion vector to the candidate motion vector list; if so, enter the step of adding the zero-value motion vector to the candidate motion vector list; if not, and based on the number of free bits, determine that the free bits of the candidate motion vector list are sufficient to continue to accommodate new motion vectors, then select any two candidate motion vectors from the candidate motion vector list, obtain a third motion vector based on the re-combination of components of the any two candidate motion vectors, and add the third motion vector to the candidate motion vector list.
[0217] Optionally, the device further comprises: a zero-valued vector adding module configured to add a zero-valued motion vector to the candidate motion vector list if there are still free bits in the candidate motion vector list when a third motion vector is added to the candidate motion vector list.
[0218] Optionally, the decoding vector adding module is configured to determine whether the difference between the recombined motion vector and the candidate motion vector in the candidate motion vector list is within a preset difference threshold range before adding the recombined motion vector to the candidate motion vector list, and if so, determine to add the recombined motion vector to the candidate motion vector list.
[0219] Optionally, the device further includes: a reconstruction module configured to reconstruct the current block using the candidate motion vector list after the decoding vector adding module adds the motion vector to the candidate motion vector list.
[0220] The above is a schematic scheme of a motion vector processing device of this embodiment. It should be noted that the technical scheme of the motion vector processing device and the technical scheme of the motion vector processing method described above belong to the same concept, and the details not described in detail in the technical scheme of the motion vector processing device can be referred to the description of the technical scheme of the motion vector processing method described above.
[0221] Corresponding to the above method embodiment, this specification also provides another motion vector processing device embodiment, Fig.10 FIG. 2 shows a schematic diagram of the structure of a motion vector processing device provided by another embodiment of the present specification. Fig.10 As shown, the device comprises:
[0222] The coding list acquisition module 1002 is configured to acquire a candidate motion vector list of a current block of a video to be encoded;
[0223] The component range determination module 1004 is configured to determine the maximum value and the minimum value of the components of the candidate motion vector in the candidate motion vector list in different component directions, and determine the moving range of the candidate motion vector in the candidate motion vector list in different component directions according to the maximum value and the minimum value of the components in different component directions;
[0224] The component extraction module 1006 is configured to extract new components in different component directions from the movement ranges in the different component directions according to the component extraction strategy, and combine the new components in the different component directions to obtain a fourth motion vector;
[0225] The coding vector adding module 1008 is configured to add the fourth motion vector to the candidate motion vector list.
[0226] Optionally, the component extraction module is configured to take out new components corresponding to equally divided points from the moving range in the horizontal component direction in a manner that the moving range is equally divided, and to take out new components corresponding to equally divided points from the moving range in the vertical component direction in a manner that the moving range is equally divided, and to combine the new components in the horizontal direction with the new components in the vertical direction to obtain a fourth motion vector.
[0227] Optionally, the device further comprises: a vector combination module configured to select any two candidate motion vectors from the candidate motion vector list, and obtain a fifth motion vector based on the re-combination of components of the any two candidate motion vectors. A coding vector adding module configured to add the fifth motion vector to the candidate motion vector list.
[0228] Optionally, the coding vector adding module is configured to determine whether the difference between the fifth motion vector and the candidate motion vector in the candidate motion vector list is within a preset difference threshold range before adding the fifth motion vector to the candidate motion vector list; if so, determine to add the fifth motion vector to the candidate motion vector list.
[0229] Optionally, the coding vector adding module is further configured to obtain a new sixth motion vector based on a pairwise average of the first two candidate motion vectors in the candidate motion vector list, and add the sixth motion vector to the candidate motion vector list.
[0230] Optionally, the device further comprises: a motion estimation module configured to perform motion estimation on the current block using the candidate motion vector list after the vector adding module adds the motion vector to the candidate motion vector list.
[0231] The above is a schematic scheme of a motion vector processing device of this embodiment. It should be noted that the technical scheme of the motion vector processing device and the technical scheme of the motion vector processing method described above belong to the same concept, and the details not described in detail in the technical scheme of the motion vector processing device can be referred to the description of the technical scheme of the motion vector processing method described above.
[0232] Fig.11 The block diagram of a computing device 1100 according to one embodiment of the present specification is shown. The components of the computing device 1100 include but are not limited to a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and a database 1150 is used to store data.
[0233] The computing device 1100 also includes an access device 1140 that enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).
[0234] In one embodiment of the present specification, the above components of the computing device 1100 and Fig.11 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.11 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.
[0235] The computing device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1100 may also be a mobile or stationary server.
[0236] The memory 1110 is used to store computer programs / instructions, and the processor 1120 is used to execute the computer programs / instructions stored in the memory 1110. When the computer programs / instructions are executed by the processor, the steps of the method described in any of the above embodiments are implemented.
[0237] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the method described in any of the above embodiments belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the method described in any of the above embodiments.
[0238] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.
[0239] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium belongs to the same concept as the technical scheme described in any of the above embodiments, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the method described in any of the above embodiments.
[0240] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.
[0241] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the method described in any of the above embodiments belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the method described in any of the above embodiments.
[0242] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0243] The computer program / instruction includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable media do not include electric carrier signals and telecommunication signals.
[0244] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.
[0245] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0246] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well. This specification is limited only by the claims and their full scope and equivalents.
Claims
1. A motion vector processing method, characterized in that: include: Obtain a list of candidate motion vectors for a current block of a video to be decoded; Determining a target motion vector to be added according to the number of free bits in the candidate motion vector list and a motion vector adding priority order, wherein the number of the target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes an order of adding priorities of a plurality of motion vectors, and among the plurality of motion vectors, a motion vector having a greater correlation with the current block has a higher adding priority; A target motion vector to be added is obtained, and the target motion vector is added to the candidate motion vector list.
2. The method according to claim 1, characterized in that The motion vector adding priority order includes: the adding priority of the first motion vector is higher than the adding priority of the second motion vector, the first motion vector is determined based on the pairwise average of the first two candidate motion vectors in the candidate motion vector list, and the second motion vector is determined based on the regrouping and determination of the components of the first two candidate motion vectors; Determining a target motion vector to be added according to the number of free bits in the candidate motion vector list and the motion vector adding priority order includes: If it is determined according to the number of free bits that the free bits in the candidate motion vector list are sufficient to accommodate the first motion vector, but insufficient to accommodate the first motion vector and the second motion vector, then, according to the motion vector adding priority order, it is determined to give priority to adding the first motion vector; If it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector and the second motion vector, it is determined to add the first motion vector and the second motion vector, and the second motion vector is located before the first motion vector.
3. The method according to claim 2, characterized in that The motion vector adding priority order includes: the adding priority of the second motion vector is higher than the adding priority of the zero-value motion vector: Determining a target motion vector to be added according to the number of free bits in the candidate motion vector list and the motion vector adding priority order includes: If it is determined according to the number of free bits that the free bits in the candidate motion vector list are sufficient to accommodate the first motion vector and the second motion vector, but insufficient to continue to accommodate the zero-valued motion vector, then according to the motion vector adding priority order, it is determined not to add the zero-valued motion vector; If it is determined according to the number of free bits that the free bits of the candidate motion vector list are sufficient to accommodate the first motion vector, the second motion vector and the zero-valued motion vector, it is determined to add the zero-valued motion vector.
4. The method according to claim 3, characterized in that Before adding the zero-valued motion vector to the candidate motion vector list, the method further includes: Determine whether the size of the current block is less than or equal to the coding block size threshold; If yes, then entering the step of adding the zero-valued motion vector to the candidate motion vector list; If not, and based on the number of free bits, it is determined that the free bits in the candidate motion vector list are sufficient to continue to accommodate new motion vectors, then any two candidate motion vectors are selected from the candidate motion vector list, and a third motion vector is obtained based on the re-combination of the components of the any two candidate motion vectors, and the third motion vector is added to the candidate motion vector list.
5. The method according to claim 4, characterized in that: Also includes: In the case where the third motion vector is added to the candidate motion vector list, if the candidate motion vector list still has free bits, a zero-valued motion vector is added to the candidate motion vector list.
6. The method according to claim 2 or 4, characterized in that: Before adding the reassembled motion vector to the candidate motion vector list, it also includes: Determine whether a difference between the recombined motion vector and a candidate motion vector in the candidate motion vector list is within a preset difference threshold range; If yes, it is determined to add the reassembled motion vector to the candidate motion vector list.
7. The method according to claim 1, characterized in that After adding the motion vector to the candidate motion vector list, also include: The current block is reconstructed using the candidate motion vector list.
8. A motion vector processing method, characterized in that: include: Obtain a list of candidate motion vectors for a current block of the video to be encoded; Determine the maximum value and the minimum value of the components of the candidate motion vectors in the candidate motion vector list in different component directions; Determine, according to the maximum and minimum component values in different component directions, the moving range of the candidate motion vector in the candidate motion vector list in different component directions; According to the component extraction strategy, new components in different component directions are extracted from the movement ranges in the different component directions, and the new components in the different component directions are combined to obtain a fourth motion vector; The fourth motion vector is added to the candidate motion vector list.
9. The method according to claim 8, characterized in that According to the component extraction strategy, new components in different component directions are extracted from the movement ranges in the different component directions, and the new components in different component directions are combined to obtain a fourth motion vector, including: According to the method of equally dividing the moving range, the new components corresponding to the equally divided points are taken from the moving range in the horizontal component direction; According to the method of equally dividing the moving range, the new components corresponding to the equally divided points are taken from the moving range in the vertical component direction; The new component in the horizontal direction is combined with the new component in the vertical direction to obtain a fourth motion vector.
10. The method according to claim 8, characterized in that Also includes: Select any two candidate motion vectors from the candidate motion vector list, and obtain a fifth motion vector based on re-combining components of the any two candidate motion vectors; The fifth motion vector is added to the candidate motion vector list.
11. The method according to claim 10, characterized in that Before adding the fifth motion vector to the candidate motion vector list, the method further includes: Determine whether a difference between the fifth motion vector and a candidate motion vector in the candidate motion vector list is within a preset difference threshold range; If yes, it is determined to add the fifth motion vector to the candidate motion vector list.
12. The method according to claim 8, characterized in that Also includes: Obtaining a new sixth motion vector based on a pairwise average of first two candidate motion vectors in the candidate motion vector list; The sixth motion vector is added to the candidate motion vector list.
13. The method according to claim 8, characterized in that After adding the motion vector to the candidate motion vector list, also include: Perform motion estimation on the current block using the candidate motion vector list.
14. A motion vector processing device, characterized in that: include: A decoding list acquisition module is configured to acquire a candidate motion vector list of a current block of a video to be decoded; a decoding vector determination module, configured to determine a target motion vector to be added according to the number of free bits in the candidate motion vector list and a motion vector adding priority order, wherein the number of target motion vectors to be added is less than or equal to the number of free bits, and the motion vector adding priority order includes an order of adding priorities of a plurality of motion vectors, and among the plurality of motion vectors, a motion vector having a greater correlation with the current block has a higher adding priority; The decoding vector adding module is configured to obtain a target motion vector to be added, and add the target motion vector to the candidate motion vector list.
15. A motion vector processing device, characterized in that: include: A coding list acquisition module, configured to acquire a candidate motion vector list of a current block of a video to be encoded; a component range determination module, configured to determine a component maximum value and a component minimum value of a candidate motion vector in the candidate motion vector list in different component directions, and determine a moving range of the candidate motion vector in the candidate motion vector list in different component directions according to the component maximum value and the component minimum value in different component directions; A component extraction module is configured to extract new components in different component directions from the movement ranges in the different component directions according to the component extraction strategy, and combine the new components in the different component directions to obtain a fourth motion vector; The coding vector adding module is configured to add the fourth motion vector to the candidate motion vector list.
16. A computing device, characterized in that include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 13 are implemented.
17. A computer-readable storage medium, characterized in that: It stores a computer program / instruction, which implements the steps of the method described in any one of claims 1 to 13 when executed by a processor.
18. A computer program product, characterized in that The invention comprises a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.
Citation Information
Patent Citations
Motion vector acquisition method and related prediction method and device
CN110636311A
Method and device for motion compensation in video decoding
CN111083490A
Video decoding method and apparatus and video encoding method and apparatus
US20200336764A1
