Candidate list sorting method, device, electronic device and computer-readable medium

Abstract

The embodiments of the present disclosure disclose a candidate list sorting method, device, electronic device and computer-readable medium. A specific implementation of the method includes: receiving a preset video frame; encoding the preset video frame to obtain an image coding unit information set corresponding to the preset video frame; obtaining a reference video frame corresponding to the preset video frame; determining the image prediction coding unit information as the current image coding information; determining each image prediction coding unit information that meets the preset adjacent condition in the image coding unit information set as each adjacent image coding information; generating each adjacent image motion vector corresponding to each adjacent image coding information; constructing a motion vector candidate list; reordering the motion vector candidate list to update the motion vector candidate list. This implementation reduces the bits consumed by writing the motion vector into the bitstream during video transmission, thereby improving video compression efficiency.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a candidate list sorting method, apparatus, electronic device, and computer-readable medium. Background Art

[0002] Video coding and decoding technologies are commonly used in video transmission. Among them, the candidate list for inter-frame prediction is a motion vector used in video coding and decoding to describe the correlation between blocks in adjacent image frames. When encoding a block in an image frame, a block similar to the current block is found in an adjacent image frame as the prediction value of the current block. The block being encoded is called the current block, and the similar block in the adjacent image frame is called the reference block. To describe the position of the reference block, the position of the reference frame is usually expressed by the reference frame index, and the position of the reference block is expressed by the motion vector between the current block and the reference block. When writing the video image into the bitstream during video transmission, it is only necessary to write the motion vector index in the motion vector candidate list into the bitstream, so there is no need to write the motion vector directly. The commonly used method for obtaining the motion vector candidate list is to determine each neighboring block around the current block, and then select the motion vectors of the neighboring blocks in a fixed order to construct the motion vector candidate list.

[0003] However, when the above method is used to establish a motion vector candidate list, the following technical problems often occur:

[0004] The motion vectors of neighboring blocks are selected in a fixed order to construct a motion vector candidate list. The index order of the generated motion vector candidate list is also fixed, but the index order is not adjusted during use. Therefore, the probability of selecting each index in the motion vector encoding process is relatively even, which makes the entropy value corresponding to each code larger in the entropy encoding process, so the consumed binary codewords are larger, resulting in more bits consumed in writing the motion vector into the bitstream and lower video compression efficiency.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the Invention

[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure propose a method, apparatus, electronic device, and computer-readable medium for sorting a motion vector candidate list to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In the first aspect, some embodiments of the present disclosure provide a method for sorting a motion vector candidate list, the method comprising: receiving a preset video frame; encoding the above-mentioned preset video frame to obtain an image coding unit information set corresponding to the above-mentioned preset video frame; obtaining a reference video frame corresponding to the above-mentioned preset video frame; for each image prediction coding unit information in the above-mentioned image coding unit information set, performing the following steps: determining the above-mentioned image prediction coding unit information as the current image coding information; based on the above-mentioned current image coding information, determining each image prediction coding unit information in the above-mentioned image coding unit information set that meets a preset adjacent condition as each adjacent image coding information; based on the above-mentioned reference video frame and the above-mentioned each adjacent image coding information, generating each adjacent image motion vector corresponding to the above-mentioned each adjacent image coding information; constructing a motion vector candidate list based on the generated each adjacent image motion vector; and reordering the above-mentioned motion vector candidate list to update the above-mentioned motion vector candidate list.

[0009] In the second aspect, some embodiments of the present disclosure provide a motion vector candidate list sorting device, the device including: a receiving unit, configured to receive a preset video frame; an encoding processing unit, configured to perform encoding processing on the above-mentioned preset video frame to obtain an image coding unit information set corresponding to the above-mentioned preset video frame; an acquisition unit, configured to obtain a reference video frame corresponding to the above-mentioned preset video frame; an execution unit, configured to perform the following steps on each image prediction coding unit information in the above-mentioned image prediction coding unit information set: determining the above-mentioned image prediction coding unit information as the current image coding information; based on the above-mentioned current image coding information, determining each image prediction coding unit information in the above-mentioned image coding unit information set that meets the preset adjacent condition as each adjacent image coding information; based on the above-mentioned reference video frame and the above-mentioned each adjacent image coding information, generating each adjacent image motion vector corresponding to the above-mentioned each adjacent image coding information; constructing a motion vector candidate list based on the generated each adjacent image motion vector; and reordering the above-mentioned motion vector candidate list to update the above-mentioned motion vector candidate list.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0012] The above-described embodiments of the present disclosure have the following beneficial effects: The motion vector candidate list sorting method of some embodiments of the present disclosure can adjust the motion vector candidate list, reducing the number of bits consumed when writing motion vectors into the bitstream. Specifically, the high bit consumption in writing motion vectors into the bitstream is caused by selecting motion vectors from neighboring blocks in a fixed order to construct the motion vector candidate list. The resulting motion vector candidate list index order is also fixed, but the index order is not adjusted during use. As a result, the probability of selecting each index during motion vector encoding is relatively even, resulting in a large entropy value corresponding to each code during entropy encoding. Consequently, a large number of binary codewords are consumed, resulting in a high bit consumption when writing motion vectors into the bitstream and low video compression efficiency. Based on this, the motion vector candidate list sorting method of some embodiments of the present disclosure first receives a preset video frame. This obtains a video frame image for generating a motion vector candidate list. Then, the preset video frame is encoded to obtain a set of image coding unit information corresponding to the preset video frame. The obtained video frame image can then be segmented to obtain information about each image prediction coding unit. Next, a reference video frame corresponding to the preset video frame is obtained. This allows the reference video frame corresponding to the preset video frame for generating the motion vector candidate list to be obtained. Next, for each image prediction coding unit information in the image coding unit information set, the following steps are performed: The image prediction coding unit information is determined as the current image coding information. This allows the image prediction coding unit information to be determined as the current image coding information to be processed. Next, based on the current image coding information, each image prediction coding unit information in the image coding unit information set that meets a preset adjacency condition is determined as each adjacent image coding information. This allows the adjacent image coding information corresponding to the current image coding information to be obtained. Next, based on the reference video frame and each adjacent image coding information, adjacent image motion vectors corresponding to each adjacent image coding information are generated. This allows the adjacent image motion vectors corresponding to the current image coding information to be generated. Next, a motion vector candidate list is constructed based on the generated adjacent image motion vectors. This allows the motion vector candidate list corresponding to the current image coding information to be obtained. Finally, the motion vector candidate list is reordered to update the motion vector candidate list. Thus, the obtained motion vector candidate list can be sorted to obtain an updated motion vector candidate list.Also, because the motion vector candidate lists generated for each image prediction coding unit information are sorted to obtain each updated motion vector candidate list, the motion vectors that are sorted earlier in each motion vector candidate list have a higher probability of being written into the bitstream, so that the coding value allocated when the motion vector is entropy coded in the subsequent operation of writing into the bitstream is smaller, thereby reducing the bits consumed in the bitstream. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flowchart of some embodiments of the method for sorting a motion vector candidate list according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of the motion vector candidate list sorting device according to the present disclosure;

[0016] Figure 3 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0019] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 , shows a process 100 of some embodiments of the motion vector candidate list sorting method according to the present disclosure. The motion vector candidate list sorting method includes the following steps:

[0024] Step 101: Receive a preset video frame.

[0025] In some embodiments, the execution subject (e.g., a computing device) of the motion vector candidate list sorting method may receive a preset video frame. The preset video frame may be a frame of video image in the video data. The execution subject may be a server for managing and storing video data. In practice, first, the execution subject may receive the video data sent by the server. The video data may be video data composed of various video frames. Then, the execution subject may determine the various video frames in the video data as various video frames to be processed according to preset extraction conditions. The preset extraction conditions may be to extract one video frame every other video frame. Finally, for each of the various video frames to be processed, the execution subject may determine the video frame to be processed as a preset video frame.

[0026] Step 102: Encode the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame.

[0027] In some embodiments, the execution entity may perform encoding processing on the preset video frame to obtain an image coding unit information set corresponding to the preset video frame. Each image prediction coding unit information in the image coding unit information set may be pixel information composed of an image of a preset size in the preset video frame. The preset size may be pre-set and is not limited herein. For example, an image prediction coding unit information may be pixel information of a PU (prediction unit).

[0028] In some optional implementations of some embodiments, the execution entity may perform encoding processing on the preset video frame through the following steps to obtain an image coding unit information set corresponding to the preset video frame:

[0029] The first step is to determine image segmentation units based on pre-stored video coding information. The video coding information may be information related to an algorithm used to compress and decompress digital video. For example, the video coding information may be H.264, H.265 / HEVC, or VP9. The video coding information may include segmentation unit information, segmentation method information, and video decoding information. The segmentation unit information may be a coding standard used for video frame image segmentation. For example, the segmentation unit information may be an 8x8 or 16x16 pixel PU, or a 16x16 pixel MB (Macroblock). The segmentation method information may be text information describing the segmentation method used to segment the video frame image corresponding to the segmentation unit information. For example, when the segmentation unit information is "16x16 pixel PU," the segmentation method information may be "Starting from the pixel in the first row and first column of the video frame, segment the video frame horizontally and vertically at intervals of 16 pixels." The video decoding information may be information related to the algorithm used for video decoding. The image segmentation unit may be a coding standard used for video frame image segmentation. In practice, the execution entity may determine the segmentation unit information included in the video encoding information as the image segmentation unit.

[0030] In the second step, the preset video frame is segmented according to the image segmentation unit to obtain information about each image prediction coding unit. In practice, the execution entity may first determine the segmentation method information included in the video coding information corresponding to the image segmentation unit as the image segmentation method. Then, the execution entity may determine the number of interleaved pixels based on the text information contained in the image segmentation method. For example, if the image segmentation method is "starting from the pixels in the first row and first column of the video frame, segmenting the video frame with 16 pixels as horizontal and vertical intervals," the corresponding number of interleaved pixels is "16." If the image segmentation method is "starting from the pixels in the first row and first column of the video frame, segmenting the video frame with 32 pixels as horizontal and vertical intervals," the corresponding number of interleaved pixels is "32." Then, the execution entity may horizontally and vertically segment the preset video frame, starting from the pixels in the first row and first column of the preset video frame, with the number of interleaved pixels as intervals, to obtain horizontal strips and vertical strips. Next, the execution entity may determine the intersection of each horizontal strip and each vertical strip as the image coding unit. Finally, the execution entity may determine information of all pixels included in each image coding unit in each image coding unit as image prediction coding unit information, thereby obtaining information of each image prediction coding unit. The number of interval pixels may be used to represent the number of pixels required to be spaced between two segments when segmenting the preset video frame.

[0031] The third step is to determine the above-mentioned image prediction coding unit information as an image coding unit information set.

[0032] Step 103: Acquire a reference video frame corresponding to the preset video frame.

[0033] In some embodiments, the execution entity may obtain a reference video frame corresponding to the preset video frame. The reference video frame may be the second video frame following the preset video frame in the video data. In practice, the execution entity may obtain the second video frame following the preset video frame from the video data as the reference video frame.

[0034] Step 104: For each image prediction coding unit information in the image coding unit information set, perform the following steps:

[0035] Step 1041: Determine the image prediction coding unit information as the current image coding information.

[0036] In some embodiments, the execution entity may determine the image prediction coding unit information as current image coding information.

[0037] Step 1042: Based on the current image coding information, determine each image prediction coding unit information that meets a preset adjacent condition in the image coding unit information set as each adjacent image coding information.

[0038] In some embodiments, the execution entity may determine, based on the current image coding information, the image prediction coding unit information in the image coding unit information set that satisfies a preset adjacent condition as each adjacent image coding information. The preset adjacent condition may be image prediction coding unit information adjacent to the current image coding information. For example, the preset adjacent condition may be image prediction coding unit information located above, below, to the left, or to the right of the current image coding information. In practice, the execution entity may find image prediction coding unit information located above or to the left of the current image coding information from the image coding unit information set and determine it as each adjacent image coding information.

[0039] Step 1043 : generating adjacent image motion vectors corresponding to the adjacent image coding information based on the reference video frame and the adjacent image coding information.

[0040] In some embodiments, the execution entity may generate adjacent image motion vectors corresponding to the adjacent image coding information based on the reference video frame and the adjacent image coding information. The adjacent image motion vectors may be vectors representing the direction of the adjacent image coding information.

[0041] In some optional implementations of some embodiments, the execution entity may generate adjacent image motion vectors corresponding to the adjacent image coding information based on the reference video frame and the adjacent image coding information through the following steps:

[0042] In the first step, for each adjacent image coding information in the above adjacent image coding information, the following steps are performed:

[0043] The first sub-step involves generating candidate range information based on the adjacent image coding information, the reference video frame, and a preset search range. The preset search range may be a preset shape centered at a preset position. The preset position may be a coordinate. For example, the preset search range may be a square area centered on the pixel in the first row and first column of the current image coding information and with a preset number of pixels as its side length. Alternatively, the preset search range may be a circular area centered on the pixel in the first row and first column of the adjacent image coding information and with a preset number of pixels as its radius. The candidate range information may be pixel information consisting of an image of a preset size in the reference video frame. The preset number of pixels and the preset size may be pre-set and are not limited herein. In practice, the execution entity may first obtain the coordinates of the location of the vector image coding information. Then, the execution entity may find the coordinates of the location from the reference video frame as the starting point of the preset search range. Next, the execution entity may obtain a preset number of search centers in a direction to the left or upward of the starting point, a preset number of pixels away. The search centers may be the coordinates of the pixels in the reference video frame located a preset number of pixels away from the starting point. Next, for each of the search centers, the execution entity may determine a preset search range with the search center as a preset position as a candidate range, thereby obtaining each candidate range. Finally, the execution entity may determine information about all pixels included in each of the candidate ranges within the reference video frame as candidate range information, thereby obtaining each candidate range information.

[0044] In a second sub-step, based on the adjacent image coding information and the candidate range information, each similarity corresponding to the adjacent image coding information is generated. In practice, the execution entity may calculate the similarity between the adjacent image coding information and the candidate range information using a similarity algorithm to obtain the similarity. The similarity algorithm may be a sum of absolute differences (SAD), mean squared error (MSE), or normalized cross-correlation (NCC).

[0045] In the third sub-step, the similarity that satisfies a preset similarity condition among the various similarities is determined as the proximity similarity, wherein the preset similarity condition may be the highest similarity.

[0046] The fourth sub-step is to determine the candidate range information corresponding to the above-mentioned proximity similarity as the reference image coding information.

[0047] The second step is to generate adjacent image motion vectors based on the adjacent image coding information and the reference image coding information.

[0048] In some optional implementations of some embodiments, the execution entity may generate an adjacent image motion vector based on the adjacent image coding information and the reference image coding information through the following steps:

[0049] The first step is to generate adjacent coordinates corresponding to the adjacent image coding information based on the adjacent image coding information and the preset video frame. The adjacent coordinates may be coordinates representing the position of the adjacent image coding information in the preset video frame. In practice, the execution entity may determine the coordinate position of the pixel in the first row and first column of the adjacent image coding information in the preset video frame as the adjacent coordinates.

[0050] The second step is to generate reference coordinates corresponding to the reference image coding information based on the reference image coding information and the reference video frame. The reference coordinates may be coordinates representing the position of the reference image coding information in the reference video frame. In practice, the execution entity may determine the coordinate position of the pixel in the first row and first column of the reference image coding information in the reference video frame as the reference coordinates.

[0051] The third step is to generate an adjacent image motion vector based on the adjacent coordinates and the reference coordinates. In practice, the execution entity may input the adjacent coordinates and the reference coordinates into a first preset formula to obtain the adjacent image motion vector. The first preset formula may be A=(x2-x1, y2-y1). The adjacent image motion vector may be represented by A. The adjacent coordinates may be represented by (x1, y1. The reference coordinates may be represented by (x2, y2).

[0052] Step 1044: construct a motion vector candidate list based on the generated motion vectors of each adjacent image.

[0053] In some embodiments, the execution entity may construct a motion vector candidate list based on the generated motion vectors of each adjacent image. The motion vector candidate list may be a list composed of the motion vectors of each adjacent image. In practice, the execution entity may combine the generated motion vectors of each adjacent image into a motion vector candidate list.

[0054] Step 1045 : reorder the motion vector candidate list to update the motion vector candidate list.

[0055] In some embodiments, the execution entity may reorder the motion vector candidate list to update the motion vector candidate list.

[0056] In some optional implementations of some embodiments, the execution entity may reorder the motion vector candidate list through the following steps to update the motion vector candidate list:

[0057] The first step is to generate a current pixel group corresponding to the current image coding information based on the current image coding information and the preset adjacent range information. The preset adjacent range information may be a preset shape range. For example, the preset adjacent range information may be a rectangular range with the pixel in the first row and first column of the preset adjacent image information as the starting point and a preset pixel width to the upper and left sides of the preset adjacent image information. Alternatively, the preset adjacent range may be a circular range with the pixel in the first row and first column of the preset adjacent image information as the starting point (center) and a preset pixel width as the radius. The preset adjacent image information may be the current image coding information or the adjacent image coding information. The preset pixel width may be 10 pixels. The current pixel group may be pixel information within a preset video frame that represents the preset adjacent range information. In practice, the execution entity may first determine the position coordinates of the pixel in the first row and first column of the current image coding information as the starting point position coordinates. Then, the execution entity may determine the preset adjacent range information with the starting point position coordinates as the adjacent range information. Finally, the execution entity may determine all pixel information within the adjacent range information in the preset video frame as the current pixel group. Specifically, when the above-mentioned preset adjacent range information is "a rectangular range with the first row and first column pixels of the preset adjacent image information as the starting point and the preset pixel width on the upper and left sides of the above-mentioned preset adjacent image information". The above-mentioned execution subject can determine the various pixel points in the horizontal direction corresponding to the first pixel point in the upper left corner of the above-mentioned current image coding information as the first row of pixels, and determine the various pixel points in the vertical direction corresponding to the first pixel point in the upper left corner of the above-mentioned current image coding information as the first column of pixels. The above-mentioned execution subject can use the first row and first column pixels of the current image coding information as the starting point, determine the number of pixels in the first row of the current image coding information as the length of the first rectangular range, and use the preset pixel width in the vertical upward direction as the width of the first rectangular range to obtain the first rectangular range. Then, the above-mentioned execution subject can determine the number of pixels in the first column of the current image coding information as the length of the second rectangular range, and use the preset pixel width in the horizontal left direction as the width of the second rectangular range to obtain the second rectangular range. Then, the execution entity may determine a third rectangular range by starting with the first row and first column pixel of the current image encoding information, setting a predetermined vertical width of pixels upward as the length of the third rectangular range, and setting a predetermined horizontal width of pixels leftward as the width of the third rectangular range. Finally, the execution entity combines the first, second, and third rectangular ranges into adjacent range information.

[0058] In the second step, for each adjacent image motion vector in the motion vector candidate list, perform the following steps:

[0059] In the first sub-step, the reference image coding information corresponding to the adjacent image motion vector is determined as the reference image coding information to be compared.

[0060] The second sub-step is to generate a reference pixel group corresponding to the reference image encoding information to be compared based on the encoding information of the reference image to be compared and the preset adjacent range information. The reference pixel group may be pixel information within the reference video frame that represents the preset adjacent range information. The method for generating the reference pixel group may refer to the method for generating the current pixel group.

[0061] The third sub-step is to generate a comparison difference based on the current pixel group and the reference pixel group. The comparison difference can be a numerical value used to characterize the pixel difference between the pixel group and the reference pixel group. In practice, the execution entity can input the current pixel group and the reference pixel group into a second preset formula to obtain the comparison difference. The second preset formula can be SAD = ∑ i，j P i，j -O i，j The above comparison difference can be represented by SAD. The above current pixel group can be represented by O. i，j It is represented by the pixel value of the current pixel in the i-th row and j-th column in the current pixel group. The above reference pixel group can be represented by P. i，j It is represented as the pixel value of the adjacent pixel in the i-th row and j-th column in the reference pixel group.

[0062] The third step is to sort the generated comparison differences to generate a comparison difference sequence. The comparison difference sequence may be a sequence of comparison differences sorted in a preset order. In practice, the execution entity may sort the comparison differences from largest to smallest to form the comparison difference sequence.

[0063] The fourth step is to reorder the motion vector candidate list based on the comparison difference sequence to update the motion vector candidate list. In practice, the execution entity may reorder the adjacent image motion vectors corresponding to the comparison difference values ​​according to the comparison difference sequence to obtain a motion vector candidate list to update the motion vector candidate list.

[0064] Optionally, after step 1045, the execution entity may further perform the following steps:

[0065] The first step is to generate a current motion vector corresponding to the current image encoding information based on the reference video frame and the current image encoding information. The current motion vector may be a vector representing the direction of the current image encoding information. The method for generating the current motion vector may refer to the method for generating the adjacent image motion vector.

[0066] The second step is to generate individual vector similarities based on the current motion vector and the updated motion vector candidate list. The vector similarities may be used to represent the degree of similarity between two vectors. In practice, the execution entity may determine the individual vector similarities as the cosine similarity between the current motion vector and each adjacent image motion vector in the updated motion vector candidate list.

[0067] In the third step, the adjacent image motion vector corresponding to the vector similarity that meets the preset vector similarity condition among the generated vector similarities is determined as the motion vector to be written. The preset vector similarity condition may be the vector similarity with the highest cosine similarity.

[0068] The fourth step is to store the motion vector to be written in a data storage server, wherein the data storage server may be a server for storing and managing video image related information.

[0069] Optionally, after step 104, the execution entity may further perform the following steps:

[0070] In the first step, for each of the updated motion vector candidate lists, the first-ranked motion vector in the motion vector candidate list is extracted as the candidate motion vector. In practice, the execution entity may determine the first-ranked adjacent image motion vector in each motion vector candidate list as the candidate motion vector.

[0071] The second step is to generate a ranked candidate motion vector sequence based on the extracted candidate motion vectors. The ranked candidate motion vector sequence may be a sequence representing the number of repetitions of each candidate motion vector. In practice, the execution entity may prioritize candidate motion vectors with the highest number of repetitions to generate the ranked candidate motion vector sequence.

[0072] In a third step, entropy coding is performed on the sorted candidate motion vector group to obtain a candidate motion vector code group, wherein each candidate motion vector code in the candidate motion vector code group may be an entropy coded code. Each candidate motion vector code in the candidate motion vector code group corresponds to a candidate motion vector. In practice, the execution entity may entropy code the sorted candidate motion vector group to obtain the candidate motion vector code group.

[0073] In the fourth step, for each image prediction coding unit information in the above image coding unit information set, the candidate motion vector encoding corresponding to the motion vector ranked first in the motion vector candidate list corresponding to the above image prediction coding unit information is determined as the video code stream encoding.

[0074] In the fifth step, the determined video code stream codes are combined to obtain video frame code data information. The video frame code data information may be information composed of entropy codes used to represent the video frame. In practice, the execution entity may combine the determined video code stream codes into video frame code data information.

[0075] Step 6: Store the above video frame encoding data information in a data storage server.

[0076] Step 7: Generate image code stream data based on the video frame encoding data information, the preset video frame, and the candidate motion vector encoding group. The image code stream data may be a code stream used to represent a video image. In practice, the execution entity may process the video frame encoding data information, the preset video frame, and the candidate motion vector encoding group into image code stream data based on the video encoding information. The video encoding information may be H.264, H.265 / HEVC, or VP9.

[0077] In step 8, the image code stream data is sent to a target terminal, so that the target terminal decodes the received image code stream data. The target terminal may be a mobile phone or a server. In practice, the execution entity may send the image code stream data to the target terminal.

[0078] In the process of adopting technical solutions to solve the above technical problems, the following problems often arise:

[0079] When parsing the bitstream data, each current block needs to be parsed through the motion vector candidate list. Since the index order of the motion vector candidate list is fixed, the time spent in the query encoding process is relatively even, resulting in a longer overall parsing time.

[0080] Faced with the above technical problems, we decided to adopt the following solutions:

[0081] In some optional implementations of some embodiments, the image code stream data is decoded through the following steps:

[0082] The first step is to receive image code stream data and record the time at which the image code stream data is received. The target terminal may be the entity that decodes the image code stream data. The reception time may be the time at which the image code stream data is received. In practice, the target terminal may first receive the image code stream data. Then, the target terminal may determine the time at which the image code stream data is received as the reception time.

[0083] The second step is to perform decoding processing based on the above-mentioned image code stream data to generate image decoding data information. The above-mentioned image decoding data information may be information used to represent the image code stream data after decoding. The above-mentioned image decoding data information may include image video frames, image motion vector coding groups, and image video frame coding data information. The above-mentioned image video frame coding data information includes the encoding of each video code stream. The above-mentioned image video frames may be the above-mentioned preset video frames. The above-mentioned image motion vector coding groups may be the above-mentioned candidate motion vector coding groups. The above-mentioned image video frame coding data information may be the above-mentioned video frame coding data information. In practice, first, the above-mentioned target terminal may determine the video decoding method based on the video decoding information included in the above-mentioned video coding information. The above-mentioned video decoding method may be algorithm-related information used for video decoding. Then, the above-mentioned target terminal may decode the above-mentioned image code stream data according to the above-mentioned video decoding method to obtain the image decoding data information.

[0084] The third step is to determine the image segmentation unit based on the pre-stored video encoding information. The method for determining the image segmentation unit can refer to step 102.

[0085] In the fourth step, the image video frame is segmented according to the image segmentation unit to obtain information of each image prediction coding unit. The method of segmenting the image video frame to obtain information of each image prediction coding unit can be referred to in step 102.

[0086] In the fifth step, for each image prediction coding unit information in the above-mentioned image prediction coding unit information, the video code stream code corresponding to the above-mentioned image prediction coding unit information in the above-mentioned image video frame coding data information is determined as the code to be decoded.

[0087] The sixth step is to generate a video image frame based on the determined codes to be decoded and the above-mentioned image motion vector coding group, and record the generation time of the generated video image frame. The above-mentioned generation time may be the time when the video image frame is generated. In practice, first, the above-mentioned target terminal may process the determined codes to be decoded and the above-mentioned image motion vector coding group into a video image frame according to the above-mentioned video coding information. The above-mentioned video coding information may be H.264, H.265 / HEVC or VP9. Then, the above-mentioned target terminal determines the time when the video image frame is generated as the generation time.

[0088] Step 7: Generate a processing time based on the generation time and the reception time. The processing time may be the time required to process the image code stream data into a video image frame. In practice, the target terminal may subtract the reception time from the generation time to obtain the processing time.

[0089] In the eighth step, the generated video image frame and the above processing time are stored in a data storage server.

[0090] The above technical solution, as an inventive feature of an embodiment of the present disclosure, addresses the technical problem that when parsing bitstream data, each current block needs to be parsed using a motion vector candidate list. Because the motion vector candidate list index order is fixed, the query code time is relatively evenly distributed, resulting in a long overall parsing time. Factors contributing to the long overall parsing time are often as follows: The motion vector candidate list index order is fixed, resulting in a relatively evenly distributed query code time. Resolving this factor can reduce the overall parsing time. To achieve this, the image bitstream data of the present disclosure is decoded through the following steps: first, receiving image bitstream data and recording the time the image bitstream data was received. This allows the image bitstream data to be received and the time the image bitstream data was received to be recorded. Then, decoding is performed based on the image bitstream data to generate decoded image data information, where the decoded image data information includes image video frames, image motion vector code groups, and image video frame code data information, which includes individual video bitstream codes. Next, image segmentation units are determined based on pre-stored video code information. Next, the image segmentation unit segments the image video frame to obtain information about individual image prediction coding units. Then, for each of the individual image prediction coding units, the video code stream code corresponding to the image prediction coding unit information in the image video frame encoding data information is determined as the code to be decoded. This allows the received image code stream data to be decoded to obtain generated information. Next, based on the determined codes to be decoded and the image motion vector code group, a video frame is generated, and the time of generation of the video frame is recorded. This allows the video frame to be obtained based on the codes to be decoded and the image motion vector code group, as well as the time of generation of the video frame. Next, a processing time is generated based on the generation time and the reception time. Finally, the generated video frame and the processing time are stored in a data storage server. This allows the video frame and processing time to be obtained. Because the image motion vector code group is obtained by decoding the received image code stream data, the resulting image motion vector code group is a sequenced code group. Since the sorted image motion vector encoding group places motion vectors with a higher probability of occurrence at the front, using the sorted image motion vector encoding group can sort motion vectors with a higher frequency of occurrence at the front, thereby reducing the time spent searching for motion vectors with a higher frequency of occurrence. This can reduce the average time spent searching for encodings, thereby reducing the overall parsing time.

[0091] The above-described embodiments of the present disclosure have the following beneficial effects: The motion vector candidate list sorting method of some embodiments of the present disclosure can adjust the motion vector candidate list, reducing the number of bits consumed when writing motion vectors into the bitstream. Specifically, the high bit consumption in writing motion vectors into the bitstream is caused by selecting motion vectors from neighboring blocks in a fixed order to construct the motion vector candidate list. The resulting motion vector candidate list index order is also fixed, but the index order is not adjusted during use. As a result, the probability of selecting each index during motion vector encoding is relatively even, resulting in a large entropy value corresponding to each code during entropy encoding. Consequently, a large number of binary codewords are consumed, resulting in a high bit consumption when writing motion vectors into the bitstream and low video compression efficiency. Based on this, the motion vector candidate list sorting method of some embodiments of the present disclosure first receives a preset video frame. This obtains a video frame image for generating a motion vector candidate list. Then, the preset video frame is encoded to obtain a set of image coding unit information corresponding to the preset video frame. The obtained video frame image can then be segmented to obtain information about each image prediction coding unit. Next, a reference video frame corresponding to the preset video frame is obtained. This allows the reference video frame corresponding to the preset video frame for generating the motion vector candidate list to be obtained. Next, for each image prediction coding unit information in the image coding unit information set, the following steps are performed: The image prediction coding unit information is determined as the current image coding information. This allows the image prediction coding unit information to be determined as the current image coding information to be processed. Next, based on the current image coding information, each image prediction coding unit information in the image coding unit information set that meets a preset adjacency condition is determined as each adjacent image coding information. This allows the adjacent image coding information corresponding to the current image coding information to be obtained. Next, based on the reference video frame and each adjacent image coding information, adjacent image motion vectors corresponding to each adjacent image coding information are generated. This allows the adjacent image motion vectors corresponding to the current image coding information to be generated. Next, a motion vector candidate list is constructed based on the generated adjacent image motion vectors. This allows the motion vector candidate list corresponding to the current image coding information to be obtained. Finally, the motion vector candidate list is reordered to update the motion vector candidate list. Therefore, the obtained motion vector candidate list can be sorted to obtain an updated motion vector candidate list.

[0092] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a motion vector candidate list sorting device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0093] like Figure 2 As shown, some embodiments of the webpage generation device 200 include: a receiving unit 201, an encoding processing unit 202, an acquisition unit 203, and an execution unit 204. The receiving unit 201 is configured to receive a preset video frame; the encoding processing unit 202 is configured to perform encoding processing on the preset video frame to obtain an image coding unit information set corresponding to the preset video frame; the acquisition unit 203 is configured to obtain a reference video frame corresponding to the preset video frame; and the execution unit 204 is configured to perform the following steps for each image prediction coding unit information in the image coding unit information set: determining the image prediction coding unit information as current image coding information; based on the current image coding information, determining each image prediction coding unit information in the image coding unit information set that meets a preset adjacent condition as each adjacent image coding information; generating each adjacent image motion vector corresponding to each adjacent image coding information based on the reference video frame and the each adjacent image coding information; constructing a motion vector candidate list based on the generated adjacent image motion vectors; and reordering the motion vector candidate list to update the motion vector candidate list.

[0094] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be repeated here.

[0095] Reference below Figure 3 , which shows a structural diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0096] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0097] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0098] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0099] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0100] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0101] The computer-readable medium may be included in the electronic device, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the electronic device: receives a preset video frame; encodes the preset video frame to obtain an image coding unit information set corresponding to the preset video frame; obtains a reference video frame corresponding to the preset video frame; and, for each image prediction coding unit information in the image coding unit information set, performs the following steps: determines the image prediction coding unit information as current image coding information; based on the current image coding information, determines each image prediction coding unit information in the image coding unit information set that meets a preset adjacent condition as each adjacent image coding information; generates each adjacent image motion vector corresponding to each adjacent image coding information based on the reference video frame and the each adjacent image coding information; constructs a motion vector candidate list based on the generated adjacent image motion vectors; and reorders the motion vector candidate list to update the motion vector candidate list.

[0102] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The units described in some embodiments of the present disclosure may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as: a processor including a receiving unit, an encoding processing unit, an acquisition unit, and an execution unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the receiving unit may also be described as a "unit for receiving a preset video frame."

[0105] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0106] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for sorting a motion vector candidate list, comprising: Receive a preset video frame; Performing encoding processing on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; Acquire a reference video frame corresponding to the preset video frame; For each image prediction coding unit information in the image coding unit information set, perform the following steps: Determining the image prediction coding unit information as current image coding information; Based on the current image coding information, determining each image prediction coding unit information that meets a preset adjacent condition in the image coding unit information set as each adjacent image coding information; generating, based on the reference video frame and the respective adjacent image coding information, respective adjacent image motion vectors corresponding to the respective adjacent image coding information; Constructing a motion vector candidate list based on the generated motion vectors of each adjacent image; The motion vector candidate list is reordered to update the motion vector candidate list.

2. The method according to claim 1, wherein The encoding process is performed on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame, including: Determining an image segmentation unit based on pre-stored video encoding information; According to the image segmentation unit, the preset video frame is segmented to obtain information of each image prediction coding unit; The information of each image prediction coding unit is determined as an image coding unit information set.

3. The method according to claim 1, wherein The step of generating, based on the reference video frame and the adjacent image coding information, respective adjacent image motion vectors corresponding to the respective adjacent image coding information comprises: For each piece of adjacent image coding information, perform the following steps: generating respective candidate range information based on the adjacent image coding information, the reference video frame and a preset search range; generating respective similarities corresponding to the adjacent image coding information based on the adjacent image coding information and the respective candidate range information; Determine the similarity that meets the preset similarity condition among the similarities as the proximity similarity; Determining the candidate range information corresponding to the proximity similarity as reference image coding information; An adjacent image motion vector is generated based on the adjacent image encoding information and the reference image encoding information.

4. The method according to claim 3, wherein: The generating of the adjacent image motion vector based on the adjacent image coding information and the reference image coding information includes: generating adjacent coordinates corresponding to the adjacent image coding information based on the adjacent image coding information and the preset video frame; generating, based on the reference image coding information and the reference video frame, reference coordinates corresponding to the reference image coding information; Based on the adjacent coordinates and the reference coordinates, an adjacent image motion vector is generated.

5. The method according to claim 1, wherein The reordering of the motion vector candidate list to update the motion vector candidate list includes: generating a current pixel group corresponding to the current image coding information based on the current image coding information and preset adjacent range information; For each adjacent image motion vector in the motion vector candidate list, perform the following steps: Determining the reference image coding information corresponding to the adjacent image motion vector as the reference image coding information to be compared; generating a reference pixel group corresponding to the reference image coding information to be compared based on the reference image coding information to be compared and the preset adjacent range information; generating a comparison difference value based on the current pixel group and the reference pixel group; Sorting the generated comparison differences to generate a comparison difference sequence; The motion vector candidate list is reordered based on the comparison difference sequence to update the motion vector candidate list.

6. The method according to claim 1, wherein After reordering the motion vector candidate list to update the motion vector candidate list, the method further includes: generating a current motion vector corresponding to the current image coding information based on the reference video frame and the current image coding information; generating respective vector similarities based on the current motion vector and the updated motion vector candidate list; Determining the adjacent image motion vector corresponding to the vector similarity that meets the preset vector similarity condition among the generated vector similarities as the motion vector to be written; The motion vector to be written is stored in a data storage server.

7. A motion vector candidate list sorting device, comprising: A receiving unit, configured to receive a preset video frame; an encoding processing unit, configured to perform encoding processing on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; an acquiring unit, configured to acquire a reference video frame corresponding to the preset video frame; The execution unit is configured to perform the following steps for each image prediction coding unit information in the image coding unit information set: determining the image prediction coding unit information as current image coding information; Based on the current image coding information, determining each image prediction coding unit information that meets a preset adjacent condition in the image coding unit information set as each adjacent image coding information; generating, based on the reference video frame and the respective adjacent image coding information, respective adjacent image motion vectors corresponding to the respective adjacent image coding information; and constructing a motion vector candidate list based on the respective generated adjacent image motion vectors; The motion vector candidate list is reordered to update the motion vector candidate list.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images