Candidate list sorting method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses a candidate list sorting method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: receiving a preset video frame; encoding the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; obtaining a reference video frame corresponding to the preset video frame; determining the image predictive coding unit information as current image coding information; determining each piece of image prediction coding unit information meeting a preset adjacent condition in the image coding unit information set as each piece of adjacent image coding information; generating each adjacent image motion vector corresponding to each adjacent image coding information; constructing a motion vector candidate list; and performing reordering processing on the motion vector candidate list so as to update the motion vector candidate list. According to the embodiment, bits consumed by writing the motion vector into the code stream in the video transmission process are reduced, and the video compression efficiency is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a candidate list sorting method, device, electronic device, and computer-readable medium. Background Art

[0002] Video coding and decoding technology is commonly used in the process of video transmission. Among them, the candidate list of inter-frame prediction is a motion vector used in video coding and decoding to describe the correlation between blocks in adjacent image frames. When encoding a block of an image frame, a block similar to the current block is found in an adjacent image frame as the prediction value of the current block. The block being encoded is called the current block, and the similar block in the adjacent image frame is called the reference block. In order to describe the position of the reference block, the position of the reference frame is usually expressed by the reference frame index, and the position of the reference block is expressed by the motion vector between the current block and the reference block. When writing the video image into the bitstream during video transmission, it is only necessary to write the motion vector index in the motion vector candidate list into the bitstream, so that the motion vector does not need to be written directly. The commonly used method for obtaining the motion vector candidate list is: determine each neighboring block around the current block, and then select the motion vector of the neighboring block in a fixed order to construct the motion vector candidate list.

[0003] However, when the above method is used to establish a motion vector candidate list, the following technical problems often occur:

[0004] The motion vector candidate list is constructed by selecting the motion vectors of neighboring blocks in a fixed order. The index order of the generated motion vector candidate list is also fixed, but the index order is not adjusted during use. Therefore, the probability of selecting each index in the process of encoding the motion vector is relatively even, which makes the entropy value corresponding to each code larger in the entropy encoding process, so the consumed binary codewords are larger, resulting in more bits consumed in writing the motion vector into the bitstream and lower video compression efficiency.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a method, apparatus, electronic device, and computer-readable medium for sorting a motion vector candidate list to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In the first aspect, some embodiments of the present disclosure provide a method for sorting a motion vector candidate list, the method comprising: receiving a preset video frame; encoding the above-mentioned preset video frame to obtain an image coding unit information set corresponding to the above-mentioned preset video frame; obtaining a reference video frame corresponding to the above-mentioned preset video frame; for each image prediction coding unit information in the above-mentioned image coding unit information set, performing the following steps: determining the above-mentioned image prediction coding unit information as the current image coding information; based on the above-mentioned current image coding information, determining each image prediction coding unit information in the above-mentioned image coding unit information set that meets a preset adjacent condition as each adjacent image coding information; based on the above-mentioned reference video frame and the above-mentioned each adjacent image coding information, generating each adjacent image motion vector corresponding to the above-mentioned each adjacent image coding information; based on the generated each adjacent image motion vector, constructing a motion vector candidate list; reordering the above-mentioned motion vector candidate list to update the above-mentioned motion vector candidate list.

[0009] In a second aspect, some embodiments of the present disclosure provide a motion vector candidate list sorting device, the device comprising: a receiving unit, configured to receive a preset video frame; an encoding processing unit, configured to perform encoding processing on the above-mentioned preset video frame to obtain an image coding unit information set corresponding to the above-mentioned preset video frame; an acquisition unit, configured to acquire a reference video frame corresponding to the above-mentioned preset video frame; an execution unit, configured to perform the following steps for each image prediction coding unit information in the above-mentioned image prediction coding unit information set: determine the above-mentioned image prediction coding unit information as current image coding information; based on the above-mentioned current image coding information, determine each image prediction coding unit information in the above-mentioned image coding unit information set that meets a preset adjacent condition as each adjacent image coding information; based on the above-mentioned reference video frame and the above-mentioned each adjacent image coding information, generate each adjacent image motion vector corresponding to the above-mentioned each adjacent image coding information; based on the generated each adjacent image motion vector, construct a motion vector candidate list; reorder the above-mentioned motion vector candidate list to update the above-mentioned motion vector candidate list.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the motion vector candidate list sorting method of some embodiments of the present disclosure, the motion vector candidate list can be adjusted, and the bits consumed in writing the motion vector into the bitstream are reduced. Specifically, the reason for consuming more bits in writing the motion vector into the bitstream is that the motion vectors of the neighboring blocks are selected in a fixed order to construct the motion vector candidate list, and the index order of the generated motion vector candidate list is also fixed, but the index order is not adjusted during use. Therefore, the probability of selecting each index in the process of encoding the motion vector is relatively even, so that the entropy value corresponding to each encoding in the entropy encoding process is large, so the consumed binary codeword is large, resulting in more bits consumed in writing the motion vector into the bitstream, and the video compression efficiency is low. Based on this, the motion vector candidate list sorting method of some embodiments of the present disclosure, first, receives a preset video frame. Thus, a video frame image to generate a motion vector candidate list can be obtained. Then, the above-mentioned preset video frame is encoded to obtain an image encoding unit information set corresponding to the above-mentioned preset video frame. Thus, the obtained video frame image can be segmented to obtain each image prediction encoding unit information. Then, a reference video frame corresponding to the preset video frame is obtained. Thus, the reference video frame corresponding to the preset video frame for generating the motion vector candidate list can be obtained. Secondly, for each image prediction coding unit information in the image coding unit information set, the following steps are performed: the image prediction coding unit information is determined as the current image coding information. Thus, the image prediction coding unit information can be determined as the current image coding information to be processed. Then, based on the current image coding information, each image prediction coding unit information that meets the preset adjacent condition in the image coding unit information set is determined as each adjacent image coding information. Thus, each adjacent image coding information corresponding to the current image coding information can be obtained. Then, based on the reference video frame and each adjacent image coding information, each adjacent image motion vector corresponding to each adjacent image coding information is generated. Thus, each adjacent image motion vector corresponding to the current image coding information can be generated. Secondly, based on each generated adjacent image motion vector, a motion vector candidate list is constructed. Thus, the motion vector candidate list corresponding to the current image coding information can be obtained. Finally, the motion vector candidate list is reordered to update the motion vector candidate list. Therefore, the obtained motion vector candidate list can be sorted to obtain an updated motion vector candidate list.Also, because the motion vector candidate lists generated for each image prediction coding unit information are sorted to obtain each updated motion vector candidate list, the motion vectors sorted earlier in each motion vector candidate list have a higher probability of being written into the bitstream, so that the encoding value allocated when the motion vector is entropy encoded in the subsequent operation of writing into the bitstream is smaller, thereby reducing the bits consumed in the bitstream. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flowchart of some embodiments of the method for sorting a motion vector candidate list according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of the motion vector candidate list sorting device according to the present disclosure;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 , shows a process 100 of some embodiments of the motion vector candidate list sorting method according to the present disclosure. The motion vector candidate list sorting method comprises the following steps:

[0024] Step 101: Receive a preset video frame.

[0025] In some embodiments, the execution subject (e.g., a computing device) of the motion vector candidate list sorting method may receive a preset video frame. The preset video frame may be a frame of video image in the video data. The execution subject may be a server for managing and storing video data. In practice, first, the execution subject may receive video data sent by the server. The video data may be video data composed of various video frames. Then, the execution subject may determine the various video frames in the video data as various video frames to be processed according to preset extraction conditions. The preset extraction conditions may be to extract one video frame every other video frame. Finally, for each of the various video frames to be processed, the execution subject may determine the video frame to be processed as a preset video frame.

[0026] Step 102: Encode the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame.

[0027] In some embodiments, the execution subject may perform encoding processing on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame. Each image prediction encoding unit information in the image encoding unit information set may be pixel information composed of an image of a preset size in the preset video frame. The preset size may be pre-set and is not limited here. For example, an image prediction encoding unit information may be pixel information of a PU (prediction unit).

[0028] In some optional implementations of some embodiments, the execution subject may perform encoding processing on the preset video frame through the following steps to obtain an image encoding unit information set corresponding to the preset video frame:

[0029] The first step is to determine the image segmentation unit based on the pre-stored video coding information. The video coding information may be information related to the algorithm used to compress and decompress digital video. For example, the video coding information may be H.264, H.265 / HEVC or VP9. The video coding information may include: segmentation unit information, segmentation method information and video decoding information. The segmentation unit information may be a coding standard for video frame image segmentation. For example, the segmentation unit information may be a PU of 8x8, 16x16 pixels or a MB (Macroblock) of 16x16 pixels. The segmentation method information may be text information of a segmentation method corresponding to the segmentation unit information for segmenting the video frame image. For example, when the segmentation unit information is "PU of 16x16 pixels", the segmentation method information may be "starting from the pixels of the first row and first column of the video frame, the video frame is segmented with 16 pixels as horizontal and vertical intervals". The video decoding information may be information related to the algorithm used for video decoding. The image segmentation unit may be a coding standard for video frame image segmentation. In practice, the execution entity may determine the segmentation unit information included in the video encoding information as the image segmentation unit.

[0030] In the second step, according to the above-mentioned image segmentation unit, the above-mentioned preset video frame is segmented to obtain information of each image prediction coding unit. In practice, first, the above-mentioned execution subject can determine the segmentation method information included in the video coding information corresponding to the above-mentioned image segmentation unit as the image segmentation method. Then, the above-mentioned execution subject can determine the number of interval pixels through the text information contained in the image segmentation method. As an example, when the above-mentioned image segmentation method is "starting from the pixels of the first row and first column of the video frame, the video frame is segmented with 16 pixels as the horizontal interval and vertical interval", the corresponding number of interval pixels is "16"; when the above-mentioned image segmentation method is "starting from the pixels of the first row and first column of the video frame, the video frame is segmented with 32 pixels as the horizontal interval and vertical interval", the corresponding number of interval pixels is "32". Then, the above-mentioned execution subject can start from the pixels of the first row and first column of the preset video frame, and horizontally and vertically segment the above-mentioned preset video frame with the above-mentioned number of interval pixels as the interval to obtain each horizontal strip and each vertical strip. Secondly, the above-mentioned execution subject can determine the intersection area of ​​each horizontal strip and each vertical strip as each image coding unit. Finally, the execution subject may determine the information of all pixels included in each image coding unit in each image coding unit as image prediction coding unit information to obtain information of each image prediction coding unit. The number of interval pixels may be used to represent the number of pixels required to be spaced between each two segmentations when segmenting the preset video frame.

[0031] The third step is to determine the above-mentioned image prediction coding unit information as an image coding unit information set.

[0032] Step 103: Acquire a reference video frame corresponding to the preset video frame.

[0033] In some embodiments, the execution subject may obtain a reference video frame corresponding to the preset video frame. The reference video frame may be the second frame of video image after the preset video frame in the video data. In practice, the execution subject may obtain the second frame of video image after the preset video frame from the video data as the reference video frame.

[0034] Step 104, for each image prediction coding unit information in the image coding unit information set, perform the following steps:

[0035] Step 1041: determine the image prediction coding unit information as the current image coding information.

[0036] In some embodiments, the execution entity may determine the image prediction coding unit information as current image coding information.

[0037] Step 1042: Based on the current image coding information, each image prediction coding unit information satisfying a preset adjacent condition in the image coding unit information set is determined as each adjacent image coding information.

[0038] In some embodiments, the execution subject may determine the image prediction coding unit information in the image coding unit information set that satisfies the preset adjacent condition as each adjacent image coding information based on the current image coding information. The preset adjacent condition may be the image prediction coding unit information adjacent to the current image coding information. For example, the preset adjacent condition may be the image prediction coding unit information adjacent to the upper side, lower side, left side or right side of the current image coding information. In practice, the execution subject may find the image prediction coding unit information adjacent to the current image coding information and located on the upper side or left side of the current image coding information from the image coding unit information set and determine it as each adjacent image coding information.

[0039] Step 1043: Generate adjacent image motion vectors corresponding to adjacent image coding information based on the reference video frame and adjacent image coding information.

[0040] In some embodiments, the execution subject may generate each adjacent image motion vector corresponding to each adjacent image encoding information based on the reference video frame and each adjacent image encoding information, wherein the adjacent image motion vector may be a vector used to characterize the direction of the adjacent image encoding information.

[0041] In some optional implementations of some embodiments, the execution subject may generate each adjacent image motion vector corresponding to each adjacent image encoding information based on the reference video frame and each adjacent image encoding information through the following steps:

[0042] The first step is to perform the following steps on each adjacent image coding information in the above adjacent image coding information:

[0043] The first sub-step is to generate each candidate range information based on the adjacent image coding information, the reference video frame and the preset search range. The preset search range may be a preset shape range centered at a preset position. The preset position may be a coordinate. For example, the preset search range may be a square area centered on the pixels of the first row and first column of the current image coding information and with a preset number of pixels as the side length. Or the preset search range may be a circular area centered on the pixels of the first row and first column of the adjacent image coding information and with a preset number of pixels as the radius. The candidate range information may be pixel information composed of an image of a preset size in the reference video frame. The preset number of pixels and the preset size may be pre-set and are not limited here. In practice, first, the execution subject may obtain the position coordinates of the vector image coding information. Then, the execution subject may find the position coordinates from the reference video frame as the starting point of the preset search range. Then, the execution subject may obtain a preset number of search centers in the left or upward direction at a preset number of pixels from the starting point. The search center may be the position coordinates of the pixels in the reference video frame at a preset number of pixels from the starting point to the left or upward direction. Secondly, for each of the above-mentioned search centers, the above-mentioned execution subject can determine a preset search range with the above-mentioned search center as a preset position as a candidate range, and obtain each candidate range. Finally, the above-mentioned execution subject can determine information of all pixels included in each of the candidate ranges in the above-mentioned reference video frame as candidate range information, and obtain each candidate range information.

[0044] The second sub-step is to generate each similarity corresponding to the adjacent image coding information based on the adjacent image coding information and the candidate range information. In practice, the execution subject can calculate the similarity between the adjacent image coding information and the candidate range information through a similarity algorithm to obtain each similarity. The similarity algorithm can be the sum of absolute differences (SAD), mean square error (MSE) or normalized cross-correlation (NCC).

[0045] In the third sub-step, the similarity that satisfies a preset similarity condition among the various similarities is determined as the proximity similarity. The preset similarity condition may be the highest similarity.

[0046] The fourth sub-step is to determine the candidate range information corresponding to the above-mentioned proximity similarity as the reference image coding information.

[0047] The second step is to generate an adjacent image motion vector based on the adjacent image coding information and the reference image coding information.

[0048] In some optional implementations of some embodiments, the execution subject may generate an adjacent image motion vector based on the adjacent image coding information and the reference image coding information through the following steps:

[0049] The first step is to generate adjacent coordinates corresponding to the adjacent image coding information based on the adjacent image coding information and the preset video frame. The adjacent coordinates may be coordinates used to represent the position of the adjacent image coding information in the preset video frame. In practice, the execution entity may determine the coordinate position of the pixel in the first row and first column of the adjacent image coding information in the preset video frame as the adjacent coordinates.

[0050] The second step is to generate reference coordinates corresponding to the reference image coding information based on the reference image coding information and the reference video frame. The reference coordinates may be coordinates used to represent the position of the reference image coding information in the reference video frame. In practice, the execution subject may determine the coordinate position of the pixel in the first row and first column of the reference image coding information in the reference video frame as the reference coordinates.

[0051] The third step is to generate an adjacent image motion vector based on the adjacent coordinates and the reference coordinates. In practice, the execution entity may input the adjacent coordinates and the reference coordinates into a first preset formula to obtain an adjacent image motion vector. The first preset formula may be A=(x2-x1, y2-y1). The adjacent image motion vector may be represented by A. The adjacent coordinates may be represented by (x1, y1. The reference coordinates may be represented by (x2, y2).

[0052] Step 1044: construct a motion vector candidate list based on the generated motion vectors of each adjacent image.

[0053] In some embodiments, the execution subject may construct a motion vector candidate list based on the generated motion vectors of each adjacent image. The motion vector candidate list may be a list composed of the motion vectors of each adjacent image. In practice, the execution subject may combine the generated motion vectors of each adjacent image into a motion vector candidate list.

[0054] Step 1045: reorder the motion vector candidate list to update the motion vector candidate list.

[0055] In some embodiments, the execution entity may reorder the motion vector candidate list to update the motion vector candidate list.

[0056] In some optional implementations of some embodiments, the execution subject may reorder the motion vector candidate list through the following steps to update the motion vector candidate list:

[0057] The first step is to generate a current pixel group corresponding to the current image coding information based on the current image coding information and the preset adjacent range information. The preset adjacent range information may be a preset shape range. For example, the preset adjacent range information may be a rectangular range with the first row and first column of the preset adjacent image information as the starting point and the preset pixel width on the upper and left sides of the preset adjacent image information. Alternatively, the preset adjacent range may be a circular range with the first row and first column of the preset adjacent image information as the starting point (center of the circle) and the preset pixel width as the radius. The preset adjacent image information may be the current image coding information or the adjacent image coding information. The preset pixel width may be 10 pixels. The current pixel group may be pixel information in a preset video frame used to characterize the inside of the preset adjacent range information. In practice, first, the execution subject may determine the position coordinates of the first row and first column of the current image coding information as the starting point position coordinates. Then, the execution subject may determine the preset adjacent range information with the starting point position coordinates as the adjacent range information. Finally, the execution subject may determine all pixel information within the adjacent range information in the preset video frame as the current pixel group. Specifically, when the above-mentioned preset adjacent range information is "a rectangular range with the first row and first column of pixels of the preset adjacent image information as the starting point and the preset pixel width on the upper and left sides of the above-mentioned preset adjacent image information". The above-mentioned execution subject can determine the various pixel points in the horizontal direction corresponding to the first pixel point in the upper left corner of the above-mentioned current image coding information as the first row of pixels, and determine the various pixel points in the vertical direction corresponding to the first pixel point in the upper left corner of the above-mentioned current image coding information as the first column of pixels. The above-mentioned execution subject can take the first row and first column of pixels of the current image coding information as the starting point, determine the number of pixels in the first row of the current image coding information as the length of the first rectangular range, and use the preset pixel width in the vertical upward direction as the width of the first rectangular range to obtain the first rectangular range. Then, the above-mentioned execution subject can determine the number of pixels in the first column of the current image coding information as the length of the second rectangular range, and use the preset pixel width in the horizontal left direction as the width of the second rectangular range to obtain the second rectangular range. Then, the execution subject may take the first row and first column pixel of the current image coding information as the starting point, the preset pixel width in the vertical upward direction as the length of the third rectangular range, and the preset pixel width in the horizontal left direction as the width of the third rectangular range to obtain a third rectangular range. Finally, the execution subject combines the first rectangular range, the second rectangular range and the third rectangular range into adjacent range information.

[0058] In the second step, for each adjacent image motion vector in the above motion vector candidate list, perform the following steps:

[0059] In the first sub-step, the reference image coding information corresponding to the adjacent image motion vector is determined as the reference image coding information to be compared.

[0060] The second sub-step is to generate a reference pixel group corresponding to the reference image coding information to be compared based on the reference image coding information to be compared and the preset adjacent range information. The reference pixel group may be pixel information in the reference video frame used to characterize the preset adjacent range information. The generation method of the reference pixel group may refer to the generation method of the current pixel group.

[0061] The third sub-step is to generate a comparison difference based on the current pixel group and the reference pixel group. The comparison difference may be a numerical value used to characterize the pixel difference between the pixel group and the reference pixel group. In practice, the execution subject may input the current pixel group and the reference pixel group into a second preset formula to obtain a comparison difference. The second preset formula may be SAD=∑ i，j P i，j -O i，j . The above comparison difference can be represented by SAD. The above current pixel group can be represented by O. i，j It is represented by the pixel value of the current pixel in the i-th row and j-th column in the current pixel group. The above reference pixel group can be represented by P. P i，j Represented as the pixel value of the adjacent pixel in the i-th row and j-th column in the reference pixel group.

[0062] The third step is to sort the generated comparison differences to generate a comparison difference sequence. The comparison difference sequence may be a sequence of comparison differences sorted in a preset order. In practice, the execution subject may sort the comparison differences from large to small to form a comparison difference sequence.

[0063] The fourth step is to reorder the motion vector candidate list based on the comparison difference sequence to update the motion vector candidate list. In practice, the execution subject can reorder the adjacent image motion vectors corresponding to the comparison difference according to the comparison difference sequence to obtain a motion vector candidate list to update the motion vector candidate list.

[0064] Optionally, after step 1045, the execution entity may further perform the following steps:

[0065] The first step is to generate a current motion vector corresponding to the current image coding information based on the reference video frame and the current image coding information. The current motion vector may be a vector used to characterize the direction of the current image coding information. The method for generating the current motion vector may refer to the method for generating the adjacent image motion vector.

[0066] The second step is to generate each vector similarity based on the current motion vector and the updated motion vector candidate list. The vector similarity can be used to characterize the degree of similarity between two vectors. In practice, the execution subject can determine the cosine similarity between the current motion vector and each adjacent image motion vector in the updated motion vector candidate list as each vector similarity.

[0067] In the third step, the adjacent image motion vector corresponding to the vector similarity satisfying the preset vector similarity condition among the generated vector similarities is determined as the motion vector to be written. The preset vector similarity condition may be the vector similarity with the highest cosine similarity.

[0068] The fourth step is to store the motion vector to be written into a data storage server. The data storage server may be a server for storing and managing video image related information.

[0069] Optionally, after step 104, the execution subject may further perform the following steps:

[0070] In the first step, for each of the updated motion vector candidate lists, the first ranked motion vector in the motion vector candidate list is extracted as the candidate motion vector. In practice, the execution subject may determine the first ranked adjacent image motion vector in each motion vector candidate list as the candidate motion vector.

[0071] The second step is to generate a sequence of sorted candidate motion vectors based on the extracted candidate motion vectors. The sorted candidate motion vector group may be a sequence used to characterize the number of times each candidate motion vector is repeated. In practice, the execution subject may sort the candidate motion vectors with the largest number of repetitions in front of each candidate motion vector to obtain a sequence of sorted candidate motion vectors.

[0072] The third step is to perform entropy coding on the sorted candidate motion vector group to obtain a candidate motion vector coding group, wherein each candidate motion vector coding in the candidate motion vector coding group can be an entropy coded coding. Each candidate motion vector coding in the candidate motion vector coding group corresponds to a candidate motion vector. In practice, the execution subject can encode the sorted candidate motion vector group by entropy coding to obtain a candidate motion vector coding group.

[0073] In the fourth step, for each image prediction coding unit information in the above-mentioned image coding unit information set, the candidate motion vector encoding corresponding to the motion vector ranked first in the motion vector candidate list corresponding to the above-mentioned image prediction coding unit information is determined as the video code stream encoding.

[0074] The fifth step is to integrate and process the determined video code stream codes to obtain video frame coding data information. The video frame coding data information may be information composed of entropy coding for representing the video frame. In practice, the execution subject may combine the above-mentioned video code stream codes into video frame coding data information.

[0075] Step 6: Store the above video frame encoding data information in a data storage server.

[0076] Step 7: Generate image code stream data based on the video frame coding data information, the preset video frame and the candidate motion vector coding group. The image code stream data may be a code stream for representing a video image. In practice, the execution subject may process the video frame coding data information, the preset video frame and the candidate motion vector coding group into image code stream data according to the video coding information. The video coding information may be H.264, H.265 / HEVC or VP9.

[0077] Step 8: Send the image code stream data to the target terminal so that the target terminal decodes the received image code stream data. The target terminal may be a mobile phone or a server. In practice, the execution subject may send the image code stream data to the target terminal.

[0078] In the process of adopting technical solutions to solve the above technical problems, the following problems are often accompanied:

[0079] When parsing the bitstream data, each current block needs to be parsed through the motion vector candidate list. Because the index order of the motion vector candidate list is fixed, the time spent in the query encoding process is relatively even, resulting in a longer overall parsing time.

[0080] Faced with the above technical problems, we decided to adopt the following solutions:

[0081] In some optional implementations of some embodiments, the above-mentioned image code stream data is decoded by the following steps:

[0082] The first step is to receive the image code stream data and record the receiving time of the image code stream data. The execution subject of decoding the image code stream data may be the target terminal. The receiving time may be the time when the image code stream data is received. In practice, first, the target terminal may receive the image code stream data. Then, the target terminal may determine the time when the image code stream data is received as the receiving time.

[0083] The second step is to perform decoding processing based on the above-mentioned image code stream data to generate image decoding data information. Among them, the above-mentioned image decoding data information can be information obtained after the image code stream data is decoded. The above-mentioned image decoding data information may include image video frames, image motion vector coding groups, and image video frame coding data information, and the above-mentioned image video frame coding data information includes each video code stream coding. The above-mentioned image video frame may be the above-mentioned preset video frame. The above-mentioned image motion vector coding group may be the above-mentioned candidate motion vector coding group. The above-mentioned image video frame coding data information may be the above-mentioned video frame coding data information. In practice, first, the above-mentioned target terminal may determine the video decoding method based on the video decoding information included in the above-mentioned video coding information. Among them, the above-mentioned video decoding method may be algorithm-related information for video decoding. Then, the above-mentioned target terminal may decode the above-mentioned image code stream data according to the above-mentioned video decoding method to obtain the image decoding data information.

[0084] The third step is to determine the image segmentation unit based on the pre-stored video encoding information. The method for determining the image segmentation unit can refer to step 102.

[0085] The fourth step is to segment the image video frame according to the image segmentation unit to obtain information of each image prediction coding unit. The method of segmenting the image video frame to obtain information of each image prediction coding unit can refer to step 102.

[0086] In the fifth step, for each image prediction coding unit information in the above-mentioned image prediction coding unit information, the video code stream code corresponding to the above-mentioned image prediction coding unit information in the above-mentioned image video frame coding data information is determined as the code to be decoded.

[0087] The sixth step is to generate a video image frame based on the determined codes to be decoded and the above-mentioned image motion vector coding group, and record the generation time of the generated video image frame. The above-mentioned generation time may be the time when the video image frame is generated. In practice, first, the above-mentioned target terminal may process the determined codes to be decoded and the above-mentioned image motion vector coding group into a video image frame according to the above-mentioned video coding information. The above-mentioned video coding information may be H.264, H.265 / HEVC or VP9. Then, the above-mentioned target terminal determines the time when the video image frame is generated as the generation time.

[0088] Step 7: Generate processing time based on the generation time and the reception time. The processing time may be the time required to characterize the image code stream data to be processed into a video image frame. In practice, the target terminal may obtain the processing time by subtracting the reception time from the generation time.

[0089] In the eighth step, the generated video image frames and the above processing time are stored in a data storage server.

[0090] As an inventive point of an embodiment of the present disclosure, the above technical solution solves the technical problem that "when parsing the code stream data, each current block needs to be parsed through the motion vector candidate list. Because the index order of the motion vector candidate list is fixed, the time consumed in the query coding process is relatively even, resulting in a long overall parsing time". The factors that lead to a long overall parsing time are often as follows: the index order of the motion vector candidate list is fixed, so the time consumed in the query coding process is relatively even. If the above factors are solved, the effect of reducing the long overall parsing time can be achieved. In order to achieve this effect, the image code stream data of the present disclosure is decoded by the following steps: first, receiving the image code stream data, and recording the receiving time of the received image code stream data. Thus, the image code stream data can be received and the time when the image code stream data is received can be recorded. Then, decoding processing is performed based on the above image code stream data to generate image decoding data information, wherein the above image decoding data information includes image video frames, image motion vector coding groups, and image video frame coding data information, and the above image video frame coding data information includes each video code stream coding. Then, based on the pre-stored video coding information, the image segmentation unit is determined. Secondly, according to the above-mentioned image segmentation unit, the above-mentioned image video frame is segmented and processed to obtain each image prediction coding unit information. Then, for each image prediction coding unit information in the above-mentioned image prediction coding unit information, the video code stream code corresponding to the above-mentioned image prediction coding unit information in the above-mentioned image video frame coding data information is determined as the code to be decoded. Thus, the received image code stream data can be decoded and processed to obtain the generated information. Then, based on the determined codes to be decoded and the above-mentioned image motion vector coding group, a video image frame is generated, and the generation time of the generated video image frame is recorded. Thus, the video image frame can be obtained through each code to be decoded and the image motion vector coding group, and the time of generating the video image frame can be recorded. Secondly, based on the above-mentioned generation time and the above-mentioned receiving time, the processing time is generated. Finally, the generated video image frame and the above-mentioned processing time are stored in the data storage server. Thus, the video image frame and the processing time can be obtained. Also because the image motion vector coding group obtained after decoding is obtained by receiving the image code stream data, the image motion vector coding group obtained is the coding group after sorting processing. Since the image motion vector coding group after sorting processing puts the motion vector with higher probability of occurrence in front, the motion vector with higher frequency of occurrence can be sorted in front by using the image motion vector coding group after sorting processing, thereby reducing the query time of the motion vector with higher frequency of occurrence. Therefore, the average time consumption in the query coding process can be reduced, thereby reducing the overall parsing time consumption.

[0091] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the motion vector candidate list sorting method of some embodiments of the present disclosure, the motion vector candidate list can be adjusted, and the bits consumed in writing the motion vector into the bitstream are reduced. Specifically, the reason for consuming more bits in writing the motion vector into the bitstream is that the motion vectors of the neighboring blocks are selected in a fixed order to construct the motion vector candidate list, and the index order of the generated motion vector candidate list is also fixed, but the index order is not adjusted during use. Therefore, the probability of selecting each index in the process of encoding the motion vector is relatively even, so that the entropy value corresponding to each encoding in the entropy encoding process is large, so the consumed binary codeword is large, resulting in more bits consumed in writing the motion vector into the bitstream, and the video compression efficiency is low. Based on this, the motion vector candidate list sorting method of some embodiments of the present disclosure, first, receives a preset video frame. Thus, a video frame image to generate a motion vector candidate list can be obtained. Then, the above-mentioned preset video frame is encoded to obtain an image encoding unit information set corresponding to the above-mentioned preset video frame. Thus, the obtained video frame image can be segmented to obtain each image prediction encoding unit information. Then, a reference video frame corresponding to the preset video frame is obtained. Thus, the reference video frame corresponding to the preset video frame for generating the motion vector candidate list can be obtained. Secondly, for each image prediction coding unit information in the image coding unit information set, the following steps are performed: the image prediction coding unit information is determined as the current image coding information. Thus, the image prediction coding unit information can be determined as the current image coding information to be processed. Then, based on the current image coding information, each image prediction coding unit information that meets the preset adjacent condition in the image coding unit information set is determined as each adjacent image coding information. Thus, each adjacent image coding information corresponding to the current image coding information can be obtained. Then, based on the reference video frame and each adjacent image coding information, each adjacent image motion vector corresponding to each adjacent image coding information is generated. Thus, each adjacent image motion vector corresponding to the current image coding information can be generated. Secondly, based on each generated adjacent image motion vector, a motion vector candidate list is constructed. Thus, the motion vector candidate list corresponding to the current image coding information can be obtained. Finally, the motion vector candidate list is reordered to update the motion vector candidate list. Thus, the obtained motion vector candidate list can be sorted to obtain an updated motion vector candidate list.

[0092] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a motion vector candidate list sorting device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0093] like Figure 2 As shown, the webpage generation device 200 of some embodiments includes: a receiving unit 201, an encoding processing unit 202, an acquisition unit 203 and an execution unit 204. The receiving unit 201 is configured to receive a preset video frame; the encoding processing unit 202 is configured to perform encoding processing on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; the acquisition unit 203 is configured to obtain a reference video frame corresponding to the preset video frame; the execution unit 204 is configured to perform the following steps for each image prediction encoding unit information in the image encoding unit information set: determine the image prediction encoding unit information as current image encoding information; based on the current image encoding information, determine each image prediction encoding unit information in the image encoding unit information set that meets a preset adjacent condition as each adjacent image encoding information; based on the reference video frame and each adjacent image encoding information, generate each adjacent image motion vector corresponding to each adjacent image encoding information; based on each generated adjacent image motion vector, construct a motion vector candidate list; reorder the motion vector candidate list to update the motion vector candidate list.

[0094] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be described in detail here.

[0095] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0096] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0097] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0098] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0099] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0100] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0101] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives a preset video frame; encodes the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; obtains a reference video frame corresponding to the preset video frame; performs the following steps for each image prediction encoding unit information in the image encoding unit information set: determines the image prediction encoding unit information as the current image encoding information; based on the current image encoding information, determines each image prediction encoding unit information in the image encoding unit information set that meets the preset adjacent condition as each adjacent image encoding information; based on the reference video frame and each adjacent image encoding information, generates each adjacent image motion vector corresponding to each adjacent image encoding information; constructs a motion vector candidate list based on each generated adjacent image motion vector; reorders the motion vector candidate list to update the motion vector candidate list.

[0102] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including a receiving unit, an encoding processing unit, an acquisition unit, and an execution unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the receiving unit may also be described as a "unit for receiving a preset video frame".

[0105] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0106] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for sorting a motion vector candidate list, comprising: Receiving a preset video frame; Performing encoding processing on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; Acquire a reference video frame corresponding to the preset video frame; For each image prediction coding unit information in the image coding unit information set, the following steps are performed: Determining the image prediction coding unit information as current image coding information; Based on the current image coding information, determining each image prediction coding unit information satisfying a preset adjacent condition in the image coding unit information set as each adjacent image coding information; Based on the reference video frame and the respective adjacent image encoding information, generating respective adjacent image motion vectors corresponding to the respective adjacent image encoding information; Based on the generated motion vectors of each adjacent image, a motion vector candidate list is constructed; The motion vector candidate list is reordered to update the motion vector candidate list.

2. The method according to claim 1, wherein: The encoding process is performed on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame, including: Determining an image segmentation unit based on pre-stored video encoding information; According to the image segmentation unit, the preset video frame is segmented to obtain information of each image prediction coding unit; The information of each image prediction coding unit is determined as an image coding unit information set.

3. The method according to claim 1, wherein: The step of generating respective adjacent image motion vectors corresponding to respective adjacent image encoding information based on the reference video frame and the respective adjacent image encoding information comprises: For each adjacent image coding information in the respective adjacent image coding information, the following steps are performed: Generate each candidate range information based on the adjacent image encoding information, the reference video frame and a preset search range; Based on the adjacent image encoding information and the respective candidate range information, generating respective similarities corresponding to the adjacent image encoding information; Determine the similarity that satisfies the preset similarity condition among the similarities as the proximity similarity; Determining the candidate range information corresponding to the proximity similarity as reference image coding information; Based on the adjacent image encoding information and the reference image encoding information, an adjacent image motion vector is generated.

4. The method according to claim 3, wherein: The step of generating an adjacent image motion vector based on the adjacent image encoding information and the reference image encoding information comprises: Based on the adjacent image coding information and the preset video frame, generating adjacent coordinates corresponding to the adjacent image coding information; Based on the reference image coding information and the reference video frame, generating reference coordinates corresponding to the reference image coding information; Based on the neighboring coordinates and the reference coordinates, a neighboring image motion vector is generated.

5. The method according to claim 1, wherein: The reordering of the motion vector candidate list to update the motion vector candidate list includes: Based on the current image coding information and the preset adjacent range information, generating a current pixel group corresponding to the current image coding information; For each adjacent image motion vector in the motion vector candidate list, perform the following steps: Determining the reference image coding information corresponding to the adjacent image motion vector as the reference image coding information to be compared; Based on the coding information of the reference image to be compared and the preset adjacent range information, generating a reference pixel group corresponding to the coding information of the reference image to be compared; Generate a comparison difference value based on the current pixel group and the reference pixel group; Sorting the generated comparison differences to generate a comparison difference sequence; The motion vector candidate list is reordered based on the comparison difference sequence to update the motion vector candidate list.

6. The method according to claim 1, wherein: After reordering the motion vector candidate list to update the motion vector candidate list, the method further includes: Based on the reference video frame and the current image coding information, generating a current motion vector corresponding to the current image coding information; Generate respective vector similarities based on the current motion vector and the updated motion vector candidate list; Determine the adjacent image motion vector corresponding to the vector similarity satisfying the preset vector similarity condition among the generated vector similarities as the motion vector to be written; The motion vector to be written is stored in a data storage server.

7. A motion vector candidate list sorting device, comprising: A receiving unit, configured to receive a preset video frame; An encoding processing unit, configured to perform encoding processing on the preset video frame to obtain an image encoding unit information set corresponding to the preset video frame; an acquisition unit, configured to acquire a reference video frame corresponding to the preset video frame; The execution unit is configured to perform the following steps for each image prediction coding unit information in the image coding unit information set: determining the image prediction coding unit information as current image coding information; Based on the current image coding information, determining each image prediction coding unit information satisfying a preset adjacent condition in the image coding unit information set as each adjacent image coding information; Based on the reference video frame and the adjacent image coding information, generate adjacent image motion vectors corresponding to the adjacent image coding information; based on the generated adjacent image motion vectors, construct a motion vector candidate list; The motion vector candidate list is reordered to update the motion vector candidate list.

8. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images