Encoding method and device of multimedia resource, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a multimedia resource encoding method and device, electronic equipment and storage medium. The method comprises: obtaining a current encoding unit corresponding to a to-be-encoded multimedia resource and candidate motion information corresponding to a merge mode with motion vector difference, the candidate motion information comprising a plurality of candidate motion vectors and adjustment information of each candidate motion vector; performing permutation and combination on each candidate motion vector, each offset direction, each list index, and each offset step in an offset step list indicated by each list index to obtain a plurality of encoding parameter combinations; encoding the current encoding unit using each encoding parameter combination to obtain rate-distortion cost data of each encoding parameter combination; and determining an encoding parameter combination whose rate-distortion cost data satisfies a preset condition as a target encoding parameter combination of the current encoding unit to obtain a target encoding result. The method can save computer resources and improve encoding effect.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method, apparatus, electronic device, and computer-readable storage medium for encoding multimedia resources. Background Technology

[0002] In video coding, MMVD (Merge with Motion Vector Difference) is an inter-frame prediction technique in the VVC (Versatile Video Coding) coding standard. MMVD constructs MMVD candidates using the first two candidate motion information from the Merge list, offsets them in both horizontal and vertical directions, and selects the optimal candidate motion information index, offset direction, and offset step size through rate-distortion optimization.

[0003] In related technologies, under the MMVD mode, eight offset steps are specified to be selected: {1 / 4, 1 / 2, 1, 2, 4, 8, 16, 32}. When the offset step selected for the coding block is large, more bits are required to encode the syntax element, which requires more computer resources; at the same time, the encoding effect of the selectable offset steps is poor.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a method for encoding multimedia resources, an apparatus for encoding multimedia resources, an electronic device, and a computer-readable storage medium. This method can save computer resources and improve encoding performance.

[0006] This disclosure provides a method for encoding multimedia resources. The method includes: acquiring a current encoding unit corresponding to the multimedia resource to be encoded and candidate motion information corresponding to a merging mode with motion vector difference. The candidate motion information includes multiple candidate motion vectors and adjustment information for each candidate motion vector. The adjustment information includes offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information includes multiple offset directions, and the offset step size adjustment information includes multiple list indices and offset step size lists indicated by each list index, with each offset step size including multiple offset steps. The method involves arranging and combining each candidate motion vector, each offset direction, each list index, and each offset step size in the offset step size lists indicated by each list index to obtain multiple encoding parameter combinations. The method involves encoding the current encoding unit using each encoding parameter combination to obtain rate-distortion cost data for each encoding parameter combination. The method involves determining the encoding parameter combination whose rate-distortion cost data satisfies a preset condition as the target encoding parameter combination for the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

[0007] In some exemplary embodiments of this disclosure, the method further includes: determining the candidate motion vector index, offset direction index, list index of the offset step size list, and offset step size index corresponding to the target coding parameter combination of the current coding unit; writing the candidate motion vector index, the offset direction index, the list index of the offset step size list, and the offset step size index corresponding to the target coding parameter combination of the current coding unit into the bit stream for transmission to the decoding end for decoding.

[0008] In some exemplary embodiments of this disclosure, the offset step list includes a first offset step list and a second offset step list; each offset step in the first offset step list is smaller than each offset step in the second offset step list; the step interval between adjacent offset steps in the first offset step list is smaller than the step interval between adjacent offset steps in the second offset step list.

[0009] In some exemplary embodiments of this disclosure, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, and the number of offset steps included in each offset step list.

[0010] In some exemplary embodiments of this disclosure, the candidate motion information includes multiple motion vector precisions; the step of arranging and combining each candidate motion vector, each offset direction, each list index, and each offset step in the offset step list indicated by each list index to obtain multiple encoding parameter combinations includes: arranging and combining each candidate motion vector, each offset direction, each list index, each offset step in the offset step list indicated by each list index, and each motion vector precision to obtain the multiple encoding parameter combinations.

[0011] In some exemplary embodiments of this disclosure, the method further includes: determining the candidate motion vector index, offset direction index, list index of the offset step size list, offset step size index, and motion vector precision index corresponding to the target coding parameter combination of the current coding unit; writing the candidate motion vector index, the offset direction index, the list index of the offset step size list, the offset step size index, and the motion vector precision index corresponding to the target coding parameter combination of the current coding unit into the bit stream for transmission to the decoding end for decoding.

[0012] In some exemplary embodiments of this disclosure, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, the number of offset steps included in each offset step list, and the number of motion vector accuracies.

[0013] This disclosure provides an encoding apparatus for multimedia resources, comprising: an acquisition module configured to acquire the current encoding unit corresponding to the multimedia resource to be encoded and candidate motion information corresponding to a merging mode with motion vector difference, wherein the candidate motion information includes multiple candidate motion vectors and adjustment information for each candidate motion vector, the adjustment information including offset direction adjustment information and offset step size adjustment information, the offset direction adjustment information including multiple offset directions, the offset step size adjustment information including multiple list indices and an offset step size list indicated by each list index, each offset step size list including multiple offset steps; and a determination module configured to determine the candidate motion vectors. The system arranges and combines various offset directions, list indices, and offset step sizes indicated by each list index to obtain multiple encoding parameter combinations. A module is configured to encode the current encoding unit using each encoding parameter combination to obtain rate-distortion cost data for each combination. A determining module is configured to determine the encoding parameter combination whose rate-distortion cost data satisfies preset conditions as the target encoding parameter combination for the current encoding unit. This is used to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

[0014] In some exemplary embodiments of this disclosure, the determining module is configured to perform: determining the candidate motion vector index, offset direction index, list index of the offset step size list, and offset step size index corresponding to the target coding parameter combination of the current coding unit; and writing the candidate motion vector index, the offset direction index, the list index of the offset step size list, and the offset step size index corresponding to the target coding parameter combination of the current coding unit into the bitstream for transmission to the decoding end for decoding.

[0015] In some exemplary embodiments of this disclosure, the offset step list includes a first offset step list and a second offset step list; each offset step in the first offset step list is smaller than each offset step in the second offset step list; the step interval between adjacent offset steps in the first offset step list is smaller than the step interval between adjacent offset steps in the second offset step list.

[0016] In some exemplary embodiments of this disclosure, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, and the number of offset steps included in each offset step list.

[0017] In some exemplary embodiments of this disclosure, the candidate motion information includes multiple motion vector precisions; the obtaining module is configured to perform: arranging and combining each candidate motion vector, each offset direction, each list index, each offset step in the offset step list indicated by each list index, and each motion vector precision to obtain the multiple encoding parameter combinations.

[0018] In some exemplary embodiments of this disclosure, the determining module is configured to perform: determining the candidate motion vector index, offset direction index, list index of the offset step size list, offset step size index, and motion vector precision index corresponding to the target coding parameter combination of the current coding unit; and writing the candidate motion vector index, the offset direction index, the list index of the offset step size list, the offset step size index, and the motion vector precision index corresponding to the target coding parameter combination of the current coding unit into the bitstream for transmission to the decoding end for decoding.

[0019] In some exemplary embodiments of this disclosure, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, the number of offset steps included in each offset step list, and the number of motion vector accuracies.

[0020] This disclosure provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement a multimedia resource encoding method as described above.

[0021] This disclosure provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the encoding method of multimedia resources as described above.

[0022] This disclosure provides a computer program product, including a computer program, and an encoding method for multimedia resources that implements any of the above-described methods when the computer program is executed by a processor.

[0023] The multimedia resource encoding method provided in this disclosure, by setting multiple list indices and multiple offset steps included in each list, can use the same offset step index to represent different offset steps when the list indices are different. This allows the same computer resources to represent more offset step information, saving computer resources. At the same time, it increases the number of selectable offset steps, so that when encoding the current encoding unit using various encoding parameter combinations, there are more selectable encoding parameter combinations, making the obtained target encoding parameter combinations and encoding results more accurate, thereby improving the encoding effect.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0026] Figure 1 A schematic diagram of an exemplary system architecture for encoding methods of multimedia resources to which embodiments of the present disclosure can be applied is shown.

[0027] Figure 2 This is a flowchart illustrating a method for encoding multimedia resources according to an exemplary embodiment.

[0028] Figure 3 This is a schematic diagram illustrating a forward reference frame and a backward reference frame according to an exemplary embodiment.

[0029] Figure 4This is a flowchart illustrating another method for encoding multimedia resources according to an exemplary embodiment.

[0030] Figure 5 This is a block diagram illustrating an encoding apparatus for a multimedia resource according to an exemplary embodiment.

[0031] Figure 6 This is a schematic diagram illustrating the structure of an electronic device suitable for implementing exemplary embodiments of the present disclosure, according to an exemplary embodiment. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0033] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0034] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] In this specification, the terms “a,” “an,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element / component / etc.; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.

[0037] Figure 1 A schematic diagram of an exemplary system architecture for encoding methods of multimedia resources to which embodiments of the present disclosure can be applied is shown.

[0038] like Figure 1 As shown, the system architecture may include server 101, network 102, terminal device 103, terminal device 104, and terminal device 105. Network 102 serves as the medium for providing a communication link between terminal device 103, terminal device 104, or terminal device 105 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0039] Server 101 may be a server that provides various services, such as a back-end management server that supports the devices operated by users using terminal devices 103, 104, or 105. The back-end management server may analyze and process received requests and other data, and feed back the processing results to terminal devices 103, 104, or 105; server 101 may be, for example, an encoder or a decoder.

[0040] Terminal devices 103, 104, and 105 can be smartphones, tablets, laptops, desktop computers, smart speakers, wearable smart devices, virtual reality devices, augmented reality devices, etc., but are not limited to these.

[0041] In this embodiment of the disclosure, server 101 can: acquire multimedia resources to be encoded, which include multiple video frames, and divide each video frame into multiple encoding units. The following description takes the unit currently being encoded as the current encoding unit as an example.

[0042] In this embodiment of the disclosure, when encoding the current coding unit, multiple coding modes are used to encode the current coding unit to obtain the rate-distortion value corresponding to each coding mode; based on the rate-distortion value corresponding to each coding mode, the target coding mode of the current coding unit and its corresponding coding result are selected.

[0043] The multiple coding modes include intra-frame coding mode and inter-frame coding mode. The inter-frame coding mode includes Merge mode, and the Merge mode includes MMVD (Merge with Motion Vector Difference) mode. The coding method provided in this embodiment is the process of selecting coding parameters when the current coding unit uses MMVD mode for coding.

[0044] Merge mode is a motion vector prediction technique in coding standards that uses the motion vectors (MVs) of neighboring blocks in the temporal or spatial domains to predict the MV of the current coding block (also known as the current coding unit). Merge mode creates a candidate list of motion information for the current coding unit, containing five candidate motion information pieces. Each candidate contains information such as the MV, prediction direction, reference frame index, and motion vector precision. By traversing these five candidate motion information pieces, the candidate motion information with the lowest rate-distortion cost is selected as the candidate motion information for the current coding unit. No motion information (such as the reference frame index, motion vector precision index, and prediction direction) needs to be transmitted in the bitstream; only the index of the optimal candidate motion information in the list needs to be transmitted.

[0045] Similar to the normal Merge mode, MMVD constructs the MV candidate list in the same way. However, MMVD only constructs two candidate lists, so it only needs to traverse the two candidate motion information and select the best one. MMVD uses the motion information of the candidate lists to perform motion compensation on the current coding block. Under the MMVD model, the MV information needs to be adjusted to a certain extent. Therefore, in MMVD mode, when transmitting motion information, in addition to transmitting the list index of the best candidate motion information as in the normal Merge mode, it is also necessary to transmit information for adjusting the MV, including the offset step size and offset direction.

[0046] Specifically, server 101 can: acquire candidate motion information corresponding to the MMVD mode, the candidate motion information including multiple candidate motion vectors and adjustment information for each candidate motion vector, the adjustment information including offset direction adjustment information and offset step size adjustment information, the offset direction adjustment information including multiple offset directions, the offset step size adjustment information including multiple list indices and offset step size lists indicated by each list index, each offset step size list including multiple offset steps; arrange and combine each candidate motion vector, each offset direction, each list index, and each offset step size in the offset step size lists indicated by each list index to obtain multiple encoding parameter combinations; use each encoding parameter combination to encode the current encoding unit to obtain rate-distortion cost data for each encoding parameter combination; determine the encoding parameter combination whose rate-distortion cost data meets preset conditions as the target encoding parameter combination for the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

[0047] In MMVD mode, after determining the target coding parameter combination for the current coding unit, the rate-distortion value corresponding to the target coding parameter combination in MMVD mode is compared with the rate-distortion value in other coding modes, and the coding mode with the smaller rate-distortion value is selected as the target coding mode for the current coding unit.

[0048] After the current coding unit is encoded, the same method can be used to encode the next coding unit until the current video frame is encoded.

[0049] It should be understood that Figure 1 The number of terminal devices 103, 104, 105, network 102, and server 101 in the diagram is merely illustrative. Server 101 can be a single physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0050] The following will describe in more detail the various steps of the multimedia resource encoding method in the exemplary embodiments of this disclosure, with reference to the accompanying drawings and examples. The method provided in the embodiments of this disclosure can be executed by any electronic device, such as the one described above. Figure 1 The server and / or terminal equipment in the process, but this disclosure does not limit this.

[0051] Figure 2 This is a flowchart illustrating a method for encoding multimedia resources according to an exemplary embodiment.

[0052] like Figure 2 As shown, the method provided in this disclosure embodiment may include the following steps.

[0053] In step S210, the current encoding unit corresponding to the multimedia resource to be encoded and the candidate motion information corresponding to the MMVD mode are obtained. The candidate motion information includes multiple candidate motion vectors and adjustment information for each candidate motion vector. The adjustment information includes offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information includes multiple offset directions. The offset step size adjustment information includes multiple list indices and an offset step size list indicated by each list index. Each offset step size list includes multiple offset steps.

[0054] In this embodiment of the disclosure, the MMVD mode includes a unidirectional MMVD mode and a bidirectional MMVD mode. When the prediction direction in the candidate motion information is unidirectional, meaning the current coding unit performs unidirectional prediction and only references the forward reference frame (L0 reference) or the backward reference frame (L1 reference), it is unidirectional MMVD. When the prediction direction in the candidate motion information is bidirectional, meaning the current coding unit performs bidirectional prediction and needs to reference both the forward reference frame and the backward reference frame, it is bidirectional MMVD. The forward reference frame (L0 reference) and the backward reference frame (L1 reference) are as follows: Figure 3 As shown.

[0055] In the examples below, the combination of encoding parameters is determined under the unidirectional MMVD mode.

[0056] In this embodiment of the disclosure, in unidirectional MMVD mode, the candidate motion information may include multiple candidate motion vectors and adjustment information for each candidate motion vector. The adjustment information may include offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information may include multiple offset directions, and the offset step size adjustment information may include multiple list indices and offset step size lists indicated by each list index. Each offset step size list may include multiple offset steps. For example, there may be 2 candidate motion vectors, 4 offset directions (e.g., positive x-axis direction, negative x-axis direction, positive y-axis direction, and negative y-axis direction), 2 list indices (e.g., List0 and List1), and offset step size lists indicated by each list index (e.g., the 0th offset step size list indicated by List0 and the 1st offset step size list indicated by List1). Each offset step size list may include 8 offset steps.

[0057] In an exemplary embodiment, the offset step list includes a first offset step list List0 and a second offset step list List1; each offset step in the first offset step list List0 is smaller than each offset step in the second offset step list List1; the step interval between adjacent offset steps in the first offset step list List0 is smaller than the step interval between adjacent offset steps in the second offset step list List1.

[0058] For example, the offset steps included in the first offset step list List0 and the second offset step list List1 are as follows:

[0059] List0={1 / 4,2 / 4,3 / 4,1,5 / 4,6 / 4,8 / 4,10 / 4},

[0060] List1={4,6,8,12,16,24,32,64},

[0061] From the offset step values ​​in the first offset step list List0 and the second offset step list List1, it can be seen that the offset step values ​​in List0 are all relatively small, making it suitable for encoding units with small motion amplitudes; the offset step values ​​and the intervals between adjacent offset steps in List1 are relatively large, making it suitable for encoding units with large motion amplitudes. Furthermore, for the offset steps in List0 and List1, the last offset step (i.e., the largest offset step) in List0 is smaller than the first offset step (i.e., the smallest offset step) in List1; and the step interval between adjacent offset steps in List1 is larger than the step interval between adjacent offset steps in List0.

[0062] In this embodiment of the disclosure, by setting each offset step in the first offset step list to be smaller than each offset step in the second offset step list, and the step interval between adjacent offset steps in the first offset step list to be smaller than the step interval between adjacent offset steps in the second offset step list, a suitable offset step list can be selected for coding units with different motion amplitudes, thereby improving coding efficiency.

[0063] In this embodiment of the disclosure, different syntax elements can be used to represent the above-mentioned candidate motion vectors, offset directions, list indices, and offset step sizes in each offset step size list.

[0064] For example, the first syntax element (mmvd_cand_flag) is used to represent the candidate motion vector, as shown in Table 1:

[0065] Table 1. Meaning of the first syntax element mmvd_cand_flag

[0066] value Meaning 0 Select the zeroth candidate motion vector 1 Select the first candidate motion vector

[0067] For example, the offset direction is represented by the second syntax element (mmvd_direction_idx), as shown in Table 2:

[0068] Table 2. Meaning of the second syntax element mmvd_direction_idx

[0069] value Meaning 00 Choose the positive x-axis direction 01 Choose the negative x-axis direction 10 Choose the positive direction of the y-axis. 11 Choose the negative y-axis direction.

[0070] For example, the third syntax element (mmvd_list_idx) is used to represent the list index, as shown in Table 3:

[0071] Table 3. Meaning of the third syntax element mmvd_list_idx

[0072] value Meaning 0 Select List0 1 Select List1

[0073] For example, the fourth syntax element (mmvd_distance_flag) represents the offset step size, and it is encoded using a truncated unary code method, as shown in Table 4:

[0074] Table 4. Meaning of the fourth syntax element mmvd_distance_flag

[0075] value Meaning 0 Choose the 0th step size 10 Select the first step size 110 Choose the second step size 1110 Choose the 3rd step size 11110 Choose the 4th step size 111110 Choose the 5th step size 1111110 Choose the 6th step size 1111111 Choose the 7th step size

[0076] As can be seen, the first syntax element mmvd_cand_flag only requires 1 bit to represent its information, the second syntax element mmvd_direction_idx requires 2 bits to represent its information, the third syntax element mmvd_list_idx only requires 1 bit to represent its information, and the fourth syntax element mmvd_distance_idx requires 1 to 7 bits to represent its information, depending on the selected step size.

[0077] Referring to Tables 3 and 4, when the third syntax element is 0 (i.e., List0 is selected), if the fourth syntax element is 0, it means selecting the 0th step size in List0, which is 1 / 4; when the third syntax element is 1 (i.e., List1 is selected), if the fourth syntax element is 0, it means selecting the 0th step size in List1, which is 4. This shows that when the list indices are different, the same offset step size index can be used to represent different offset step sizes. Thus, the same computer resources can be used to represent more offset step size information, while increasing the number of selectable offset step sizes, thereby improving coding performance.

[0078] In this embodiment of the disclosure, by setting multiple list indices and multiple offset steps included in the offset step list indicated by each list index, when encoding the current coding unit, the list index is selected first, and then a suitable offset step is selected from the corresponding offset step list according to the list index. This increases the number of selectable offset steps and can improve coding performance. At the same time, in subsequent data transmission, only the list index and the offset step index need to be transmitted, which can save transmission resources when representing a large offset step.

[0079] In step S220, each candidate motion vector, each offset direction, each list index, and each offset step in the offset step list indicated by each list index are arranged and combined to obtain multiple combinations of encoding parameters.

[0080] In this embodiment of the disclosure, each candidate motion vector, each offset direction, each list index, and each offset step in the offset step list indicated by each list index can be traversed and arranged and combined. Each candidate motion vector, each offset direction, each list index, and each offset step in the offset step list indicated by each list index is used to form a combination of encoding parameters.

[0081] In an exemplary embodiment, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, and the number of offset steps included in each offset step list.

[0082] For example, if there are 2 candidate motion vectors, 4 offset directions, 2 list indices, and each offset step list includes 8 offset steps, the total number of encoding parameter combinations is 2×4×2×8=128.

[0083] In this embodiment of the disclosure, the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, and the number of offset steps included in each offset step list is the number of encoding parameter combinations. The encoding parameter combinations determined in this way are more comprehensive, thereby ensuring that the target encoding parameter combinations determined based on each encoding parameter combination are more accurate.

[0084] In step S230, the current coding unit is encoded using each combination of coding parameters to obtain the rate-distortion cost data for each combination of coding parameters.

[0085] In this embodiment of the disclosure, the rate-distortion cost data can be a rate-distortion cost value; the current coding unit is encoded using the candidate motion vector, offset direction and offset step size in each coding parameter combination to obtain the rate-distortion cost value of each coding parameter combination.

[0086] Specifically, starting from the position pointed to by the candidate motion vector in the reference frame, different new motion vectors are formed with different offset directions and different offset step sizes. For example... Figure 3 As shown, the origin 301 of the dashed line in the figure is the starting point of the candidate motion vector in the reference frame (forward reference frame or backward reference frame). The candidate motion vector will be offset in the horizontal or vertical direction to obtain a new motion vector (solid circle in the figure).

[0087] For example, assuming the candidate motion vector MV is {4,6} and the offset step size is 2, then the offsets MVoffset in the four directions are: positive x-axis direction {2,0}, negative x-axis direction {-2,0}, positive y-axis direction {0,2}, and negative y-axis direction {0,-2}. The adjusted new motion vector is: MVfinal = MV + MVoffset. Taking the positive x-axis direction as an example, the new motion vector is: MVfinal = {4,6} + {2,0} = {6,6}.

[0088] Therefore, based on 2 candidate motion vectors, 4 offset directions, 2 list indices, and 8 offset steps included in each offset step list, 128 new motion vectors can be obtained. These 128 new motion vectors are used to encode the current coding unit respectively, and the rate-distortion value of each combination of coding parameters is obtained.

[0089] In step S240, the combination of encoding parameters that meets the preset conditions of the rate-distortion cost data is determined as the target encoding parameter combination of the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

[0090] In this embodiment of the disclosure, the combination of coding parameters with the lowest rate-distortion cost can be used as the target coding parameter combination of the current coding unit.

[0091] In an exemplary embodiment, the method may further include: determining the candidate motion vector index, offset direction index, list index of the offset step size list, and offset step size index corresponding to the target coding parameter combination of the current coding unit; and writing the candidate motion vector index, offset direction index, list index of the offset step size list, and offset step size index corresponding to the target coding parameter combination of the current coding unit into the bit stream for transmission to the decoding end for decoding.

[0092] In this embodiment of the disclosure, after determining the target encoding parameter combination, according to Tables 1 to 4, the candidate motion vector index (e.g., the candidate motion vector index corresponding to the zeroth candidate motion vector is 0), the offset direction index (e.g., the offset direction index corresponding to the positive x-axis direction is 00), the list index and the offset step index (e.g., the list index and the offset step index corresponding to the offset step 1 / 4 are 0 and 0 respectively) are determined; the candidate motion vector index, offset direction index, list index and offset step index are written into the bitstream for transmission to the decoding end.

[0093] In this embodiment of the disclosure, after determining the target encoding parameter combination, the candidate motion vector index, offset direction index, list index of the offset step size list, and offset step size index corresponding to the target encoding parameter combination are determined. Only the above indexes need to be written into the bit stream, and only the above indexes need to be transmitted during data transmission, which can save transmission resources and improve transmission efficiency.

[0094] The multimedia resource encoding method provided in this disclosure, by setting multiple list indices and multiple offset steps included in each list, can use the same offset step index to represent different offset steps when the list indices are different. This allows the same computer resources to represent more offset step information, saving computer resources. At the same time, it increases the number of selectable offset steps, so that when encoding the current encoding unit using various encoding parameter combinations, there are more selectable encoding parameter combinations, making the obtained target encoding parameter combinations and encoding results more accurate, thereby improving the encoding effect.

[0095] Figure 4 This is a flowchart illustrating another method for encoding multimedia resources according to an exemplary embodiment.

[0096] like Figure 4 As shown, the method provided in this disclosure embodiment may include the following steps.

[0097] In step S410, the current encoding unit corresponding to the multimedia resource to be encoded and the candidate motion information corresponding to the merging mode with motion vector difference are obtained. The candidate motion information includes multiple candidate motion vectors, adjustment information for each candidate motion vector and multiple motion vector accuracies. The adjustment information includes offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information includes multiple offset directions. The offset step size adjustment information includes multiple list indices and an offset step size list indicated by each list index. Each offset step size list includes multiple offset steps.

[0098] In this embodiment of the disclosure, in addition to obtaining multiple candidate motion vectors, multiple offset directions, multiple list indices, and a list of offset step sizes indicated by each list index, the precision of multiple motion vectors of the current coding unit can also be obtained.

[0099] In MMVD mode, the motion vector precision of the coding unit can be either half precision or quarter precision.

[0100] In related technologies, under MMVD mode, the motion vector precision of each coding unit is inherited from the candidate motion vector. If the motion vector precision of the candidate motion vector is one-quarter pixel precision, then the motion vector precision of the current coding unit is also one-quarter; otherwise, the motion vector precision of the current coding unit is set to one-half.

[0101] In this embodiment of the disclosure, rate-distortion optimization is used to determine the optimal motion vector precision for the coding unit of a unidirectional MMVD. Specifically, during the process of traversing all combinations of candidate motion vectors, offset directions, offset candidate lists, and offset step sizes in the current unidirectional MMVD, the rate-distortion cost of the motion vectors at one-quarter precision and one-half precision is calculated using rate-distortion optimization for each combination, and the optimal motion vector precision for the current coding block is selected from these.

[0102] In this embodiment of the disclosure, different syntax elements can be used to represent candidate motion vectors, offset directions, list indices, offset step sizes in each offset step size list, and motion vector precision.

[0103] For example, the first to fourth syntax elements are used to represent candidate motion vectors, offset directions, list indices, and offset step sizes in each offset step size list, as shown in Tables 1 to 4; the fifth syntax element (mmvd_mv_resolution_idx) is used to represent the motion vector precision, as shown in Table 5.

[0104] Table 5. Meaning of the fifth syntax element mmvd_mv_resolution_idx

[0105] value Meaning 0 Select half-pixel precision 1 Select quarter pixel precision

[0106] As shown in Table 5, the fifth syntax element mmvd_mv_resolution_idx only requires 1 bit to represent its syntax meaning.

[0107] In step S420, each candidate motion vector, each offset direction, each list index, each offset step in the offset step list indicated by each list index, and each motion vector precision are arranged and combined to obtain multiple combinations of encoding parameters.

[0108] In this embodiment of the disclosure, each candidate motion vector, each offset direction, each list index, each offset step in the offset step list indicated by each list index, and each motion vector precision can be traversed and arranged and combined. Each candidate motion vector, each offset direction, each list index, each offset step in the offset step list indicated by each list index, and each motion vector precision are used to form a combination of encoding parameters.

[0109] In an exemplary embodiment, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, the number of offset steps included in each offset step list, and the number of motion vector precisions.

[0110] For example, if there are 2 candidate motion vectors, 4 offset directions, 2 list indices, 8 offset steps in each offset step list, and 2 motion vector precisions, the total number of encoding parameter combinations is: 2×4×2×8×2=256.

[0111] In this embodiment of the disclosure, the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, the number of offset steps included in each offset step list, and the number of motion vector accuracies is the number of encoding parameter combinations. The encoding parameter combinations determined in this way are more comprehensive, thereby ensuring that the target encoding parameter combinations determined based on each encoding parameter combination are more accurate.

[0112] In step S430, the current coding unit is encoded using each combination of coding parameters to obtain the rate-distortion cost data for each combination of coding parameters.

[0113] In this embodiment of the disclosure, the candidate motion vector, offset direction, offset step size and motion vector precision in each combination of encoding parameters are used to encode the current encoding unit to obtain the rate-distortion value of each combination of encoding parameters.

[0114] Specifically, based on 2 candidate motion vectors, 4 offset directions, 2 list indices, 8 offset steps included in each offset step list, and 2 motion vector accuracies, 256 combinations of coding parameters can be obtained. These 256 combinations of coding parameters are used to encode the current coding unit respectively, and the rate-distortion value of each combination of coding parameters is obtained.

[0115] In step S440, the combination of encoding parameters that meets the preset conditions of the rate-distortion cost data is determined as the target encoding parameter combination of the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

[0116] In this embodiment of the disclosure, the combination of coding parameters with the lowest rate-distortion cost can be used as the target coding parameter combination of the current coding unit.

[0117] In an exemplary embodiment, the method may further include: determining the candidate motion vector index, offset direction index, list index of the offset step size list, offset step size index, and motion vector precision index corresponding to the target coding parameter combination of the current coding unit; writing the candidate motion vector index, offset direction index, list index of the offset step size list, offset step size index, and motion vector precision index corresponding to the target coding parameter combination of the current coding unit into the bit stream for transmission to the decoding end for decoding.

[0118] In this embodiment of the disclosure, after determining the target encoding parameter combination, according to Tables 1 to 5, the candidate motion vector index (e.g., the candidate motion vector index corresponding to the zeroth candidate motion vector is 0), the offset direction index (e.g., the offset direction index corresponding to the positive x-axis direction is 00), the list index and offset step index (e.g., the list index and offset step index corresponding to the offset step of 1 / 4 are 0 and the offset step index is 0), and the motion vector precision index (e.g., 0 corresponding to half precision) are determined. The candidate motion vector index, offset direction index, list index, offset step index, and motion vector precision index are written into the bitstream for transmission to the decoding end.

[0119] In this embodiment of the disclosure, after determining the target encoding parameter combination, the candidate motion vector index, offset direction index, list index of the offset step size list, offset step size index, and motion vector precision index corresponding to the target encoding parameter combination are determined. Only the above indexes need to be written into the bit stream, and only the above indexes need to be transmitted during data transmission, which can save transmission resources and improve transmission efficiency.

[0120] The multimedia resource encoding method provided in this disclosure, during the process of traversing all combinations of candidate motion vectors, offset directions, offset candidate lists, and offset step sizes, also calculates the rate-distortion cost of the motion vectors at one-quarter and one-half precision through rate-distortion optimization for each combination. Compared with the method of directly inheriting the precision of motion vectors in related technologies, this method can make the obtained target encoding parameter combinations and encoding results more accurate, thereby improving encoding performance.

[0121] It should also be understood that the above is only to help those skilled in the art better understand the embodiments of this disclosure, and is not intended to limit the scope of the embodiments of this disclosure. Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given above. For example, some steps in the above methods may be unnecessary, or new steps may be added, etc. Alternatively, any combination of any two or more of the above embodiments may be used. Such modifications, changes, or combinations also fall within the scope of the embodiments of this disclosure.

[0122] It should also be understood that the above description of the embodiments of this disclosure focuses on highlighting the differences between the various embodiments. Similarities or differences not mentioned can be referred to each other, and for the sake of brevity, they will not be repeated here.

[0123] It should also be understood that the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0124] It should also be understood that, in the various embodiments of this disclosure, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between different embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0125] The foregoing has detailed examples of multimedia resource encoding methods provided in this disclosure. It is understood that, in order to implement the above functions, a computer device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0126] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0127] Figure 5 This is a block diagram illustrating an encoding apparatus for multimedia resources according to an exemplary embodiment. (Refer to...) Figure 5 The device 500 may include an acquisition module 510, a determination module 520, and an acquisition module 530.

[0128] The acquisition module 510 is configured to acquire the current encoding unit corresponding to the multimedia resource to be encoded and the candidate motion information corresponding to the merging mode with motion vector difference. The candidate motion information includes multiple candidate motion vectors and adjustment information for each candidate motion vector. The adjustment information includes offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information includes multiple offset directions, and the offset step size adjustment information includes multiple list indices and offset step size lists indicated by each list index. Each offset step size list includes multiple offset steps. The determination module 520 is configured to determine the candidate motion vectors, offset directions, and offset step size information. The list indexes and the offset steps in the offset step list indicated by each list index are arranged and combined to obtain multiple encoding parameter combinations; the obtaining module 530 is configured to encode the current encoding unit using each encoding parameter combination to obtain rate-distortion cost data for each encoding parameter combination; the determining module is configured to determine the encoding parameter combination that satisfies the rate-distortion cost data of each encoding parameter combination as the target encoding parameter combination for the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

[0129] In some exemplary embodiments of this disclosure, the determining module 520 is further configured to perform: determining the candidate motion vector index, offset direction index, list index of the offset step size list, and offset step size index corresponding to the target coding parameter combination of the current coding unit; and writing the candidate motion vector index, the offset direction index, the list index of the offset step size list, and the offset step size index corresponding to the target coding parameter combination of the current coding unit into the bitstream for transmission to the decoding end for decoding.

[0130] In some exemplary embodiments of this disclosure, the offset step list includes a first offset step list and a second offset step list; each offset step in the first offset step list is smaller than each offset step in the second offset step list; the step interval between adjacent offset steps in the first offset step list is smaller than the step interval between adjacent offset steps in the second offset step list.

[0131] In some exemplary embodiments of this disclosure, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, and the number of offset steps included in each offset step list.

[0132] In some exemplary embodiments of this disclosure, the candidate motion information includes multiple motion vector precisions; the obtaining module 530 is configured to perform: arranging and combining each candidate motion vector, each offset direction, each list index, each offset step in the offset step list indicated by each list index, and each motion vector precision to obtain the multiple encoding parameter combinations.

[0133] In some exemplary embodiments of this disclosure, the determining module 520 is further configured to perform: determining the candidate motion vector index, offset direction index, list index of the offset step size list, offset step size index, and motion vector precision index corresponding to the target coding parameter combination of the current coding unit; and writing the candidate motion vector index, the offset direction index, the list index of the offset step size list, the offset step size index, and the motion vector precision index corresponding to the target coding parameter combination of the current coding unit into the bitstream for transmission to the decoding end for decoding.

[0134] In some exemplary embodiments of this disclosure, the number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, the number of offset steps included in each offset step list, and the number of motion vector accuracies.

[0135] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor terminal devices and / or microcontroller terminal devices.

[0136] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0137] The following reference Figure 6 To describe an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0138] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), and a display unit 640.

[0139] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 610 can perform actions such as... Figure 2 The steps shown are as follows.

[0140] For example, electronic devices can achieve such Figure 2 The steps shown.

[0141] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 621 and / or cache memory 622, and may further include read-only memory (ROM) 623.

[0142] Storage unit 620 may also include a program / utility 624 having a set (at least one) of program modules 625, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0143] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0144] Electronic device 600 can also communicate with one or more external devices 670 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0145] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0146] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions that can be executed by a processor of the device to perform the described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0147] In an exemplary embodiment, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, implement the multimedia resource encoding method described in the above embodiments.

[0148] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0149] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for encoding multimedia resources, characterized in that, include: Obtain the current encoding unit corresponding to the multimedia resource to be encoded and the candidate motion information corresponding to the merging mode with motion vector difference. The candidate motion information includes multiple candidate motion vectors and adjustment information for each candidate motion vector. The adjustment information includes offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information includes multiple offset directions. The offset step size adjustment information includes multiple list indices and an offset step size list indicated by each list index. Each offset step size list includes multiple offset steps. Arrange and combine each candidate motion vector, each offset direction, each list index, and each offset step in the offset step list indicated by each list index to obtain multiple combinations of encoding parameters. Encode the current coding unit using various combinations of coding parameters to obtain rate-distortion cost data for each combination of coding parameters. The combination of encoding parameters that meets the preset conditions for rate-distortion cost data is determined as the target encoding parameter combination of the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

2. The method according to claim 1, characterized in that, The method further includes: Determine the candidate motion vector index, offset direction index, offset step size list index, and offset step size index corresponding to the target coding parameter combination of the current coding unit; The candidate motion vector index, the offset direction index, the list index of the offset step size list, and the offset step size index corresponding to the target encoding parameter combination of the current encoding unit are written into the bit stream for transmission to the decoding end for decoding.

3. The method according to claim 1, characterized in that, The offset step size list includes a first offset step size list and a second offset step size list; Each offset step in the first offset step list is smaller than each offset step in the second offset step list; The step interval between adjacent offset steps in the first offset step list is smaller than the step interval between adjacent offset steps in the second offset step list.

4. The method according to claim 1 or 3, characterized in that, The number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, and the number of offset steps included in each offset step list.

5. The method according to claim 1, characterized in that, The candidate motion information includes the precision of multiple motion vectors; The process involves arranging and combining each candidate motion vector, each offset direction, each list index, and each offset step size in the offset step size list indicated by each list index to obtain multiple combinations of encoding parameters, including: The candidate motion vectors, offset directions, list indices, offset step sizes in the offset step size list indicated by each list index, and the precision of each motion vector are arranged and combined to obtain the multiple encoding parameter combinations.

6. The method according to claim 5, characterized in that, The method further includes: Determine the candidate motion vector index, offset direction index, offset step size list index, offset step size index, and motion vector precision index corresponding to the target coding parameter combination of the current coding unit; The candidate motion vector index, the offset direction index, the list index of the offset step size list, the offset step size index, and the motion vector precision index corresponding to the target encoding parameter combination of the current encoding unit are written into the bit stream for transmission to the decoding end for decoding.

7. The method according to claim 5, characterized in that, The number of combinations of encoding parameters is the product of the number of candidate motion vectors, the number of offset directions, the number of list indices, the number of offset steps included in each offset step list, and the number of motion vector precisions.

8. An encoding device for multimedia resources, characterized in that, include: The acquisition module is configured to acquire the current encoding unit corresponding to the multimedia resource to be encoded and the candidate motion information corresponding to the merging mode with motion vector difference. The candidate motion information includes multiple candidate motion vectors and adjustment information for each candidate motion vector. The adjustment information includes offset direction adjustment information and offset step size adjustment information. The offset direction adjustment information includes multiple offset directions. The offset step size adjustment information includes multiple list indices and an offset step size list indicated by each list index. Each offset step size list includes multiple offset steps. The determination module is configured to perform a combination of candidate motion vectors, offset directions, list indices, and offset steps in the list of offset steps indicated by each list index to obtain multiple combinations of encoded parameters. The acquisition module is configured to encode the current coding unit using various combinations of coding parameters, and obtain rate-distortion cost data for each combination of coding parameters. The determining module is configured to execute the combination of encoding parameters that satisfies the preset conditions for the rate-distortion cost data of each encoding parameter combination, and determine it as the target encoding parameter combination of the current encoding unit, so as to obtain the target encoding result of the multimedia resource to be encoded based on the encoding result of the current encoding unit corresponding to the target encoding parameter combination.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the executable instructions to implement the encoding method for multimedia resources as described in any one of claims 1 to 7.

10. A computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform an encoding method for a multimedia resource as described in any one of claims 1 to 7.

Images