Multimedia data coding method and apparatus, readable medium and electronic device

Abstract

Disclosed in the present application are a multimedia data coding method and apparatus, a computer readable medium and an electronic device. The method comprises: according to a long-term reference frame corresponding to a current multimedia frame to be coded in multimedia data, coding said current multimedia frame to obtain a current coded multimedia frame; acquiring the coding quality of the long-term reference frame, and calculating the coding quality of the current coded multimedia frame; according to a motion vector corresponding to a coding block in the current coded multimedia frame, determining the type of the current coded multimedia frame; and, according to the coding quality of the current coded multimedia frame, the type of the current coded multimedia frame and the coding quality of the long-term reference frame, updating the long-term reference frame. The technical solution provided in the embodiments of the present application ensures that updated long-term reference frames are more reasonable and accurate, thus improving the coding quality of multimedia data.

Description

Multimedia data encoding method, device, readable medium and electronic device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 25, 2023, with application number 2023110952664 and application name “Encoding method, device, readable medium and electronic device for multimedia data”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of multimedia technology, and specifically relates to a method, device, computer-readable medium, and electronic device for encoding multimedia data. Background Art

[0003] In order to efficiently transmit video data, the video data is usually encoded first and then the encoded video data is transmitted. During the video encoding process, in order to reduce the volume of the encoded video data, the video data is usually broken down into multiple frames, and then the differences between the frames are encoded. Therefore, when encoding, the selection of the reference frame will have a significant impact on the quality of the encoded video. Generally, when encoding a certain frame in a video, the video frame that is closest to it is selected as the reference frame. However, this encoding method has the problem of error propagation, resulting in the encoding quality of the video frame decreasing as the encoding process progresses. Therefore, this encoding method needs to be improved.

[0004] Summary of the Invention

[0005] The purpose of this application is to provide a method, device, computer-readable medium and electronic device for encoding multimedia data, which can improve the encoding quality of multimedia data.

[0006] An embodiment of the present application provides a method for encoding multimedia data, including:

[0007] encoding the current multimedia frame to be encoded according to a long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame;

[0008] Obtaining the encoding quality of the long-term reference frame and calculating the encoding quality of the currently encoded multimedia frame;

[0009] determining a type of the currently encoded multimedia frame according to a motion vector corresponding to a coding block in the currently encoded multimedia frame;

[0010] The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, and the multimedia frame to be encoded in the multimedia data is encoded according to the updated long-term reference frame.

[0011] An embodiment of the present application provides, on one hand, a multimedia data encoding device, including:

[0012] an encoding module, configured to encode the current multimedia frame to be encoded according to a long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, to obtain a current encoded multimedia frame;

[0013] an encoding quality calculation module, configured to obtain the encoding quality of the long-term reference frame and calculate the encoding quality of the currently encoded multimedia frame;

[0014] a multimedia frame type calculation module, configured to determine the type of the currently encoded multimedia frame according to a motion vector corresponding to a coding block in the currently encoded multimedia frame;

[0015] A reference frame updating module is configured to update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, and encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0016] In one embodiment of the present application, the encoding quality calculation module includes:

[0017] a coding cost calculation unit, configured to calculate a target coding cost value of each coding block included in the currently encoded multimedia frame according to a coding mode corresponding to the currently encoded multimedia frame;

[0018] a statistical unit, configured to calculate a target coding cost value of the currently encoded multimedia frame according to a statistical value of the target coding cost value of each coding block;

[0019] The quality calculation unit is configured to determine the encoding quality of the currently encoded multimedia frame according to a target encoding cost value of the currently encoded multimedia frame, wherein the target encoding cost value of the currently encoded multimedia frame is negatively correlated with the encoding quality.

[0020] In one embodiment of the present application, the encoding cost calculation unit includes:

[0021] A first subunit is configured to calculate a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block if no pre-encoding process is used in the encoding process of the currently encoded multimedia frame;

[0022] The second subunit is configured to calculate a target coding cost value of the coding block according to the original coding cost value of the coding block and the pre-coding cost value of the coding block if pre-coding processing is adopted in the coding process of the currently coded multimedia frame.

[0023] In one embodiment of the present application, the first subunit is specifically configured to:

[0024] The original coding cost value of the coding block is multiplied by a quantization parameter corresponding to the coding block to obtain a target coding cost value of the coding block, where the quantization parameter is used to amplify the original coding cost value of the coding block.

[0025] In one embodiment of the present application, the second subunit is specifically configured to:

[0026] Multiplying the original coding cost value of the coding block by the quantization parameter corresponding to the coding block to obtain a candidate coding cost value of the coding block;

[0027] A weighted sum is performed on the candidate coding cost values ​​of the coding block and the precoding cost value of the coding block to obtain a target coding cost value of the coding block, wherein the weight of the candidate coding cost value is greater than the weight of the precoding cost value.

[0028] In one embodiment of the present application, the multimedia frame type calculation module includes:

[0029] a target coding block determining unit, configured to determine, based on a motion vector corresponding to a coding block in the currently encoded multimedia frame, a target coding block having a motion vector amplitude greater than a preset amplitude in the currently encoded multimedia frame;

[0030] The type determining unit is configured to determine the type of the currently encoded multimedia frame according to the number of target coding blocks contained in the currently encoded multimedia frame.

[0031] In one embodiment of the present application, the target coding block determination unit is specifically configured to:

[0032] Extracting a candidate coding block to be encoded based on an inter-frame prediction mode from a plurality of coding blocks included in the currently encoded multimedia frame;

[0033] Calculating the magnitude of the motion vector corresponding to the candidate coding block;

[0034] When the magnitude of the motion vector corresponding to the candidate coding block is greater than a preset magnitude, it is determined that the candidate coding block belongs to the target coding block.

[0035] In one embodiment of the present application, the type determination unit is specifically configured to:

[0036] Calculating a ratio of the number of target coding blocks included in the current encoded multimedia frame to the total number of coding blocks included in the current encoded multimedia frame;

[0037] When the ratio is less than a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame;

[0038] When the ratio is greater than a preset threshold, it is determined that the currently encoded multimedia frame is a dynamic multimedia frame.

[0039] In one embodiment of the present application, the reference frame updating module is specifically configured to:

[0040] When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as the updated long-term reference frame.

[0041] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; and the reference frame updating module is specifically configured to:

[0042] When the number of currently encoded frames reaches a preset number of frames, clearing the currently encoded number of frames and updating the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame;

[0043] The preset number of frames is an integer and is smaller than the total number of multimedia frames included in the multimedia frame group where the current multimedia frame to be encoded is located.

[0044] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; and the apparatus further includes:

[0045] The key frame setting module is used to use the first frame in the multimedia frame group as a long-term reference frame when encoding the multimedia frame to be encoded in the multimedia frame group for the first time, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

[0046] In one embodiment of the present application, the reference frame updating module is specifically configured to:

[0047] If the long-term reference frame is a key frame, obtaining a quality weight of the long-term reference frame according to a time interval between the current encoded multimedia frame and the long-term reference frame, wherein the quality weight is negatively correlated with the time interval;

[0048] generating a target encoding quality of the long-term reference frame according to a quality weight and an encoding quality of the long-term reference frame, wherein the target encoding quality of the long-term reference frame is less than the encoding quality of the long-term reference frame;

[0049] The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame.

[0050] In one embodiment of the present application, the encoding module is specifically configured to:

[0051] Obtaining the number of short-term reference frames according to the difference between the total number of reference frames corresponding to the multimedia data and the number of long-term reference frames;

[0052] using a plurality of encoded multimedia frames corresponding to the number of short-term reference frames and located before the current multimedia frame to be encoded as short-term reference frames;

[0053] The current multimedia frame to be encoded is encoded according to the short-term reference frame and the long-term reference frame.

[0054] On the one hand, an embodiment of the present application provides an electronic device, including a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0055] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the method in the embodiment of the present application.

[0056] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium; a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method in the embodiment of the present application.

[0057] In the technical solution provided in the embodiment of the present application, after encoding the current multimedia frame to be encoded, the encoding quality of the long-term reference frame is obtained, the encoding quality of the current encoded multimedia frame is calculated, the type of the current encoded multimedia frame is determined according to the motion vector corresponding to the encoding block in the current encoded multimedia frame, and the long-term reference frame is updated according to the encoding quality of the current encoded multimedia frame, the type of the current encoded multimedia frame and the encoding quality of the long-term reference frame, thereby realizing dynamic adjustment of the long-term reference frame during the multimedia data encoding process, avoiding the fixed long-term reference frame from being unable to provide effective reference information for multimedia frames that are farther away, and improving the encoding quality of the multimedia data; at the same time, the long-term reference frame is updated based on the encoding quality and the multimedia frame type, thereby ensuring that the updated long-term reference frame is more reasonable and accurate, and further improving the encoding quality of the multimedia data. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG1 schematically shows a block diagram of an exemplary system architecture applying the technical solution of the present application.

[0059] FIG2 schematically shows a flow chart of a method for encoding multimedia data provided by an embodiment of the present application.

[0060] FIG3 schematically shows a flow chart of a method for encoding multimedia data provided by an embodiment of the present application.

[0061] FIG4 schematically shows a diagram of a multimedia frame encoding process.

[0062] FIG5 schematically shows a flow chart of a multimedia data encoding method provided by an embodiment of the present application.

[0063] FIG6 schematically shows a schematic diagram of a reference structure provided by an embodiment of the present application.

[0064] FIG7 schematically shows a schematic diagram of a reference structure provided by an embodiment of the present application.

[0065] FIG8 schematically shows a flowchart of a method for encoding multimedia data provided by an embodiment of the present application.

[0066] FIG9 schematically illustrates a long-term reference frame replacement process using the technical solution of the present application.

[0067] FIG10 schematically shows a structural block diagram of a multimedia data encoding device provided in an embodiment of the present application.

[0068] FIG11 schematically shows a block diagram of a computer system structure of an electronic device suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0069] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0070] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0071] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0072] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0073] It should be noted that, in the embodiments of the present application, the steps involving the judgment conditions, the case where the judgment result is equal can be classified as either the judgment result is greater than or the judgment result is less than according to actual needs. The "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0074] FIG1 schematically shows a block diagram of an exemplary system architecture applying the technical solution of the present application.

[0075] 1 , a system architecture 100 includes a plurality of terminal devices that can communicate with each other via, for example, a network 150. For example, the system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected via the network 150. In the embodiment of FIG1 , the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0076] For example, the first terminal device 110 may encode multimedia data (such as a video picture stream collected by the first terminal device 110) for transmission to the second terminal device 120 via the network 150. The encoded multimedia data is transmitted in the form of one or more encoded multimedia code streams. The second terminal device 120 may receive the encoded multimedia data from the network 150, decode the encoded multimedia data to restore the multimedia data, and display multimedia content based on the restored multimedia data. For example, taking a low-latency live broadcast scenario as an example, the multimedia data is a video picture stream collected by the first terminal device 110, referred to as video data. The first terminal device 110 encodes the collected video data according to the multimedia data encoding method provided in any embodiment of the present application, generates encoded video data, and transmits the encoded video data to the second terminal device 120 in the form of a code stream via the network 150. The second terminal device 120 receives a code stream of encoded video data via the network 150, decodes the received encoded video data using a decoding method corresponding to the encoding method, and restores the encoded video data into video data. The second terminal device 120 then plays or displays a corresponding video image based on the video data.

[0077] In one embodiment of the present application, the system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 for bidirectional transmission of encoded multimedia data, such as during a video conference. For bidirectional data transmission, for example, the third terminal device 130 may encode multimedia data (e.g., a captured video image stream) and transmit it to the fourth terminal device 140 via the network 150. The third terminal device 130 may also receive the encoded multimedia data transmitted by the fourth terminal device 140, decode the encoded multimedia data to recover the multimedia data, and display the multimedia content on an accessible display device based on the recovered multimedia data.

[0078] In the embodiment of FIG1 , the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 represents any number of networks, including, for example, wired and / or wireless communication networks, for transmitting encoded multimedia data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140. The communication network 150 may exchange data using circuit-switched and / or packet-switched channels. The network may include a telecommunications network, a local area network, a wide area network, and / or the Internet. For the purposes of this application, unless otherwise explained below, the architecture and topology of network 150 may be immaterial to the operations disclosed herein.

[0079] The multimedia data encoding method provided in this application is described in detail below in conjunction with specific implementation methods.

[0080] FIG2 schematically illustrates a flowchart of a multimedia data encoding method provided by one embodiment of the present application. This method can be implemented by a multimedia data encoding device provided by any embodiment of the present application. The following describes the specific implementation of the method using a multimedia data encoding device (which can be any terminal device exemplified in FIG1 ) as the execution subject. As shown in FIG2 , the multimedia data encoding method provided by an embodiment of the present application includes steps 210 to 240, which are specifically as follows:

[0081] Step 210 : Encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame.

[0082] Specifically, multimedia data is data composed of a series of multimedia frames. For example, video data is data composed of a series of video frames. To encode multimedia data, each multimedia frame is generally encoded in sequence according to the time sequence of the multimedia frames in the multimedia data. To encode a multimedia frame, it is necessary to obtain the corresponding reference frame. A long-term reference frame is a multimedia frame that remains unchanged as a reference frame during a coding process. A long-term reference frame is a multimedia frame that retains more information. For example, a long-term reference frame is a key frame in multimedia data. The encoding device of multimedia data will specify a long-term reference frame before encoding the current multimedia frame to be encoded. During encoding, encoding can be performed based on the difference between the current multimedia frame to be encoded and the long-term reference frame. After the current multimedia frame to be encoded is encoded, it becomes the current encoded multimedia frame.

[0083] In one embodiment of the present application, the reference frames used during multimedia frame encoding include long-term reference frames and short-term reference frames. Short-term reference frames are multimedia frames that change with the position of the current multimedia frame to be encoded. Short-term reference frames are also called normal reference frames. Before encoding multimedia data, the total number of reference frames and the number of long-term reference frames to be used during the encoding process must be set. The total number of reference frames is the sum of the number of long-term reference frames and the number of short-term reference frames, also known as the maximum number of reference frames. The number of short-term reference frames is calculated based on the difference between the total number of reference frames and the number of long-term reference frames. During encoding, a long-term reference frame is obtained based on the number of long-term reference frames. The multimedia frame serving as the long-term reference frame has a specific identifier, and the identifier can be used to determine which multimedia frame is the long-term reference frame. Simultaneously, a short-term reference frame is obtained based on the number of short-term reference frames. Generally, the short-term reference frame is the encoded multimedia frame closest to the current multimedia frame to be encoded. Finally, the current multimedia frame to be encoded is encoded based on the long-term reference frame and the short-term reference frame. For example, assuming the total number of reference frames is 3, the number of long-term reference frames is 1, and the number of long-term reference frames is frame 1, if the current multimedia frame to be encoded is frame t, then the corresponding short-term reference frames are frames t-1 and t-2. After encoding frame t, encoding continues on frame t+1. The short-term reference frames corresponding to frame t+1 are the encoded frames t and t-1. If the long-term reference frame has not been updated at this time, the long-term reference frame remains frame 1.

[0084] In one embodiment of the present application, a multimedia data encoding device may use a reference frame list to record reference frames in the encoding process, which may be divided into a long-term reference frame list and a short-term reference frame list. The long-term reference frame list records the identifiers of long-term reference frames, and the short-term reference frame list records the identifiers of short-term reference frames. During encoding, the corresponding reference frame may be determined by querying the reference frame list. The reference frame list is updated as the encoding process progresses, and may only record reference frames, removing multimedia frame identifiers that are not reference frames from the list. For example, in the aforementioned example, when encoding the t-th frame, the short-term reference frame list records short-term reference frames, including the t-1th frame and the t-2th frame. When encoding the t+1th frame, the short-term reference frame list removes the t-2th frame from the list, and then adds the tth frame to the list, so that the short-term reference frames recorded in the short-term reference frame list are the t-th frame and the t-1th frame.

[0085] Step 220: Obtain the encoding quality of the long-term reference frame and calculate the encoding quality of the currently encoded multimedia frame.

[0086] Specifically, during encoding, the multimedia data encoding device may divide the multimedia frame into multiple coding blocks and then encode each coding block. The encoding quality of the currently encoded multimedia frame is calculated using the same method as the encoding quality of the long-term reference frame. The encoding quality of the multimedia frame can be calculated based on the encoding quality of the coding blocks contained in the multimedia frame.

[0087] In one embodiment of the present application, during the encoding process, the encoding device of multimedia data calculates the encoding quality of the multimedia frame and stores or records it after encoding each multimedia frame. When calculating the encoding quality of the currently encoded multimedia frame, the encoding quality of the long-term reference frame has been calculated, and the encoding quality of the long-term reference frame can be obtained from the corresponding storage or recording location without having to calculate it again.

[0088] In one embodiment of the present application, after encoding a coding block, the encoding device of multimedia data generates a coding cost value for the coding block, which can be simply referred to as the coding cost of the coding block. The coding cost of a coding block generally reflects the difference between the coding block and a reference coding block, which is a coding block in a reference frame. Therefore, the coding cost of a coding block can also reflect the coding quality of the coding block. Generally, the smaller the coding cost of a coding block, the higher the coding quality of the coding block; the larger the coding cost of a coding block, the lower the coding quality of the coding block. The coding cost of the currently encoded multimedia frame can be regarded as the sum of the coding costs of each coding block contained in the currently encoded multimedia frame. Then, the smaller the sum of the coding costs of each coding block, the higher the coding quality of the currently encoded multimedia frame; the larger the sum of the coding costs of each coding block, the lower the coding quality of the currently encoded multimedia frame.

[0089] Step 230: Determine the type of the current encoded multimedia frame according to the motion vector corresponding to the coding block in the current encoded multimedia frame.

[0090] Specifically, there are two types of multimedia frames: dynamic multimedia frames and static multimedia frames. A dynamic multimedia frame means that the content of the multimedia frame is dynamic content. For example, a multimedia frame corresponding to a moving object is generally a dynamic multimedia frame. A static multimedia frame means that the content of the multimedia frame does not change much and can be regarded as static content. For example, a multimedia frame corresponding to a fixed background information is generally a static multimedia frame. The motion vector of the coding block represents the motion characteristics of the coding block, and is also a reflection of the difference between the coding block and the reference frame coding block. The motion vector corresponding to the coding block can be used to determine whether the content represented by the coding block is motion content, and then the type of the currently encoded multimedia frame can be determined.

[0091] In one embodiment of the present application, when determining the type of the currently encoded multimedia frame, whether the currently encoded multimedia frame is a dynamic multimedia frame can be determined based on the number of coding blocks with motion characteristics in the currently encoded multimedia frame. If the currently encoded multimedia frame is not a dynamic multimedia frame, it is determined to be a static multimedia frame. For example, if the number of coding blocks with motion characteristics contained in the currently encoded multimedia frame exceeds or reaches a threshold, the currently encoded multimedia frame is considered to be a dynamic multimedia frame. If the number of coding blocks with motion characteristics contained in the currently encoded multimedia frame does not exceed the threshold, the currently encoded multimedia frame is considered to be a static multimedia frame.

[0092] Step 240: Update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, and encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0093] Specifically, different types of multimedia frames contain different information. Static multimedia frames contain richer information and are more suitable as reference frames. Therefore, when updating a long-term reference frame, the multimedia data encoding device can select a static multimedia frame as the long-term reference frame. Furthermore, considering the importance of reference frames in the encoding process, the multimedia data encoding device can select a multimedia frame with high encoding quality as the reference frame. Specifically, the multimedia data encoding device determines whether the currently encoded multimedia frame is a static multimedia frame based on its type. If the currently encoded multimedia frame is a static multimedia frame, it then determines the difference between the encoding quality of the currently encoded multimedia frame and the encoding quality of the long-term reference frame. If the encoding quality of the currently encoded multimedia frame is greater than that of the long-term reference frame, the long-term reference frame is updated. For example, the currently encoded multimedia frame can be used as the long-term reference frame. Subsequently, the multimedia frame to be encoded in the multimedia data can be encoded based on the updated long-term reference frame. It can be understood that the embodiments of the present application do not limit the execution order of the steps of determining whether the currently encoded multimedia frame is a static multimedia frame and the steps of determining the size of the encoding quality of the currently encoded multimedia frame compared with the encoding quality of the long-term reference frame. For example, the multimedia frame type can be determined first and then the encoding quality can be compared, or the encoding quality can be compared first and then the multimedia frame type can be determined.

[0094] In the technical solution provided in the embodiment of the present application, after encoding the current multimedia frame to be encoded, the encoding quality of the long-term reference frame is obtained, the encoding quality of the current encoded multimedia frame is calculated, and the type of the current encoded multimedia frame is determined according to the motion vector corresponding to the encoding block in the current encoded multimedia frame. Finally, the long-term reference frame is updated according to the encoding quality of the current encoded multimedia frame, the type of the current encoded multimedia frame and the encoding quality of the long-term reference frame, thereby realizing dynamic adjustment of the long-term reference frame during the multimedia data encoding process, avoiding the fixed long-term reference frame from being unable to provide effective reference information for multimedia frames that are farther away, and improving the encoding quality of the multimedia data; at the same time, the long-term reference frame is updated based on the encoding quality and the multimedia frame type, thereby ensuring that the updated long-term reference frame is more reasonable and accurate, thereby improving the encoding quality of the multimedia data.

[0095] FIG3 schematically shows a flowchart of a method for encoding multimedia data provided by an embodiment of the present application. This embodiment is a further refinement of the above embodiment. As shown in FIG3 , the method for encoding multimedia data provided by the embodiment of the present application includes steps 310 to 380, which are as follows:

[0096] Step 310 : Encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame.

[0097] The process of encoding the current multimedia frame to be encoded in this step is the same as step 210 in the aforementioned embodiment, and will not be repeated here.

[0098] Step 320: Calculate the target coding cost of each coding block included in the currently encoded multimedia frame according to the coding mode corresponding to the currently encoded multimedia frame.

[0099] Specifically, depending on whether the encoding process adopts pre-encoding processing, the encoding method is divided into two cases, one is the first encoding method that adopts pre-encoding processing, and the other is the second encoding method that does not adopt pre-encoding processing. Pre-encoding processing (lookahead) is to reasonably allocate the quantization parameters of discrete cosine changes involved in the encoding process of multimedia frames. In order to speed up the pre-encoding processing speed, the pre-encoding processing process will scale the multimedia data. For example, taking video data as an example, the pre-encoding processing is to reduce the width and height of the video image by half. Assuming that the original size of the video image is 1920*1080, the image size during pre-encoding processing is 960*540. The pre-encoding processing (lookahead) is to encode according to the image size of 960*540. The encoding cost obtained by the first encoding method that adopts pre-encoding processing is different from the encoding cost obtained by the second encoding method that does not adopt pre-encoding processing.

[0100] For example, Figure 4 schematically illustrates a multimedia frame encoding process. As shown in Figure 4 , during encoding, the multimedia data encoding device divides a multimedia frame into multiple coding blocks, also known as macroblocks. The multimedia data encoding device sequentially encodes each coding block in sequence, thereby completing the encoding of a multimedia frame. For example, the multimedia data encoding device sequentially encodes each coding block from left to right and from top to bottom.

[0101] In one embodiment of the present application, if the currently encoded multimedia frame does not adopt pre-coding processing during the encoding process, the target coding cost value of the coding block is calculated based on the original coding cost value of the coding block. If the currently encoded multimedia frame adopts pre-coding processing during the encoding process, the target coding cost value of the coding block is calculated based on the original coding cost value and the pre-coding cost value of the coding block. Specifically, if pre-coding processing is not adopted during encoding, then the coding block will have an original coding cost value after the encoding is completed, and then the target coding cost value is calculated based on the original coding cost value. If pre-coding processing is adopted during encoding, then the coding block will have two coding cost values ​​after completion, one is the original coding cost value and the other is the pre-coding cost value, and then the target coding cost value needs to be calculated based on these two coding cost values.

[0102] The embodiments of the present application do not limit the calculation method of the original coding cost value of the coding block, nor do they limit the calculation method of the pre-coding cost value of the coding block. For example, the Sum of Absolute Difference (SAD) algorithm can be used to calculate the original coding cost value and the pre-coding cost value of the coding block, or the Sum of Absolute Transformed Difference (SATDI) algorithm can be used to calculate the original coding cost value and the pre-coding cost value of the coding block. The following describes how to calculate the original coding cost value and the pre-coding cost value of the coding block using the SAD algorithm. Please refer to the following formulas (1) and (2):

[0103] Among them, m and n in formula (1) can represent the size of the coding block and the reference coding block, for example, the size of the coding block and the reference coding block are both 16*16, f(x, y) can represent the pixel value of the coding block at (x, y), g(x, y) can represent the pixel value of the reference coding block at (x, y), and P can represent the original coding cost value of the coding block. That is, the original coding cost value is the absolute difference between the pixel values ​​of each pixel coordinate of the coding block and the reference coding block after encoding a video frame, and the value obtained by summing the absolute differences of all pixel points. The embodiments of the present application do not limit the encoding method, such as intra-frame DC prediction ("DC" represents direct current component) and inter-frame SKIP prediction (SKIP represents skip).

[0104] As can be seen from the above representation, the precoding process includes two steps. The first step is to scale (i.e., downsample) the multimedia data to obtain the scaled multimedia data. The second step is to encode the scaled multimedia data. Therefore, m' and n' in formula (2) can represent the sizes of the scaled coding block and the scaled reference coding block. For example, if the sizes of the coding block and the reference coding block are both 16*16 and the scaling value is 2, the sizes of the scaled coding block and the scaled reference coding block are both 8*8. h(x, y) can represent the pixel value of the scaled coding block at (x, y), d(x, y) can represent the pixel value of the scaled reference coding block at (x, y), and P' can represent the precoding cost of the coding block. In other words, the precoding cost is the absolute difference between the pixel values ​​at each pixel coordinate of the scaled coding block and the scaled reference coding block after precoding a video frame, and the sum of the absolute differences for all pixels.

[0105] In summary, if no precoding process is used when encoding the coding block, the original coding cost of the coding block can be calculated using formula (1); if precoding process is used when encoding the coding block, the original coding cost of the coding block can be calculated using formula (1), and the precoding cost of the coding block can be calculated using formula (2).

[0106] In one embodiment of the present application, if pre-coding processing is not used during encoding, then when calculating the target coding cost, the original coding cost of the coding block is multiplied by the quantization parameter (QP value) corresponding to the coding block to obtain the target coding cost of the coding block. In this process, the quantization parameter is used to further amplify the original coding cost of the coding block to obtain a larger target coding cost. Since the coding cost actually reflects the coding quality, a larger target coding cost can make the difference between multimedia frames of different coding qualities more obvious when the long-term reference frame is subsequently updated according to the coding quality, thereby making it easier to screen out multimedia frames with high coding quality. For example, assuming that the original coding cost value of coding block 1 is 10 and the original coding cost value of coding block 2 is 12, then the difference between the two is 2, which is a small difference and may be considered to be the same quality of the two encoding blocks. If we use the quantization parameter for processing, assuming that the quantization parameter corresponding to coding block 1 is 10, then the target coding cost value of coding block 1 is 100; assuming that the quantization parameter corresponding to coding block 2 is 12, then the target coding cost value of coding block 2 is 144. The difference between the two is 44, which is obviously increasing, and it can be clearly identified that the encoding quality of coding block 1 and coding block 2 is different. Of course, the quantization parameter cannot be set too high. The quantization parameter and encoding quality are negatively correlated, and a too high quantization parameter will reduce the encoding quality.

[0107] In one embodiment of the present application, if precoding is used during encoding, then when calculating the target coding cost, a candidate coding cost is calculated based on the original coding cost of the coding block. The candidate coding cost is calculated in the same manner as the target coding cost when precoding is not used, that is, the product of the original coding cost of the coding block and the quantization parameter corresponding to the coding block is the candidate coding cost. The precoding cost value obtained based on the precoding process for the coding block is obtained, and then the candidate coding cost value and the precoding cost value are weighted and summed to obtain the target coding cost. When the two coding cost values ​​are weighted and summed, the weight of the candidate coding cost value is greater than the weight of the precoding cost value. For example, the weight of the candidate coding cost value can be between 0.7 and 0.8, and the weight of the precoding cost value can be between 0.2 and 0.3.

[0108] Step 330: Calculate the target coding cost value of the currently encoded multimedia frame according to the statistical value of the target coding cost value of each coding block.

[0109] Specifically, after calculating the target coding cost value of each coding block contained in the current encoded multimedia frame, the target coding cost value of each coding block is statistically calculated to obtain the target coding cost value of the current encoded multimedia frame. For example, the sum of the target coding cost values ​​of each coding block is used as the target coding cost value of the current encoded multimedia frame.

[0110] Step 340: Determine the encoding quality of the currently encoded multimedia frame according to the target encoding cost of the currently encoded multimedia frame. The target encoding cost of the currently encoded multimedia frame is negatively correlated with the encoding quality.

[0111] Specifically, a mapping relationship between the target encoding cost and encoding quality can be pre-established, and then the encoding quality of the currently encoded multimedia frame can be determined by querying the mapping relationship. Alternatively, the encoding quality of the currently encoded multimedia frame can be directly represented by the target encoding cost of the currently encoded multimedia frame. In this case, it should be noted that the target encoding cost and encoding quality are negatively correlated.

[0112] Step 350: Determine a target coding block in the current coded multimedia frame whose motion vector amplitude is greater than a preset amplitude according to the motion vector corresponding to the coding block in the current coded multimedia frame.

[0113] Specifically, the target coding block refers to a coding block with a larger motion amplitude, and the motion amplitude can be reflected by the motion vector amplitude. Therefore, the motion vector amplitude of the coding block is calculated, and based on the size of the motion vector amplitude, it can be determined whether the coding block is the target coding block. When the motion vector amplitude is greater than the preset amplitude, it is considered that the motion amplitude of the coding block is large and it is the target coding block. When the motion vector amplitude is less than the preset amplitude, it is considered that the motion amplitude of the coding block is small and it is not the target coding block. It can be understood that the situation where the motion vector amplitude is equal to the preset amplitude can be classified into any one of the following: the motion vector amplitude is greater than the preset amplitude or the motion vector amplitude is less than the preset amplitude. The specific classification can be set according to actual needs.

[0114] In one embodiment of the present application, when encoding a multimedia frame, each coding block of the multimedia frame can be encoded using both inter-frame prediction and intra-frame prediction. Intra-frame prediction refers to predicting the current coding block using coding blocks within the same multimedia frame, while inter-frame prediction refers to predicting the current coding block using coding blocks within other multimedia frames. During the encoding process, the prediction mode with the lowest coding cost is typically selected. It can be seen that intra-frame prediction does not involve encoding between different multimedia frames, while inter-frame prediction can reflect changes between different multimedia frames. Therefore, coding blocks using intra-frame prediction mode generally do not have motion vectors, while coding blocks using inter-frame prediction mode do have motion vectors. Therefore, when determining the target coding block, one can first determine which coding blocks use inter-frame prediction mode, select the coding blocks using inter-frame prediction mode as candidate coding blocks, and then calculate the motion vector amplitude of the candidate coding blocks. When the motion vector amplitude of a candidate coding block is greater than a preset amplitude, the candidate coding block is designated as the target coding block.

[0115] In one embodiment of the present application, the number of motion vectors of a coding block using an inter-frame prediction mode varies according to the size of the predicted pixel range. Generally, when the predicted pixel range is the same as the pixel range of the coding block, the candidate coding block has one motion vector. When the predicted pixel range is smaller than the pixel range of the coding block, the number of motion vectors of the candidate coding block is greater than 1. For example, assuming that the pixel range of a coding block is P16*16 (indicating that it includes 16*16 pixels), if the predicted pixel range is P16*16, the coding block includes 1 motion vector. If the predicted pixel range is P8*8, the coding block includes 4 motion vectors.

[0116] When a candidate coding block has multiple motion vectors, whether the candidate coding block is the target coding block is determined by determining whether the maximum amplitude of the motion vector of the candidate coding block is greater than a preset amplitude. When the maximum amplitude of the motion vector of the candidate coding block is greater than the preset amplitude, the candidate coding block can be considered to be the target coding block; when the maximum amplitude of the motion vector of the candidate coding block is less than the preset amplitude, the candidate coding block is considered not to be the target coding block.

[0117] Step 360: Determine the type of the currently encoded multimedia frame according to the number of target coding blocks contained in the currently encoded multimedia frame.

[0118] Specifically, a multimedia frame may include multiple target coding blocks, and the type of the multimedia frame can be reflected by the number of target coding blocks it contains. If the number of target coding blocks is large, it indicates that most of the content of the multimedia frame is moving, and then the multimedia frame can be considered to be a dynamic multimedia frame. If the number of target coding blocks is small, it indicates that the multimedia frame has less moving content, and then the multimedia frame can be considered to be a static multimedia frame. Therefore, the number of target coding blocks contained in the current encoded multimedia frame can be compared with a preset threshold. If the number of target coding blocks contained in the current encoded multimedia frame is greater than the preset threshold, the current encoded multimedia frame is considered to be a dynamic multimedia frame; if the number of target coding blocks contained in the current encoded multimedia frame is less than or equal to the preset threshold, the current encoded multimedia frame is considered to be a static multimedia frame.

[0119] In one embodiment of the present application, the process of determining the type of the currently encoded multimedia frame includes: calculating the ratio of the number of target coding blocks contained in the currently encoded multimedia frame to the total number of coding blocks contained in the currently encoded multimedia frame; when the ratio is less than or equal to a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame; when the ratio is greater than the preset threshold, determining that the currently encoded multimedia frame is a dynamic multimedia frame.

[0120] Specifically, the encoding device for multimedia data can determine the type of the currently encoded multimedia frame by the proportion of target coding blocks contained in the currently encoded multimedia frame. The ratio of the number of target coding blocks contained in the currently encoded multimedia frame to the total number of coding blocks contained in the currently encoded multimedia frame is the proportion of target coding blocks contained in the currently encoded multimedia frame. When the ratio is less than or equal to a preset threshold, it indicates that the number of target coding blocks is small, and the currently encoded multimedia frame is considered to be a static multimedia frame. When the ratio is greater than the preset threshold, it indicates that the number of target coding blocks is large, and the currently encoded multimedia frame is considered to be a dynamic multimedia frame.

[0121] Step 370: Obtain the encoding quality of the long-term reference frame.

[0122] The method for obtaining the coding quality of the long-term reference frame in this step is the same as the method for obtaining or calculating the coding quality of the long-term reference frame in step 220 in the aforementioned embodiment, and will not be repeated here.

[0123] Step 380: When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as the updated long-term reference frame, and the to-be-encoded multimedia frame in the multimedia data is encoded according to the updated long-term reference frame.

[0124] Specifically, if the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, it indicates that a multimedia frame with better encoding quality has appeared. In this case, the long-term reference frame can be updated based on the multimedia frame with better encoding quality. It should be noted that if the encoding quality is represented by a target encoding cost value, then if the target encoding cost value of the currently encoded multimedia frame is lower than the target encoding cost value of the long-term reference frame, it means that the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame.

[0125] The encoding process of multimedia data consists of two parts: the encoding of syntactic elements and the encoding of the multimedia data itself. The syntactic elements and the encoded data of the multimedia data together constitute the encoded bitstream data. Each multimedia frame of the multimedia data has a corresponding syntactic element. The syntactic element of the multimedia frame includes descriptive information of the multimedia frame. For example, the syntactic element may include whether the multimedia frame is set as a long-term reference frame, the identification information of the multimedia frame, etc. It can be seen that whether a multimedia frame can be used as a long-term reference frame can be determined by the content of the syntactic element corresponding to the multimedia frame. However, during the encoding process, the encoding process of the syntactic element occurs before the encoding process of the multimedia data itself. That is, the syntactic elements of a multimedia frame are written into the bitstream first, and the encoded data of the multimedia frame is written into the bitstream later. Therefore, when the currently encoded multimedia frame is obtained, the syntactic elements of the currently encoded multimedia frame have already been written into the bitstream and cannot be modified. In other words, the currently encoded multimedia frame can no longer be used as a long-term reference frame. However, in multimedia data, the similarity of the content contained in two adjacent multimedia frames is usually high, especially for static multimedia frames. Therefore, it can be considered that the next multimedia frame to be encoded, which has not yet been encoded, has the same encoding quality and type as the currently encoded multimedia frame. Therefore, when updating the long-term reference frame, the next multimedia frame to be encoded, which has not yet been encoded, can be used as the updated long-term reference frame, and the multimedia frames to be encoded in the multimedia data are subsequently encoded based on the updated long-term reference frame.

[0126] When encoding subsequent multimedia frames to be encoded in multimedia data, since the next multimedia frame to be encoded has been used as an updated long-term reference frame, the multimedia frame will generally not be encoded using itself as a reference frame. Therefore, the long-term reference frame corresponding to the next multimedia frame to be encoded is still the long-term reference frame before the update, and the long-term reference frame corresponding to the next multimedia frame to be encoded is the updated long-term reference frame. For example, assuming that the currently encoded multimedia frame is the tth frame and the current long-term reference frame is the kth frame, if the tth frame is a static multimedia frame and the encoding quality of the tth frame is higher than the encoding quality of the kth frame, then the t+1th frame is used as the updated long-term reference frame. When encoding the t+1th frame, the kth frame is used as the long-term reference frame for encoding. When encoding the k+2th frame and subsequent frames, the t+1th frame is used as the long-term reference frame for encoding.

[0127] In the technical solution provided in the embodiment of the present application, the encoding quality of the current multimedia frame to be encoded is calculated based on the target encoding cost value of each encoding block contained in the current multimedia frame to be encoded, which can quantify the encoding quality and improve the accuracy of encoding quality judgment. The type of the current multimedia frame to be encoded is determined by calculating the number of target encoding blocks in the current multimedia frame to be encoded, and the judgment of the multimedia frame type is achieved from the coding block granularity, thereby improving the accuracy of multimedia frame type judgment. When the current encoded multimedia frame is a static multimedia frame and the encoding quality of the current encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next multimedia frame to be encoded of the current encoded multimedia frame is used as the updated long-term reference frame, thereby achieving the purpose of updating the long-term reference frame with a high-quality static multimedia frame, and effectively improving the encoding quality of subsequent multimedia frames.

[0128] In one embodiment of the present application, the long-term reference frame can be updated periodically. The specific process is: when the current number of encoded frames reaches a preset number of frames, the current number of encoded frames is cleared, and the long-term reference frame is updated based on the encoding quality of the current encoded multimedia frame, the type of the current encoded multimedia frame, and the encoding quality of the long-term reference frame; wherein the preset number of frames is an integer and is less than the total number of multimedia frames contained in the multimedia frame group where the current multimedia frame to be encoded is located.

[0129] Multimedia data can be divided into multiple multimedia frame groups, each of which includes multiple multimedia frames. When encoding the multimedia data, the multimedia frames of each multimedia frame group are encoded sequentially, taking the multimedia frame group as a unit. Therefore, the long-term reference frame is periodically updated during the encoding process of a multimedia frame group, and the update period of the long-term reference frame cannot exceed the duration of the multimedia frame group. The preset number of frames is an integer greater than zero and less than the total number of multimedia frames included in the multimedia frame group containing the current multimedia frame to be encoded.

[0130] Since the number of multimedia frames in a multimedia frame group is determinable, the current number of encoded frames can be used to determine whether the long-term reference frame update cycle has arrived. The current number of encoded frames refers to the number of multimedia frames that have been encoded at the current moment. When the current number of encoded frames reaches a preset number, the update cycle has arrived. At this point, the long-term reference frame can be updated based on the encoding quality and multimedia frame type. It should be noted that updating the long-term reference frame does not necessarily require the generation of a new long-term reference frame. When the encoding quality and multimedia frame type do not meet the requirements, the original long-term reference frame remains unchanged. When the update cycle arrives, the current number of encoded frames is reset to facilitate counting for the next update cycle. For example, the preset number of frames is 5, the currently encoded multimedia frame is the fifth static multimedia frame, and the long-term reference frame is frame 0. If the current number of encoded frames reaches 5, the encoding quality of frame 5 and frame 0 is determined. If the encoding quality of frame 5 is higher than that of frame 0, frame 6 is used as the new long-term reference frame. In the next update cycle, the encoding quality of frame 11 is compared with that of frame 6 to update the long-term reference frame. If the encoding quality of frame 5 is lower than that of frame 0, frame 0 remains as the long-term reference frame. In the next update cycle, the encoding quality of frame 10 is compared with that of frame 0 to update the long-term reference frame.

[0131] The technical solution of the present application achieves the purpose of updating the long-term reference frame every preset number of frames. The periodic update can reduce the computing resources consumed by the long-term reference frame update while ensuring the quality of the long-term reference frame.

[0132] FIG5 schematically shows a flowchart of a method for encoding multimedia data provided by an embodiment of the present application. This embodiment is a further refinement of the above embodiment. As shown in FIG5 , the method for encoding multimedia data provided by the embodiment of the present application includes steps 501 to 511, which are specifically as follows:

[0133] Step 501: Encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain the current encoded multimedia frame; wherein, when encoding the multimedia frame to be encoded in the multimedia frame group for the first time, use the first frame in the multimedia frame group as the long-term reference frame, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

[0134] The process of encoding the current multimedia frame to be encoded in this step is the same as step 210 in the aforementioned embodiment, and will not be repeated here.

[0135] In one embodiment of the present application, multimedia frames in multimedia data are in the form of multimedia frame groups, where a multimedia frame group includes multiple multimedia frames, and a multimedia frame group represents a continuous multimedia content. For example, the multimedia data is video data, and a video frame group in the video data is also called a group of pictures (GOP). A GOP includes multiple video frames, and a GOP represents a continuous video picture. During encoding, the data is divided into multimedia frame groups, and different multimedia frame groups correspond to different long-term reference frames. Therefore, the encoding device of the multimedia data can obtain the corresponding long-term reference frame for encoding based on the multimedia frame group in which the current multimedia frame to be encoded is located.

[0136] Taking multimedia data as an example, video data has three types of video frames: I-frames, P-frames, and B-frames. I-frames represent key frames, and during the encoding process, I-frames only reference the already encoded parts of themselves. P-frames represent difference frames, which represent the difference between the current frame and the already encoded frame. During the encoding process, P-frames can only reference the encoded frame that comes before it in the temporal order. B-frames represent bidirectional difference frames, which represent the difference between the current frame and the frames before and after it. During the encoding process, B-frames can reference either the frame that comes before it in the temporal order or the frame that comes after it in the temporal order, as long as the referenced frame is an encoded frame.

[0137] Key frames are usually the frames with the most complete information and the highest quality among various multimedia frames. Therefore, in an embodiment of the present application, the key frames in each multimedia frame group are first set as the long-term reference frames of the multimedia frame group. When a multimedia frame in a multimedia frame group is encoded for the first time, the long-term reference frame obtained is the first frame of the multimedia frame group, and the first frame is also the key frame. Exemplarily, the reference structure shown in Figure 6, when referring to other frames that have been encoded, selects the number of reference frames and the distance between the reference frames and the current encoded frame, which can be regarded as a reference structure. Assume that the total number of reference frames is 2 frames, the long-term reference frame is 1 frame, and the key frame is frame 0 / 0. After the key frame is fixed as the long-term reference frame, if the currently encoded frame is frame 7 / 7, frame 7 / 7 is encoded with reference to frame 0 / 0 and frame 6 / 6.

[0138] Step 502: Calculate the target coding cost of each coding block included in the currently encoded multimedia frame according to the coding mode corresponding to the currently encoded multimedia frame.

[0139] Step 503: Calculate the target coding cost value of the currently encoded multimedia frame according to the statistical value of the target coding cost value of each coding block.

[0140] Step 504: Determine the encoding quality of the currently encoded multimedia frame according to the target encoding cost of the currently encoded multimedia frame. The target encoding cost of the currently encoded multimedia frame is negatively correlated with the encoding quality.

[0141] Step 505: Determine a target coding block in the current coded multimedia frame whose motion vector amplitude is greater than a preset amplitude according to the motion vector corresponding to the coding block in the current coded multimedia frame.

[0142] Step 506: Determine the type of the currently encoded multimedia frame according to the number of target coding blocks contained in the currently encoded multimedia frame.

[0143] Step 507: Obtain the encoding quality of the long-term reference frame.

[0144] Steps 502 to 507 are the same as steps 320 to 370 in the aforementioned embodiment and are not described again here.

[0145] Step 508: When the current number of encoded frames reaches a preset number of frames, the current number of encoded frames is cleared.

[0146] Step 508 can refer to the description of periodic updating of long-term reference frames in the above embodiment, which will not be repeated here. If the number of currently encoded frames does not reach the preset number of frames, encoding is continued without updating the long-term reference frames.

[0147] Step 509: When the long-term reference frame is a key frame, the quality weight of the long-term reference frame is obtained according to the time interval between the current encoded multimedia frame and the long-term reference frame. The quality weight is negatively correlated with the time interval.

[0148] Specifically, considering that a key frame is usually the frame with the highest encoding quality in a multimedia frame group, in order to avoid the encoding quality of subsequent multimedia frames being always lower than that of the key frame, which would cause the long-term reference frame to be unable to be updated, when the long-term reference frame is a key frame, a quality weight is assigned to the key frame so that the encoding quality of the key frame matches the current moment. Although the key frame is the frame that retains the most complete information in the multimedia frame group, as time moves, the farther the frame is from the key frame, the smaller the correlation between the key frame and the key frame, that is, the key frame is difficult to provide effective reference information for distant multimedia frames. Therefore, the quality weight of the key frame is obtained based on the time interval between the current encoded multimedia frame and the current long-term reference frame (i.e., the key frame), and the quality weight is negatively correlated with the time interval. At the same time, the quality weight is in the range of (0,1). For example, the multimedia frame group has a total of 24 multimedia frames. If the currently encoded multimedia frame is the 4th frame, the quality weight can be set to 0.8; if the currently encoded multimedia frame is the 8th frame, the quality weight can be set to 0.6; if the currently encoded multimedia frame is the 12th frame, the quality weight can be set to 0.4; if the currently encoded multimedia frame is the 16th frame, the quality weight can be set to 0.2.

[0149] If the long-term reference frame is not a key frame, the process directly proceeds to step 511 .

[0150] Step 510: Generate a target coding quality of the long-term reference frame according to the quality weight and the coding quality of the long-term reference frame, wherein the target coding quality of the long-term reference frame is less than the coding quality of the long-term reference frame.

[0151] Specifically, the target encoding quality of the long-term reference frame (i.e., the key frame) is obtained by multiplying the quality weight by the encoding quality of the current long-term reference frame. Because the quality weight is less than 1, the target encoding quality of the long-term reference frame is lower than the encoding quality of the long-term reference frame. Assuming the encoding quality of the key frame is 100 and the quality weight is 0.8, the target encoding quality of the key frame is 80.

[0152] Step 511: Update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame, and encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0153] Step 511 is the same as step 380 in the above embodiment and will not be described in detail herein. If the long-term reference frame is not a key frame, the target encoding quality of the long-term reference frame is the same as the encoding quality of the long-term reference frame.

[0154] For example, taking the example of Figure 6 as an example, the current long-term reference frame is the key frame 0 / 0, and the currently encoded multimedia frame is the 7 / 7 frame. Assuming that the 7 / 7 frame is a static multimedia frame, the encoding quality of the 7 / 7 frame is 90, and the encoding quality of the 0 / 0 frame is 100. If the key frame is not assigned a quality weight, then the encoding quality of the 7 / 7 frame is less than the encoding quality of the 0 / 0 frame, and a new long-term reference frame cannot be generated. If the key frame is assigned a quality weight of 0.8, the target encoding quality of the 0 / 0 frame is 80. At this time, the encoding quality of the 7 / 7 frame is greater than the encoding quality of the 0 / 0 frame, so the 8 / 8 frame can be used as the updated long-term reference frame. Subsequently, when encoding the 9 / 9 frame, the 7 / 7 frame is used as the long-term reference frame, and the updated reference structure is shown in Figure 7.

[0155] In the technical solution provided in the embodiment of the present application, the key frame of the multimedia frame group is set as the long-term reference frame, and when updating, the key frame is assigned a quality weight that is negatively correlated with time, so as to avoid the situation where the encoding quality of subsequent multimedia frames cannot always be higher than the encoding quality of the key frame, resulting in the inability to update the long-term reference frame, thereby improving the accuracy of the long-term reference frame update.

[0156] The implementation process of the technical solution of this application will be described below with a specific embodiment.

[0157] Figure 8 schematically shows a flow chart of a method for encoding multimedia data provided by an embodiment of the present application. This embodiment takes a low-latency live broadcast scenario using the H264 encoding protocol for video compression as an example, and the multimedia data is video data. A video includes multiple GOPs, a GOP includes at least one I frame, and the remaining frames can be P frames or B frames. For example, if the GOP is 24 and the frame rate of the current video is 24fps, then there will be an I frame every 1 second, and there will be 23 P frames or B frames between two I frames. For low-latency live video, there are only I frames and P frames in a GOP. If the GOP is 24, then a GOP consists of one I frame and twenty-three P frames.

[0158] As shown in FIG8 , the multimedia data encoding method provided in the embodiment of the present application includes the following steps:

[0159] Step 801: Set the first I frame as a long-term reference frame. That is, set the first I frame in the GOP as a long-term reference frame. In the H264 coding standard, the I frame contains a syntax element "long_term_reference_flag", which is used to indicate whether the long-term reference frame mechanism is used. When long_term_reference_flag is 0, it indicates that the long-term reference frame mechanism is disabled; when long_term_reference_flag is 1, it indicates that the long-term reference frame mechanism is enabled, and the I frame automatically becomes a long-term reference frame. During the encoding process, set long_term_reference_flag to 1 and write it into the encoded bitstream to make the I frame a long-term reference frame.

[0160] Step 802: Encode the first I frame. The encoding of the I frame only refers to the already encoded part of the frame.

[0161] Step 803: Encode the P frame. The P frame encoding refers to the long-term reference frame and the short-term reference frame. The short-term reference frame is the frame or frames closest to the P frame. When encoding the P frame for the first time, the long-term reference frame is the I frame. If the maximum number of reference frames is 2, and the t-th frame is currently being encoded, then the reference frame of the t-th frame is not only the t-1th frame, but also the 1st frame in the current GOP (sometimes also recorded as the 0th frame): the I frame. Since the encoding quality of the I frame is higher than that of the P frame, the t-th frame that refers to the I frame and the t-1th frame will usually have higher encoding quality than the reference frames t-1 and t-2.

[0162] Step 804: Count the coding cost and the number of macroblocks with large motion.

[0163] During the encoding process, the encoding cost is calculated differently depending on whether lookahead is enabled. During the encoding process, the encoder divides a frame into multiple coding blocks, called macroblocks. The encoder encodes each macroblock sequentially, from left to right and from top to bottom. This is shown in Figure 4 below. After encoding each macroblock, the corresponding raw coding cost is obtained. This raw coding cost is multiplied by the quantization parameter used to encode the macroblock to obtain the target coding cost for that macroblock. If lookahead is disabled, the target coding cost for the frame is calculated by summing the target coding costs of all macroblocks in the frame. If the precoding process (lookahead) is enabled, in addition to counting the coding cost of disabling the precoding process (lookahead) (as the first coding cost), the precoding cost of the coded frame obtained in the precoding process (lookahead) must also be counted. The precoding cost of the coded frame obtained by the precoding process (lookahead) is recorded as α, and the first coding cost of the coded frame obtained by disabling the precoding process (lookahead) is recorded as β. The target coding cost γ of a coded frame is calculated by weighting α and β. The size of the target coding cost of the coded frame can reflect the quality of the coding to a certain extent. Generally speaking, the smaller the coding cost, the better the coding quality. This way of judging the coding quality is relatively simple, consisting of only a few steps of multiplication and addition. It does not require additional time-consuming processing and will not increase computing power significantly.

[0164] The number of large motion macroblocks is used to determine whether the encoded frame is a static video frame. Generally speaking, the quality of static video frames is higher than that of dynamic video frames, and static video frames contain richer information that can be used for reference. During the encoding process, macroblocks can perform inter-frame prediction and intra-frame prediction, and select the prediction method with the lowest encoding cost. For macroblocks that select inter-frame prediction, the macroblock will have one or more motion vectors that point to the area selected as the reference in the reference frame. The amplitude of each motion vector of the macroblock is counted, and a threshold TH_MV is set. If the maximum value of the amplitude of each motion vector of the macroblock exceeds TH_MV, the macroblock is considered to be a large motion macroblock.

[0165] Step 805: Determine whether a preset number of P frames have been encoded. Specifically, the encoding quality is compared every preset number of frames, and the encoded frame with the best quality is selected. The preset number of frames actually represents the interval between two consecutive encoding quality comparisons, so it can be represented by Gap. The preset number of frames, Gap, is an integer greater than 0, with a maximum value less than the number of frames in a GOP. If the preset number of P frames (Gap) has been encoded, proceed to step 806. If the preset number of P frames (Gap) has not been encoded, return to step 802 and continue encoding P frames.

[0166] Step 806: Select high-quality static P frames.

[0167] A static P-frame is a P-frame that is considered static. The percentage of macroblocks with large motion in the coded frame is calculated, and a threshold TH_P is set. If the percentage of macroblocks with large motion is less than TH_P, the coded frame is considered static. The P-frame encoding quality is also compared with the encoding quality of the current long-term reference frame, and the frame with higher quality is selected as the updated long-term reference frame. If the current long-term reference frame is an I-frame, a quality weight is assigned to the I-frame before comparing its encoding quality with the P-frame.

[0168] In one embodiment of the present application, to achieve more accurate selection, high-quality static P frames can be selected using a machine learning model. A machine learning model can be constructed, using various statistical characteristics of coded frames as input parameters to select high-quality static P frames.

[0169] Step 807: Set the frame following the high-quality static P frame as the long-term reference frame and delete the previous long-term reference frame. The high-quality static P frame can be selected through the aforementioned steps. Due to the limitation that the syntax elements are written before encoding, it is impossible to set the high-quality static P frame as the long-term reference frame after encoding is completed. However, the quality and content of two adjacent frames are very similar, especially for static scenes. Therefore, the frame following the high-quality static P frame that has not yet been encoded can be set as the long-term reference frame. The syntax element adaptive_ref_pic_marking_mode_flag contained in the frame following the high-quality static P frame (also a P frame) is marked as 1, allowing it to be used as a long-term reference frame. Then, in the bitstream, the syntax element is used to set the P frame as the long-term reference frame, and the I frame or P frame that was originally used as the long-term reference frame is removed from the reference frame queue. For example, Figure 9 schematically illustrates the long-term reference frame replacement process using the technical solution of the present application, where the long-term reference frame is replaced by the I frame with the t+1 frame.

[0170] The technical solution of this application targets the characteristics of low-latency live video, namely, the characteristics of mostly static scenes and short GOP length. On the one hand, it actively selects high-quality I frames as reference frames to improve the encoding quality of subsequent video frames. On the other hand, by analyzing the characteristics of the already encoded P frames, namely the quality of the P frames and the video content of the P frames, high-quality static P frames are selected as reference frames. These P frames are used to replace the I frames used as reference frames, providing a higher-quality reference for subsequent P frames, thereby improving the encoding quality of subsequent video frames. Based on the above two aspects, the H264 encoder will be able to ensure that the reference frames always have high quality, thereby improving the BD-rate of the encoded video sequence, such as PSNR (peak signal to noise ratio), SSIM (structural similarity index measurement), and VMAF (Video Multimethod Assessment Fusion). BD-rate is an indicator for evaluating the performance of a video coding algorithm. A negative BD-rate indicates that the coding performance of the optimized algorithm has been improved.

[0171] Experimental verification shows that in a low-latency scenario, using the technical solution of this application, the average BD-rate gains corresponding to PSNR, SSIM, and VMAF of the test sequence after H264 encoding are -0.35%, -0.4%, and -0.48%, respectively. It can be seen that the technical solution of this application can effectively improve the encoding quality of video data without increasing latency or bit rate. At the same time, the calculation process involved in this application is simple and basically does not increase computing power consumption.

[0172] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0173] The following describes an embodiment of a device of the present application, which can be used to execute the multimedia data encoding method of the above-mentioned embodiment of the present application. FIG10 schematically shows a block diagram of the structure of the multimedia data encoding device provided in the embodiment of the present application. As shown in FIG10, the multimedia data encoding device provided in the embodiment of the present application includes:

[0174] The encoding module 1010 is configured to encode the current multimedia frame to be encoded according to a long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame;

[0175] The coding quality calculation module 1020 is configured to obtain the coding quality of the long-term reference frame and calculate the coding quality of the currently coded multimedia frame;

[0176] A multimedia frame type calculation module 1030, configured to determine the type of the currently encoded multimedia frame according to the motion vector corresponding to the coding block in the currently encoded multimedia frame;

[0177] The reference frame update module 1040 is used to update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, and encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0178] In one embodiment of the present application, the encoding quality calculation module 1020 includes:

[0179] a coding cost calculation unit, configured to calculate a target coding cost value of each coding block included in the currently encoded multimedia frame according to a coding mode corresponding to the currently encoded multimedia frame;

[0180] a statistical unit, configured to calculate a target coding cost value of the currently encoded multimedia frame according to a statistical value of the target coding cost value of each coding block;

[0181] The quality calculation unit is configured to determine the encoding quality of the currently encoded multimedia frame according to a target encoding cost value of the currently encoded multimedia frame, wherein the target encoding cost value of the currently encoded multimedia frame is negatively correlated with the encoding quality.

[0182] In one embodiment of the present application, the encoding cost calculation unit includes:

[0183] A first subunit is configured to calculate a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block if no pre-encoding process is used in the encoding process of the currently encoded multimedia frame;

[0184] The second subunit is configured to calculate a target coding cost value of the coding block according to the original coding cost value of the coding block and the pre-coding cost value of the coding block if pre-coding processing is adopted in the coding process of the currently coded multimedia frame.

[0185] In one embodiment of the present application, the first subunit is specifically configured to:

[0186] The original coding cost value of the coding block is multiplied by a quantization parameter corresponding to the coding block to obtain a target coding cost value of the coding block, where the quantization parameter is used to amplify the original coding cost value of the coding block.

[0187] In one embodiment of the present application, the second subunit is specifically configured to:

[0188] Multiplying the original coding cost value of the coding block by the quantization parameter corresponding to the coding block to obtain a candidate coding cost value of the coding block;

[0189] A weighted sum is performed on the candidate coding cost values ​​of the coding block and the precoding cost value of the coding block to obtain a target coding cost value of the coding block, wherein the weight of the candidate coding cost value is greater than the weight of the precoding cost value.

[0190] In one embodiment of the present application, the multimedia frame type calculation module 1030 includes:

[0191] a target coding block determining unit, configured to determine, based on a motion vector corresponding to a coding block in the currently encoded multimedia frame, a target coding block having a motion vector amplitude greater than a preset amplitude in the currently encoded multimedia frame;

[0192] The type determining unit is configured to determine the type of the currently encoded multimedia frame according to the number of target coding blocks contained in the currently encoded multimedia frame.

[0193] In one embodiment of the present application, the target coding block determination unit is specifically configured to:

[0194] Extracting a candidate coding block to be encoded based on an inter-frame prediction mode from a plurality of coding blocks included in the currently encoded multimedia frame;

[0195] Calculating the magnitude of the motion vector corresponding to the candidate coding block;

[0196] When the magnitude of the motion vector corresponding to the candidate coding block is greater than a preset magnitude, it is determined that the candidate coding block belongs to the target coding block.

[0197] In one embodiment of the present application, the type determination unit is specifically configured to:

[0198] Calculating a ratio of the number of target coding blocks included in the current encoded multimedia frame to the total number of coding blocks included in the current encoded multimedia frame;

[0199] When the ratio is less than a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame;

[0200] When the ratio is greater than a preset threshold, it is determined that the currently encoded multimedia frame is a dynamic multimedia frame.

[0201] In one embodiment of the present application, the reference frame updating module 1040 is specifically configured to:

[0202] When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as the updated long-term reference frame.

[0203] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; the reference frame updating module 1040 is specifically configured to:

[0204] When the number of currently encoded frames reaches a preset number of frames, clearing the currently encoded number of frames and updating the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame;

[0205] The preset number of frames is an integer and is smaller than the total number of multimedia frames included in the multimedia frame group where the current multimedia frame to be encoded is located.

[0206] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; and the apparatus further includes:

[0207] The key frame setting module is used to use the first frame in the multimedia frame group as a long-term reference frame when encoding the multimedia frame to be encoded in the multimedia frame group for the first time, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

[0208] In one embodiment of the present application, the reference frame updating module 1040 is specifically configured to:

[0209] If the long-term reference frame is a key frame, obtaining a quality weight of the long-term reference frame according to a time interval between the current encoded multimedia frame and the long-term reference frame, wherein the quality weight is negatively correlated with the time interval;

[0210] generating a target encoding quality of the long-term reference frame according to a quality weight and an encoding quality of the long-term reference frame, wherein the target encoding quality of the long-term reference frame is less than the encoding quality of the long-term reference frame;

[0211] The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame.

[0212] In one embodiment of the present application, the encoding module 1010 is specifically configured to:

[0213] Obtaining the number of short-term reference frames according to the difference between the total number of reference frames corresponding to the multimedia data and the number of long-term reference frames;

[0214] using a plurality of encoded multimedia frames corresponding to the number of short-term reference frames and located before the current multimedia frame to be encoded as short-term reference frames;

[0215] The current multimedia frame to be encoded is encoded according to the short-term reference frame and the long-term reference frame.

[0216] The specific details of the multimedia data encoding device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0217] FIG11 schematically shows a block diagram of a computer system structure of an electronic device for implementing an embodiment of the present application.

[0218] It should be noted that the computer system 1100 of the electronic device shown in FIG11 is merely an example and should not impose any limitation on the functions and scope of use of the embodiments of the present application.

[0219] As shown in Figure 11, the computer system 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1102 or the program loaded from the storage part 1108 into the random access memory (RAM) 1103. Various programs and data required for system operation are also stored in the random access memory 1103. The CPU 1101, the read-only memory 1102, and the random access memory 1103 are connected to each other via a bus 1104. An input / output interface 1105 (i.e., an I / O interface) is also connected to the bus 1104.

[0220] The following components are connected to the input / output interface 1105: an input section 1106 including a keyboard, a mouse, and the like; an output section 1107 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1108 including a hard disk; and a communication section 1109 including a network interface card such as a local area network card or a modem. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output interface 1105 as needed. Removable media 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1110 as needed, so that computer programs read therefrom can be installed into the storage section 1108 as needed.

[0221] In particular, according to embodiments of the present application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 1109 and / or installed from removable media 1111. When the computer program is executed by the central processing unit 1101, the various functions defined in the system of the present application are performed.

[0222] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0223] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0224] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0225] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0226] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0227] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for encoding multimedia data, characterized in that: The method is performed by a multimedia data encoding device, and includes: Encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame; Acquire the encoding quality of the long-term reference frame, and calculate the encoding quality of the currently encoded multimedia frame; Determining the type of the current encoded multimedia frame according to the motion vector corresponding to the encoding block in the current encoded multimedia frame; The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, and the multimedia frame to be encoded in the multimedia data is encoded according to the updated long-term reference frame.

2. The method for encoding multimedia data according to claim 1, characterized in that: The calculating the encoding quality of the currently encoded multimedia frame comprises: Calculating, according to the encoding mode corresponding to the currently encoded multimedia frame, a target encoding cost value of each encoding block included in the currently encoded multimedia frame; Calculating a target encoding cost value of the currently encoded multimedia frame according to the statistical value of the target encoding cost value of each encoding block; The encoding quality of the currently encoded multimedia frame is determined according to the target encoding cost value of the currently encoded multimedia frame, and the target encoding cost value of the currently encoded multimedia frame is negatively correlated with the encoding quality.

3. The method for encoding multimedia data according to any one of claims 1 to 2, characterized in that: The calculating, according to the encoding mode corresponding to the currently encoded multimedia frame, a target encoding cost value of each encoding block included in the currently encoded multimedia frame comprises: If the currently encoded multimedia frame is not pre-encoded during the encoding process, calculating a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block; If the currently encoded multimedia frame adopts pre-encoding processing during the encoding process, the target encoding cost value of the encoding block is calculated according to the original encoding cost value of the encoding block and the pre-encoding cost value of the encoding block.

4. The method for encoding multimedia data according to any one of claims 1 to 3, characterized in that: The calculating the target encoding cost value of the encoding block according to the original encoding cost value of the encoding block includes: An original coding cost value of a coding block is multiplied by a quantization parameter corresponding to the coding block to obtain a target coding cost value of the coding block, wherein the quantization parameter is used to amplify the original coding cost value of the coding block.

5. The method for encoding multimedia data according to any one of claims 1 to 4, characterized in that: The calculating the target coding cost value of the coding block according to the original coding cost value of the coding block and the pre-coding cost value of the coding block includes: Multiplying an original coding cost value of a coding block by a quantization parameter corresponding to the coding block to obtain a candidate coding cost value of the coding block; A weighted sum is performed on the candidate coding cost value of the coding block and the pre-coding cost value of the coding block to obtain a target coding cost value of the coding block, wherein the weight of the candidate coding cost value is greater than the weight of the pre-coding cost value.

6. The method for encoding multimedia data according to any one of claims 1 to 5, characterized in that: The determining the type of the currently encoded multimedia frame according to the motion vector corresponding to the encoding block in the currently encoded multimedia frame includes: Determine, according to the motion vector corresponding to the coding block in the current coded multimedia frame, a target coding block having a motion vector amplitude greater than a preset amplitude in the current coded multimedia frame; The type of the currently encoded multimedia frame is determined according to the number of target encoding blocks contained in the currently encoded multimedia frame.

7. The method for encoding multimedia data according to any one of claims 1 to 6, characterized in that: The step of determining, according to the motion vector corresponding to the coding block in the currently encoded multimedia frame, a target coding block having a motion vector amplitude greater than a preset amplitude in the currently encoded multimedia frame comprises: Extracting a candidate coding block to be encoded based on the inter-frame prediction mode from a plurality of coding blocks included in the currently encoded multimedia frame; Calculating the magnitude of the motion vector corresponding to the candidate coding block; When the magnitude of the motion vector corresponding to the candidate coding block is greater than a preset magnitude, it is determined that the candidate coding block belongs to the target coding block.

8. The method for encoding multimedia data according to any one of claims 1 to 7, characterized in that: The determining the type of the currently encoded multimedia frame according to the number of target coding blocks included in the currently encoded multimedia frame comprises: Calculating a ratio of the number of target coding blocks included in the current encoded multimedia frame to the total number of coding blocks included in the current encoded multimedia frame; When the ratio is less than or equal to a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame; When the ratio is greater than a preset threshold, it is determined that the currently encoded multimedia frame is a dynamic multimedia frame.

9. The method for encoding multimedia data according to any one of claims 1 to 8, characterized in that: The updating of the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame comprises: When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as an updated long-term reference frame.

10. The method for encoding multimedia data according to any one of claims 1 to 9, characterized in that: The multimedia data includes a plurality of multimedia frame groups; The updating of the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame comprises: When the number of currently encoded frames reaches a preset number of frames, the currently encoded number of frames is cleared, and the long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame; The preset frame number is an integer and is smaller than the total number of multimedia frames included in the multimedia frame group where the current multimedia frame to be encoded is located; the multimedia frame group where the current multimedia frame to be encoded is located belongs to the multimedia data including multiple multimedia frame groups.

11. The multimedia data encoding method according to any one of claims 1 to 10, characterized in that: The multimedia data includes a plurality of multimedia frame groups; Before encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, the method further includes: When encoding the to-be-encoded multimedia frame in the multimedia frame group for the first time, the first frame in the multimedia frame group is used as a long-term reference frame, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

12. The multimedia data encoding method according to any one of claims 1 to 11, characterized in that: The updating of the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame comprises: If the long-term reference frame is a key frame, obtaining a quality weight of the long-term reference frame according to a time interval between the current encoded multimedia frame and the long-term reference frame, wherein the quality weight is negatively correlated with the time interval; generating a target encoding quality of the long-term reference frame according to a quality weight and an encoding quality of the long-term reference frame, wherein the target encoding quality of the long-term reference frame is less than an encoding quality of the long-term reference frame; The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame.

13. The method for encoding multimedia data according to any one of claims 1 to 12, characterized in that: The step of encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data comprises: The number of short-term reference frames is obtained according to the difference between the total number of reference frames corresponding to the multimedia data and the number of long-term reference frames; Using the encoded multimedia frames that are located before the current multimedia frame to be encoded and adjacent to the current multimedia frame to be encoded and whose number is the number of the short-term reference frames as short-term reference frames; The current multimedia frame to be encoded is encoded according to the short-term reference frame and the long-term reference frame.

14. A multimedia data encoding device, characterized in that: include: An encoding module, configured to encode the current multimedia frame to be encoded according to a long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, to obtain a current encoded multimedia frame; A coding quality calculation module, used to obtain the coding quality of the long-term reference frame and calculate the coding quality of the currently encoded multimedia frame; A multimedia frame type calculation module, configured to determine the type of the currently encoded multimedia frame according to a motion vector corresponding to a coding block in the currently encoded multimedia frame; A reference frame updating module is used to update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, and encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

15. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for encoding multimedia data described in any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor executes the executable instructions so that the electronic device executes the multimedia data encoding method described in any one of claims 1 to 13.

17. A computer program product, characterized in that The computer program product includes computer instructions stored in a computer-readable storage medium; The processor of the computer device reads and executes the computer instructions from the computer-readable storage medium, so that the computer device executes the multimedia data encoding method according to any one of claims 1 to 13.

Images