Multimedia data coding method and device, readable medium and electronic equipment

Abstract

The invention discloses a multimedia data coding method and device, a computer readable medium and electronic equipment, and the method comprises the steps: carrying out the coding of a current to-be-coded multimedia frame according to a long-term reference frame corresponding to the current to-be-coded multimedia frame in multimedia data, and obtaining a current coded multimedia frame; calculating the coding quality of the currently coded multimedia frame and the coding quality of the long-term reference frame; determining the type of the currently coded multimedia frame according to the motion vector corresponding to the coding block in the currently coded multimedia frame; and updating the long-term reference frame according to the coding quality of the currently coded multimedia frame, the type of the currently coded multimedia frame and the coding quality of the long-term reference frame. According to the technical scheme, it is guaranteed that the updated long-term reference frame is more reasonable and accurate, and then the coding quality of multimedia data can be improved.

Description

Technical Field

[0001] The present application belongs to the field of multimedia technology, and specifically relates to a method, device, computer-readable medium and electronic device for encoding multimedia data. Background Art

[0002] In order to efficiently transmit video data, the video data is usually encoded first and then the encoded video data is transmitted. During the video encoding process, in order to reduce the volume of the encoded video data, the video data is usually broken down into multiple frames, and then the differences between frames are encoded. Therefore, when encoding, the selection of reference frames will have an important impact on the quality of the encoded video. Generally, when encoding a certain frame in a video, a video frame that is closer to it will be selected as a reference frame. However, this encoding method has the problem of error transmission, resulting in the lower encoding quality of the video frame in the encoding process. Therefore, this encoding method needs to be improved. Summary of the invention

[0003] The purpose of the present application is to provide a method, device, computer-readable medium and electronic device for encoding multimedia data, which can improve the encoding quality of multimedia data.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a method for encoding multimedia data is provided, including:

[0006] Encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame;

[0007] Calculating the encoding quality of the currently encoded multimedia frame and the encoding quality of the long-term reference frame;

[0008] Determining the type of the current encoded multimedia frame according to the motion vector corresponding to the encoding block in the current encoded multimedia frame;

[0009] The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0010] According to one aspect of an embodiment of the present application, there is provided a multimedia data encoding device, including:

[0011] An encoding module, configured to encode the current multimedia frame to be encoded according to a long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, to obtain a current encoded multimedia frame;

[0012] A coding quality calculation module, used for calculating the coding quality of the currently coded multimedia frame and the coding quality of the long-term reference frame;

[0013] A multimedia frame type calculation module, configured to determine the type of the currently encoded multimedia frame according to a motion vector corresponding to a coding block in the currently encoded multimedia frame;

[0014] A reference frame updating module is used to update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0015] In one embodiment of the present application, the encoding quality calculation module includes:

[0016] A coding cost calculation unit, configured to calculate a target coding cost value of each coding block included in the currently encoded multimedia frame according to a coding mode corresponding to the currently encoded multimedia frame;

[0017] A statistical unit, configured to calculate a target coding cost value of the currently encoded multimedia frame according to a statistical value of a target coding cost value of each coding block;

[0018] The quality calculation unit is used to determine the encoding quality of the currently encoded multimedia frame according to the target encoding cost value of the currently encoded multimedia frame, and the target encoding cost value is negatively correlated with the encoding quality.

[0019] In one embodiment of the present application, the encoding cost calculation unit includes:

[0020] A first subunit is configured to calculate a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block if the currently encoded multimedia frame does not adopt pre-encoding processing during the encoding process;

[0021] The second subunit is configured to calculate a target encoding cost value of the encoding block according to an original encoding cost value and a pre-encoding cost value of the encoding block if the pre-encoding process is adopted in the encoding process of the currently encoded multimedia frame.

[0022] In one embodiment of the present application, the first subunit is specifically used for:

[0023] The original coding cost value of the coding block is multiplied by a quantization parameter corresponding to the coding block to obtain a target coding cost value of the coding block, wherein the quantization parameter is used to amplify the original coding cost value of the coding block.

[0024] In one embodiment of the present application, the second subunit is specifically used for:

[0025] Multiplying the original coding cost value of the coding block by the quantization parameter corresponding to the coding block to obtain a candidate coding cost value of the coding block;

[0026] A weighted sum is performed on the candidate coding cost values ​​and the pre-coding cost value of the coding block to obtain a target coding cost value of the coding block, wherein the weight of the candidate coding cost value is greater than the weight of the pre-coding cost value.

[0027] In one embodiment of the present application, the multimedia frame type calculation module includes:

[0028] a target coding block determining unit, configured to determine a target coding block having a motion vector amplitude greater than a preset amplitude in the current encoded multimedia frame according to a motion vector corresponding to the coding block in the current encoded multimedia frame;

[0029] The type determination unit is used to determine the type of the currently encoded multimedia frame according to the number of target coding blocks contained in the currently encoded multimedia frame.

[0030] In one embodiment of the present application, the target coding block determination unit is specifically used to:

[0031] Extracting a candidate coding block to be encoded based on the inter-frame prediction mode from a plurality of coding blocks included in the currently encoded multimedia frame;

[0032] Calculating the magnitude of the motion vector corresponding to the candidate coding block;

[0033] When the magnitude of the motion vector corresponding to the candidate coding block is greater than a preset magnitude, it is determined that the candidate coding block belongs to the target coding block.

[0034] In one embodiment of the present application, the type determination unit is specifically used to:

[0035] Calculating a ratio of the number of target coding blocks included in the current encoded multimedia frame to the total number of coding blocks included in the current encoded multimedia frame;

[0036] When the ratio is less than a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame;

[0037] When the ratio is greater than a preset threshold, it is determined that the currently encoded multimedia frame is a dynamic multimedia frame.

[0038] In one embodiment of the present application, the reference frame updating module is specifically used for:

[0039] When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as an updated long-term reference frame.

[0040] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; the reference frame updating module is specifically used for:

[0041] When the number of currently encoded frames reaches a preset number of frames, the currently encoded number of frames is cleared, and the long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame;

[0042] The preset frame number is an integer and is smaller than the total number of multimedia frames included in the multimedia frame group where the current multimedia frame to be encoded is located.

[0043] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; the apparatus further includes:

[0044] The key frame setting module is used to use the first frame in the multimedia frame group as a long-term reference frame when encoding the multimedia frame to be encoded in the multimedia frame group for the first time, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

[0045] In one embodiment of the present application, the reference frame updating module is specifically used for:

[0046] If the long-term reference frame is a key frame, obtaining a quality weight of the long-term reference frame according to a time interval between the current encoded multimedia frame and the long-term reference frame, wherein the quality weight is negatively correlated with the time interval;

[0047] generating a target encoding quality of the long-term reference frame according to a quality weight and an encoding quality of the long-term reference frame, wherein the target encoding quality of the long-term reference frame is less than an encoding quality of the long-term reference frame;

[0048] The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame.

[0049] In one embodiment of the present application, the encoding module is specifically used for:

[0050] The number of short-term reference frames is obtained according to the difference between the total number of reference frames corresponding to the multimedia data and the number of long-term reference frames;

[0051] Using a plurality of encoded multimedia frames corresponding to the number of short-term reference frames and located before the current multimedia frame to be encoded as short-term reference frames;

[0052] The current multimedia frame to be encoded is encoded according to the short-term reference frame and the long-term reference frame.

[0053] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for encoding multimedia data in the above technical solution is implemented.

[0054] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions so that the electronic device executes a method for encoding multimedia data as in the above technical solution.

[0055] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the multimedia data encoding method in the above technical solution.

[0056] In the technical solution provided in the embodiment of the present application, after encoding the current multimedia frame to be encoded, the encoding quality of the current encoded multimedia frame and the encoding quality of the long-term reference frame are calculated, and the type of the current encoded multimedia frame is determined according to the motion vector corresponding to the encoding block in the current encoded multimedia frame. Finally, the long-term reference frame is updated according to the encoding quality of the current encoded multimedia frame, the type of the current encoded multimedia frame and the encoding quality of the long-term reference frame, thereby realizing dynamic adjustment of the long-term reference frame during the multimedia data encoding process, avoiding the fixed long-term reference frame from being unable to provide effective reference information for multimedia frames that are far away, and improving the encoding quality of the multimedia data; at the same time, the long-term reference frame is updated based on the encoding quality and the multimedia frame type, thereby ensuring that the updated long-term reference frame is more reasonable and accurate, and further improving the encoding quality of the multimedia data.

[0057] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0060] Figure 2 A flowchart of a multimedia data encoding method provided by an embodiment of the present application is schematically shown.

[0061] Figure 3 A flowchart of a multimedia data encoding method provided by an embodiment of the present application is schematically shown.

[0062] Figure 4 A schematic diagram of a multimedia frame encoding process is schematically shown.

[0063] Figure 5 A flowchart of a multimedia data encoding method provided by an embodiment of the present application is schematically shown.

[0064] Figure 6 A schematic diagram of a reference structure provided by an embodiment of the present application is schematically shown.

[0065] Figure 7 A schematic diagram of a reference structure provided by an embodiment of the present application is schematically shown.

[0066] Figure 8 A flowchart of a multimedia data encoding method provided by an embodiment of the present application is schematically shown.

[0067] Fig. 9 The long-term reference frame replacement process using the technical solution of the present application is schematically illustrated.

[0068] Fig.10 The structural block diagram of the multimedia data encoding device provided in an embodiment of the present application is schematically shown.

[0069] Fig.11 The structure block diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0070] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0071] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0072] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0073] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0074] It should be noted that, in the embodiments of the present application, the steps involving the judgment conditions, the case where the judgment result is equal can be classified into any one of the judgment result being greater than or the judgment result being less than according to actual needs. The "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0075] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0076] like Figure 1 As shown, the system architecture 100 includes a plurality of terminal devices, which can communicate with each other through, for example, a network 150. For example, the system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected through the network 150. Figure 1In the embodiment of the present invention, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0077] For example, the first terminal device 110 may encode multimedia data (e.g., a video picture stream collected by the terminal device 110) for transmission to the second terminal device 120 via the network 150, and the encoded multimedia data is transmitted in the form of one or more encoded multimedia code streams. The second terminal device 120 may receive the encoded multimedia data from the network 150, decode the encoded multimedia data to restore the multimedia data, and display the multimedia content according to the restored multimedia data. Exemplarily, taking a low-latency live broadcast scenario as an example, the multimedia data is a video picture stream collected by the first terminal device 110, referred to as video data. The first terminal device 110 encodes the collected video data according to the multimedia data encoding method provided in any embodiment of the present application, generates encoded video data, and transmits the encoded video data to the second terminal device 120 in the form of a code stream via the network 120. The second terminal device 120 receives the bit stream of the encoded video data through the network 120, and decodes the received encoded video data using a decoding method corresponding to the encoding method to restore the encoded video data into video data, and then the second terminal device 120 plays or displays the corresponding video image based on the video data.

[0078] In one embodiment of the present application, the system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded multimedia data, which may occur during a video conference, for example. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 may encode multimedia data (e.g., a video picture stream collected by the terminal device) to transmit to the other terminal device of the third terminal device 130 and the fourth terminal device 140 through the network 150. Each of the third terminal device 130 and the fourth terminal device 140 may also receive the encoded multimedia data transmitted by the other terminal device of the third terminal device 130 and the fourth terminal device 140, and may decode the encoded multimedia data to restore the multimedia data, and may display the multimedia content on an accessible display device according to the restored multimedia data.

[0079] exist Figure 1In the embodiment of the present invention, the first terminal device 110, the second terminal device 120, the third terminal device 130 and the fourth terminal device 140 may be servers, personal computers and smart phones, but the principles disclosed in the present application may not be limited thereto. The embodiments disclosed in the present application are applicable to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. The network 150 represents any number of networks that transmit encoded multimedia data between the first terminal device 110, the second terminal device 120, the third terminal device 130 and the fourth terminal device 140, including, for example, wired and / or wireless communication networks. The communication network 150 may exchange data in circuit switching and / or packet switching channels. The network may include a telecommunications network, a local area network, a wide area network and / or the Internet. For the purpose of the present application, unless explained below, the architecture and topology of the network 150 may be irrelevant to the operation disclosed in the present application.

[0080] The multimedia data encoding method provided by the present application is described in detail below in conjunction with specific implementation methods.

[0081] Figure 2 The flowchart of the multimedia data encoding method provided by one embodiment of the present application is schematically shown. The method can be implemented by the multimedia data encoding device provided by any embodiment of the present application. The specific implementation process of the method will be described below with the multimedia data encoding device as the execution subject. Figure 2 As shown, the multimedia data encoding method provided in the embodiment of the present application includes steps 210 to 240, which are specifically as follows:

[0082] Step 210: Encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame.

[0083] Specifically, multimedia data is data composed of a series of multimedia frames. For example, video data is data composed of a series of video frames. To encode multimedia data, each multimedia frame is generally encoded in turn according to the time sequence of the multimedia frames in the multimedia data. To encode a multimedia frame, it is necessary to obtain the corresponding reference frame. A long-term reference frame is a multimedia frame that can be used as a reference frame in a coding process. A long-term reference frame is a multimedia frame that retains more information. For example, a long-term reference frame can be a key frame in multimedia data. The encoding device of multimedia data will specify a long-term reference frame before encoding the current multimedia frame to be encoded. During encoding, encoding can be performed based on the difference between the current multimedia frame to be encoded and the long-term reference frame. After the current multimedia frame to be encoded is encoded, it becomes the current encoded multimedia frame.

[0084] In one embodiment of the present application, the reference frames of the multimedia frame during encoding include a long-term reference frame and a short-term reference frame. The short-term reference frame is a multimedia frame that changes with the position of the current multimedia frame to be encoded. The short-term reference frame is also called a common reference frame. Before encoding the multimedia data, it is necessary to set the total number of reference frames and the number of long-term reference frames of the multimedia data during the encoding process. The total number of reference frames is the sum of the number of long-term reference frames and the number of short-term reference frames, which can also be called the maximum number of reference frames. Based on the set total number of reference frames and the number of long-term reference frames, the number of short-term reference frames can be obtained according to the difference between the total number of reference frames and the number of long-term reference frames. When encoding, the long-term reference frame is obtained according to the number of long-term reference frames. The multimedia frame as the long-term reference frame has a specific identifier, and the specific identifier can be identified to determine which multimedia frame is the long-term reference frame. At the same time, the short-term reference frame is obtained according to the number of short-term reference frames. Generally, the short-term reference frame is the encoded multimedia frame closest to the current multimedia frame to be encoded. Finally, the current multimedia frame to be encoded is encoded according to the long-term reference frame and the short-term reference frame. For example, assuming that the total number of reference frames is 3, the number of long-term reference frames is 1, and the number of long-term reference frames is frame 1, if the current multimedia frame to be encoded is frame t, then the corresponding short-term reference frames are frame t-1 and frame t-2. After encoding frame t, continue to encode frame t+1, then the short-term reference frames corresponding to frame t+1 are the encoded frame t and frame t-1, and if the long-term reference frame is not updated at this time, then the long-term reference frame is still frame 1.

[0085] In one embodiment of the present application, a multimedia data encoding device may use a reference frame list to record reference frames in the encoding process, which may be divided into a long-term reference frame list and a short-term reference frame list. The long-term reference frame list records the identifiers of long-term reference frames, and the short-term reference frame list records the identifiers of short-term reference frames. During encoding, the corresponding reference frame may be determined by querying the reference frame list. The reference frame list is updated as the encoding process progresses, and may only record reference frames, removing multimedia frame identifiers that are not reference frames from the list. For example, in the aforementioned example, when encoding the t-th frame, the short-term reference frame list records short-term reference frames including the t-1th frame and the t-2th frame. When encoding the t+1th frame, the short-term reference frame list removes the t-2th frame from the list, and then adds the tth frame to the list, so that the short-term reference frames recorded in the short-term reference frame list are the t-th frame and the t-1th frame.

[0086] Step 220: Calculate the encoding quality of the currently encoded multimedia frame and the encoding quality of the long-term reference frame.

[0087] Specifically, when encoding, the encoding device of multimedia data can divide the multimedia frame into multiple encoding blocks, and then encode each encoding block. The encoding quality of the currently encoded multimedia frame and the encoding quality of the long-term reference frame are calculated in the same way, and the encoding quality of the multimedia frame can be calculated by the encoding quality of the encoding blocks contained in the multimedia frame.

[0088] In one embodiment of the present application, during the encoding process, the encoding device of multimedia data calculates the encoding quality of the multimedia frame and stores or records it after encoding each multimedia frame. When calculating the encoding quality of the currently encoded multimedia frame, the encoding quality of the long-term reference frame has been calculated, and the encoding quality of the long-term reference frame can be obtained from the corresponding storage or recording location without having to calculate it again.

[0089] In one embodiment of the present application, after encoding a coding block, a coding device for multimedia data generates a coding cost value of the coding block, which can be referred to as coding cost. The coding cost generally reflects the difference between the coding block and the reference coding block, and the reference coding block is the coding block in the reference frame. Therefore, the coding cost can also reflect the coding quality of the coding block. Generally, the smaller the coding cost, the higher the coding quality; the larger the coding cost, the lower the coding quality. The coding cost of the currently encoded multimedia frame can be regarded as the sum of the coding costs of each coding block contained in the currently encoded multimedia frame. Then, the smaller the sum of the coding costs of each coding block, the higher the coding quality of the currently encoded multimedia frame; the larger the sum of the coding costs of each coding block, the lower the coding quality of the currently encoded multimedia frame.

[0090] Step 230: Determine the type of the current encoded multimedia frame according to the motion vector corresponding to the encoding block in the current encoded multimedia frame.

[0091] Specifically, there are two types of multimedia frames: dynamic multimedia frames and static multimedia frames. A dynamic multimedia frame means that the content of the multimedia frame is a dynamic content. For example, the multimedia frame corresponding to a moving object is generally a dynamic multimedia frame. A static multimedia frame means that the content of the multimedia frame does not change much and can be regarded as static content. For example, the multimedia frame corresponding to a fixed background information is generally a static multimedia frame. The motion vector of the coding block represents the motion characteristics of the coding block, and is also a reflection of the difference between the coding block and the reference frame coding block. The motion vector corresponding to the coding block can be used to determine whether the content represented by the coding block is motion content, and then the type of the currently encoded multimedia frame can be determined.

[0092] In one embodiment of the present application, when determining the type of the currently encoded multimedia frame, it is possible to determine whether the currently encoded multimedia frame is a dynamic multimedia frame based on the number of encoding blocks with motion characteristics in the currently encoded multimedia frame. If the currently encoded multimedia frame is not a dynamic multimedia frame, it is determined to be a static multimedia frame. For example, if the number of encoding blocks with motion characteristics contained in the currently encoded multimedia frame exceeds a threshold, the currently encoded multimedia frame is considered to be a dynamic multimedia frame. If the number of encoding blocks with motion characteristics contained in the currently encoded multimedia frame does not exceed a threshold, the currently encoded multimedia frame is considered to be a static multimedia frame.

[0093] Step 240: Update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0094] Specifically, different types of multimedia frames contain different information, and generally static multimedia frames contain richer information and are more suitable as reference frames. Therefore, when updating the long-term reference frame, the encoding device of the multimedia data can select the static multimedia frame as the long-term reference frame. At the same time, considering the importance of the reference frame in the encoding process, the encoding device of the multimedia data can select a multimedia frame with high encoding quality as the reference frame. That is, the encoding device of the multimedia data determines whether the current encoded multimedia frame is a static multimedia frame according to the type of the current encoded multimedia frame. If the current encoded multimedia frame is a static multimedia frame, the encoding quality of the current encoded multimedia frame is determined to be greater than the encoding quality of the long-term reference frame. The long-term reference frame is updated, for example, the current encoded multimedia frame can be used as a long-term reference frame. Subsequently, the multimedia frame to be encoded in the multimedia data can be encoded based on the updated long-term reference frame. It can be understood that the embodiments of the present application do not limit the execution order of the steps of determining whether the currently encoded multimedia frame is a static multimedia frame and the steps of determining the size of the encoding quality of the currently encoded multimedia frame compared with the encoding quality of the long-term reference frame. For example, the multimedia frame type can be determined first and then the encoding quality can be compared, or the encoding quality can be compared first and then the multimedia frame type can be determined.

[0095] In the technical solution provided in the embodiment of the present application, after encoding the current multimedia frame to be encoded, the encoding quality of the current encoded multimedia frame and the encoding quality of the long-term reference frame are calculated, and the type of the current encoded multimedia frame is determined according to the motion vector corresponding to the encoding block in the current encoded multimedia frame. Finally, the long-term reference frame is updated according to the encoding quality of the current encoded multimedia frame, the type of the current encoded multimedia frame and the encoding quality of the long-term reference frame, thereby realizing dynamic adjustment of the long-term reference frame during the multimedia data encoding process, avoiding the fixed long-term reference frame from being unable to provide effective reference information for multimedia frames that are far away, and improving the encoding quality of the multimedia data; at the same time, the long-term reference frame is updated based on the encoding quality and the multimedia frame type, thereby ensuring that the updated long-term reference frame is more reasonable and accurate, thereby improving the encoding quality of the multimedia data.

[0096] Figure 3 The flowchart of the multimedia data encoding method provided by one embodiment of the present application is schematically shown. This embodiment is a further refinement of the above embodiment. Figure 3 As shown, the multimedia data encoding method provided in the embodiment of the present application includes steps 310 to 380, which are specifically as follows:

[0097] Step 310: Encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame.

[0098] The process of encoding the current multimedia frame to be encoded in this step is the same as step 210 in the above embodiment, and will not be repeated here.

[0099] Step 320: Calculate the target coding cost value of each coding block included in the currently encoded multimedia frame according to the coding mode corresponding to the currently encoded multimedia frame.

[0100] Specifically, depending on whether the pre-coding process is used in the encoding process, the encoding method is divided into two cases, one is the first encoding method using the pre-coding process, and the other is the second encoding method not using the pre-coding process. The pre-coding process (lookahead) is to reasonably allocate the quantization parameters of the discrete cosine change involved in the encoding process of the multimedia frame. In order to speed up the pre-coding process, the pre-coding process will scale the multimedia data to a certain extent. For example, taking video data as an example, the pre-coding process is to reduce the width and height of the video image by half. Assuming that the original size of the video image is 1920*1080, the image size during the pre-coding process is 960*540, and the pre-coding process (lookahead) is to encode according to the image size of 960*540. The encoding cost obtained by the first encoding method using the pre-coding process is different from the encoding cost obtained by the second encoding method not using the pre-coding process.

[0101] For example, Figure 4 The schematic diagram of the multimedia frame encoding process is schematically shown. Figure 4 As shown, during encoding, the multimedia data encoding device divides a multimedia frame into multiple encoding blocks, which are also called macroblocks. The multimedia data encoding device will encode each encoding block in a certain order, thereby completing the encoding of a multimedia frame. For example, the multimedia data encoding device encodes each encoding block in order from left to right and from top to bottom.

[0102] In one embodiment of the present application, if the current encoded multimedia frame does not use pre-coding processing during the encoding process, the target encoding cost value of the encoding block is calculated based on the original encoding cost value of the encoding block. If the current encoded multimedia frame uses pre-coding processing during the encoding process, the target encoding cost value of the encoding block is calculated based on the original encoding cost value and the pre-coding cost value of the encoding block. Specifically, if the pre-coding processing is not used during encoding, then the encoding block will have an original encoding cost value after the encoding is completed, and then the target encoding cost value is calculated based on the original encoding cost value. If the pre-coding processing is used during encoding, then the encoding block will have two encoding cost values ​​after completion, one is the original encoding cost value, and the other is the pre-coding cost value, and then the target encoding cost value needs to be calculated based on these two encoding cost values.

[0103] In one embodiment of the present application, if pre-coding processing is not used during encoding, then when calculating the target coding cost value, the original coding cost value of the coding block is multiplied by the quantization parameter corresponding to the coding block to obtain the target coding cost value of the coding block. In this process, the quantization parameter is used to further amplify the original coding cost value of the coding block to obtain a larger target coding cost value. Since the coding cost value actually reflects the coding quality, the larger target coding cost value can make the difference between multimedia frames of different coding qualities more obvious when the long-term reference frame is updated according to the coding quality in the future, thereby making it easier to screen out multimedia frames with high coding quality. For example, assuming that the original coding cost value of coding block 1 is 10 and the original coding cost value of coding block 2 is 12, then the difference between the two is 2, which is a small difference and may be considered that the two encodings have the same quality. If the quantization parameter is used for processing, assuming that the quantization parameter corresponding to coding block 1 is 10, then the target coding cost value of coding block 1 is 100; assuming that the quantization parameter corresponding to coding block 2 is 12, then the target coding cost value of coding block 2 is 144, and the difference between the two is 44. Obviously, the difference increases, and it can be clearly identified that the coding quality of coding block 1 is different from that of coding block 2. Of course, the quantization parameter cannot be set too large. The quantization parameter is negatively correlated with the coding quality. If the quantization parameter is too large, the coding quality will be reduced.

[0104] In one embodiment of the present application, if precoding is used during encoding, then when calculating the target coding cost, the candidate coding cost is first calculated based on the original coding cost of the coding block. The candidate coding cost is calculated in the same way as the target coding block when precoding is used, that is, the product of the original coding cost of the coding block and the quantization parameter corresponding to the coding block is the candidate coding cost. Then, the precoding cost of the coding block based on the precoding process is obtained, and the candidate coding cost and the precoding cost are weighted and summed to obtain the target coding cost. When the two coding cost values ​​are weighted and summed, the weight of the candidate coding cost value is greater than the weight of the precoding cost value. For example, the weight of the candidate coding cost value can be between 0.7 and 0.8, and the weight of the precoding cost value can be between 0.2 and 0.3.

[0105] Step 330: Calculate the target coding cost value of the currently encoded multimedia frame according to the statistical value of the target coding cost value of each coding block.

[0106] Specifically, after calculating the target coding cost value of each coding block contained in the current encoded multimedia frame, the target coding cost value of each coding block is statistically calculated to obtain the target coding cost value of the current encoded multimedia frame. For example, the sum of the target coding cost values ​​of each coding block is used as the target coding cost value of the current encoded multimedia frame.

[0107] Step 340: Determine the encoding quality of the currently encoded multimedia frame according to the target encoding cost value of the currently encoded multimedia frame, where the target encoding cost value is negatively correlated with the encoding quality.

[0108] Specifically, a mapping relationship between the target coding cost value and the coding quality can be established in advance, and then the coding quality of the currently coded multimedia frame can be determined by querying the mapping relationship. The coding quality of the currently coded multimedia frame can also be directly represented by the target coding cost value of the currently coded multimedia frame. At this time, it should be noted that the target coding cost value is negatively correlated with the coding quality.

[0109] Step 350: Determine a target coding block in the current coded multimedia frame whose motion vector amplitude is greater than a preset amplitude according to the motion vector corresponding to the coding block in the current coded multimedia frame.

[0110] Specifically, the target coding block refers to a coding block with a larger motion amplitude, and the motion amplitude can be reflected by the motion vector amplitude. Therefore, by calculating the motion vector amplitude of the coding block, based on the size of the motion vector amplitude, it can be determined whether the coding block is the target coding block. When the motion vector amplitude is greater than the preset amplitude, it is considered that the motion amplitude of the coding block is large and it is the target coding block. When the motion vector amplitude is less than the preset amplitude, it is considered that the motion amplitude of the coding block is small and it is not the target coding block. It can be understood that the situation where the motion vector amplitude is equal to the preset amplitude can be classified into any one of the following: the motion vector amplitude is greater than the preset amplitude or the motion vector amplitude is less than the preset amplitude. The specific classification can be set according to actual needs.

[0111] In one embodiment of the present application, when encoding a multimedia frame, each coding block of the multimedia frame can be encoded using two modes: inter-frame prediction and intra-frame prediction. Intra-frame prediction refers to using a coding block in the same multimedia frame to predict the current coding block, and inter-frame prediction refers to using a coding block in other multimedia frames to predict the current coding block. In the encoding process, a prediction mode with a smaller coding cost is usually selected for encoding. It can be seen that intra-frame prediction does not involve encoding between different multimedia frames, while inter-frame prediction can reflect the changes between different multimedia frames. Therefore, a coding block using an intra-frame prediction mode usually does not have a motion vector, while a coding block using an inter-frame prediction mode has a motion vector. Therefore, when determining a target coding block, it is possible to first determine which coding blocks use an inter-frame prediction mode, use the coding blocks using the inter-frame prediction mode as candidate coding blocks, and then calculate the motion vector amplitude of the candidate coding block. When the motion vector amplitude of a candidate coding block is greater than a preset amplitude, the candidate coding block is the target coding block.

[0112] In one embodiment of the present application, the number of motion vectors of a coding block using an inter-frame prediction mode varies according to the size of the predicted pixel range. Generally, when the predicted pixel range is the same as the pixel range of the coding block, the candidate coding block has one motion vector. When the predicted pixel range is smaller than the pixel range of the coding block, the number of motion vectors of the candidate coding block is greater than 1. For example, assuming that the pixel range of a coding block is P16*16 (indicating that it includes 16*16 pixels), if the predicted pixel range is P16*16, the coding block includes 1 motion vector. If the predicted pixel range is P8*8, the coding block includes 4 motion vectors.

[0113] When a candidate coding block has multiple motion vectors, whether the candidate coding block is a target coding block is determined by determining whether the maximum amplitude of the motion vector of the candidate coding block is greater than a preset amplitude. When the maximum amplitude of the motion vector of the candidate coding block is greater than the preset amplitude, the candidate coding block can be considered to be the target coding block; when the maximum amplitude of the motion vector of the candidate coding block is less than the preset amplitude, the candidate coding block is considered not to be the target coding block.

[0114] Step 360: Determine the type of the currently encoded multimedia frame according to the number of target coding blocks included in the currently encoded multimedia frame.

[0115] Specifically, a multimedia frame may include multiple target coding blocks, and the type of the multimedia frame can be reflected by the number of target coding blocks it contains. If the number of target coding blocks is large, it indicates that most of the content of the multimedia frame is moving, so the multimedia frame can be considered to be a dynamic multimedia frame. If the number of target coding blocks is small, it indicates that the multimedia frame has less motion content, so the multimedia frame can be considered to be a static multimedia frame. Therefore, the number of target coding blocks contained in the current encoded multimedia frame can be compared with a preset threshold. If the number of target coding blocks contained in the current encoded multimedia frame is greater than the preset threshold, the current encoded multimedia frame is considered to be a dynamic multimedia frame; if the number of target coding blocks contained in the current encoded multimedia frame is less than the preset threshold, the current encoded multimedia frame is considered to be a static multimedia frame.

[0116] In one embodiment of the present application, the process of determining the type of the currently encoded multimedia frame includes: calculating the ratio of the number of target coding blocks contained in the currently encoded multimedia frame to the total number of coding blocks contained in the currently encoded multimedia frame; when the ratio is less than a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame; when the ratio is greater than the preset threshold, determining that the currently encoded multimedia frame is a dynamic multimedia frame.

[0117] Specifically, the encoding device of multimedia data can determine the type of the currently encoded multimedia frame by the proportion of target coding blocks contained in the currently encoded multimedia frame. The ratio of the number of target coding blocks contained in the currently encoded multimedia frame to the total number of coding blocks contained in the currently encoded multimedia frame is the proportion of target coding blocks contained in the currently encoded multimedia frame. When the ratio is less than a preset threshold, it indicates that the number of target coding blocks is small, and the currently encoded multimedia frame is considered to be a static multimedia frame. When the ratio is greater than the preset threshold, it indicates that the number of target coding blocks is large, and the currently encoded multimedia frame is considered to be a dynamic multimedia frame.

[0118] Step 370: Obtain the encoding quality of the long-term reference frame.

[0119] The method for obtaining the encoding quality of the long-term reference frame in this step is the same as the method for obtaining or calculating the encoding quality of the long-term reference frame in step 220 in the aforementioned embodiment, and will not be repeated here.

[0120] Step 380: When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as an updated long-term reference frame to encode the to-be-encoded multimedia frame in the multimedia data according to the updated long-term reference frame.

[0121] Specifically, when the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, it indicates that a multimedia frame with better encoding quality has appeared, and the long-term reference frame can be updated based on the multimedia frame with better encoding quality. It should be noted that if the encoding quality is represented by a target encoding cost value, then the target encoding cost value of the currently encoded multimedia frame is lower than the target encoding cost value of the long-term reference frame, which indicates that the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame.

[0122] The encoding process of multimedia data includes two parts, one is the encoding process of syntax elements, and the other is the encoding process of multimedia data itself. The syntax elements and the encoded data of multimedia data together constitute the encoded code stream data. Each multimedia frame of multimedia data has a corresponding syntax element. The syntax element of the multimedia frame includes the description information of the multimedia frame. For example, the syntax element may include whether the multimedia frame is set as a long-term reference frame, the identification information of the multimedia frame, etc. It can be seen that whether a multimedia frame can be used as a long-term reference frame can be determined by the content of the syntax element corresponding to the multimedia frame. However, in the encoding process, the encoding process of the syntax element is earlier than the encoding process of the multimedia data itself, that is, the syntax element of a multimedia frame is written into the code stream first, and the encoded data of the multimedia frame itself is written into the code stream later. Then, when the current encoded multimedia frame is obtained, the syntax element of the current encoded multimedia frame has been written into the code stream and cannot be changed, that is, the current encoded multimedia frame can no longer be used as a long-term reference frame. However, in multimedia data, the similarity of the contents contained in two adjacent multimedia frames is usually high, especially for static multimedia frames. Therefore, it can be considered that the next multimedia frame to be encoded that has not yet been encoded has the same encoding quality and type as the currently encoded multimedia frame. Therefore, when updating the long-term reference frame, the next multimedia frame to be encoded that has not yet been encoded can be used as the updated long-term reference frame, and the multimedia frames to be encoded in the multimedia data are subsequently encoded according to the updated long-term reference frame.

[0123] When encoding subsequent multimedia frames to be encoded in multimedia data, since the next multimedia frame to be encoded has been used as an updated long-term reference frame, the multimedia frame will usually not be encoded using itself as a reference frame. Therefore, the long-term reference frame corresponding to the next multimedia frame to be encoded is still the long-term reference frame before the update, and the long-term reference frame corresponding to the next multimedia frame to be encoded is the updated long-term reference frame. Exemplarily, assuming that the currently encoded multimedia frame is the tth frame and the current long-term reference frame is the kth frame, if the tth frame is a static multimedia frame and the encoding quality of the tth frame is higher than the encoding quality of the kth frame, then the t+1th frame is used as the updated long-term reference frame. When encoding the t+1th frame, the kth frame is used as the long-term reference frame for encoding. When encoding the k+2th frame and subsequent frames, the t+1th frame is used as the long-term reference frame for encoding.

[0124] In the technical solution provided in the embodiment of the present application, the encoding quality of the current multimedia frame to be encoded is calculated according to the target encoding cost value of each encoding block contained in the current multimedia frame to be encoded, so that the encoding quality can be quantified and the accuracy of the encoding quality judgment can be improved. The type of the current multimedia frame to be encoded is determined by calculating the number of target encoding blocks in the current multimedia frame to be encoded, and the judgment of the multimedia frame type is realized from the coding block granularity, thereby improving the accuracy of the multimedia frame type judgment. Finally, when the current encoded multimedia frame is a static multimedia frame and the encoding quality of the current encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next multimedia frame to be encoded of the current encoded multimedia frame is used as the updated long-term reference frame, thereby achieving the purpose of updating the long-term reference frame with a high-quality static multimedia frame, and effectively improving the encoding quality of subsequent multimedia frames.

[0125] In one embodiment of the present application, the long-term reference frame can be updated periodically, and the specific process is: when the current number of encoded frames reaches a preset number of frames, the current number of encoded frames is cleared, and the long-term reference frame is updated according to the encoding quality of the current encoded multimedia frame, the type of the current encoded multimedia frame, and the encoding quality of the long-term reference frame; wherein the preset number of frames is an integer and is less than the total number of multimedia frames contained in the multimedia frame group where the current multimedia frame to be encoded is located.

[0126] Multimedia data can be divided into multiple multimedia frame groups, and one multimedia frame group includes multiple multimedia frames. When encoding multimedia data, the multimedia frames of each multimedia frame group are encoded in sequence, taking the multimedia frame group as a unit. Therefore, the periodic update of the long-term reference frame is performed during the encoding process of one multimedia frame group, and the update period of the long-term reference frame cannot exceed the time length of the multimedia frame group. Then the preset number of frames is an integer greater than zero and less than the total number of multimedia frames contained in the multimedia frame group where the current multimedia frame to be encoded is located.

[0127] Since the number of multimedia frames included in the multimedia frame group can be determined, it is possible to determine whether the update cycle of the long-term reference frame has been reached by the size of the current number of encoded frames. The current number of encoded frames refers to the number of multimedia frames that have been encoded at the current moment. When the current number of encoded frames reaches the preset number of frames, it means that the update cycle has been reached. At this time, the long-term reference frame can be updated based on the encoding quality and the multimedia frame type. It should be noted that updating the long-term reference frame does not necessarily generate a new long-term reference frame. When the encoding quality and the multimedia frame type do not meet the requirements, the original long-term reference frame remains unchanged in this update. When the update cycle is reached, the current number of encoded frames is cleared to facilitate counting the next update cycle. Exemplarily, the preset number of frames is 5, the current encoded multimedia frame is the 5th frame and is a static multimedia frame, and the long-term reference frame is the 0th frame. The current number of encoded frames has reached 5 frames, so the encoding quality of the 5th frame and the 0th frame is judged. If the encoding quality of the 5th frame is higher than that of the 0th frame, the 6th frame is used as the new long-term reference frame, and in the next update cycle, the encoding quality of the 11th frame and the 6th frame is judged to update the long-term reference frame. If the encoding quality of the 5th frame is lower than that of the 0th frame, the 0th frame is kept as the long-term reference frame unchanged, and in the next update cycle, the encoding quality of the 10th frame and the 0th frame is judged to update the long-term reference frame.

[0128] The technical solution of the present application achieves the purpose of updating the long-term reference frame every preset number of frames. The periodic update can reduce the computing resources consumed by the long-term reference frame update while ensuring the quality of the long-term reference frame.

[0129] Figure 5 The flowchart of the multimedia data encoding method provided by one embodiment of the present application is schematically shown. This embodiment is a further refinement of the above embodiment. Figure 5 As shown, the multimedia data encoding method provided in the embodiment of the present application includes steps 501 to 511, which are specifically as follows:

[0130] Step 501, encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain the current encoded multimedia frame; wherein, when the multimedia frame to be encoded in the multimedia frame group is encoded for the first time, the first frame in the multimedia frame group is used as the long-term reference frame, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

[0131] The process of encoding the current multimedia frame to be encoded in this step is the same as step 210 in the above embodiment, and will not be repeated here.

[0132] In one embodiment of the present application, multimedia frames in multimedia data are in the form of multimedia frame groups, one multimedia frame group includes multiple multimedia frames, and one multimedia frame group represents a continuous multimedia content. For example, the multimedia data is video data, and a video frame group in the video data is also called a group of pictures (Group of Pictures, GOP), one GOP includes multiple video frames, and one GOP represents a continuous video picture. When encoding, it is divided into multimedia frame groups, and different multimedia frame groups correspond to different long-term reference frames. Therefore, the encoding device of multimedia data can obtain the corresponding long-term reference frame for encoding according to the multimedia frame group where the current multimedia frame to be encoded is located.

[0133] Taking multimedia data as video data as an example, there are three types of video frames: I-frame, P-frame and B-frame. I-frame means key frame. During the encoding process, I-frame only refers to the part that has been encoded. P-frame means difference frame, which indicates the difference between the current frame and the frame that has been encoded. During the encoding process, P-frame can only refer to the frame that has been encoded in the previous time domain sequence. B-frame means bidirectional difference frame, which indicates the difference between the current frame and the previous and next frames. During the encoding process, B-frame can refer to the frame that has been encoded in the previous time domain sequence or the frame that has been encoded in the next time domain sequence, as long as the referenced frame is a frame that has been encoded.

[0134] Key frames are usually the frames with the most complete information and the highest quality among various multimedia frames. Therefore, in the embodiment of the present application, the key frames in each multimedia frame group are first set as the long-term reference frames of the multimedia frame group. When encoding multimedia frames in a multimedia frame group for the first time, the long-term reference frame obtained is the first frame of the multimedia frame group, which is also the key frame. For example, Figure 6 The reference structure shown in the figure can be regarded as a reference structure by selecting the number of reference frames and the distance between the reference frames and the current encoded frame when referring to other frames that have been encoded. Assume that the total number of reference frames is 2 frames, the long-term reference frame is 1 frame, and the key frame is the 0 / 0 frame. After the key frame is fixed as the long-term reference frame, if the current encoding is the 7 / 7 frame, the 7 / 7 frame is encoded with reference to the 0 / 0 frame and the 6 / 6 frame.

[0135] Step 502: Calculate the target coding cost value of each coding block included in the currently encoded multimedia frame according to the coding mode corresponding to the currently encoded multimedia frame.

[0136] Step 503: Calculate the target coding cost value of the currently encoded multimedia frame according to the statistical value of the target coding cost value of each coding block.

[0137] Step 504: Determine the encoding quality of the currently encoded multimedia frame according to the target encoding cost value of the currently encoded multimedia frame, where the target encoding cost value is negatively correlated with the encoding quality.

[0138] Step 505: Determine a target coding block in the current coded multimedia frame whose motion vector amplitude is greater than a preset amplitude according to the motion vector corresponding to the coding block in the current coded multimedia frame.

[0139] Step 506: Determine the type of the currently encoded multimedia frame according to the number of target coding blocks included in the currently encoded multimedia frame.

[0140] Step 507: Obtain the encoding quality of the long-term reference frame.

[0141] Step 502 to step 507 are the same as step 320 to step 370 in the aforementioned embodiment, and will not be described again here.

[0142] Step 508: When the current number of encoded frames reaches a preset number of frames, the current number of encoded frames is cleared.

[0143] Step 508 can refer to the description of periodically updating the long-term reference frame in the above embodiment, which will not be repeated here. If the number of currently encoded frames does not reach the preset number of frames, the encoding will not be continued, and the long-term reference frame will not be updated.

[0144] Step 509: When the long-term reference frame is a key frame, the quality weight of the long-term reference frame is obtained according to the time interval between the current encoded multimedia frame and the long-term reference frame, and the quality weight is negatively correlated with the time interval.

[0145] Specifically, considering that a key frame is usually the frame with the highest encoding quality in a multimedia frame group, in order to avoid the encoding quality of subsequent multimedia frames being lower than that of the key frame and causing the long-term reference frame to be unable to be updated, when the long-term reference frame is a key frame, a quality weight is assigned to the key frame so that the encoding quality of the key frame matches the current moment. Although the key frame is the frame that retains the most complete information in the multimedia frame group, as time moves, the farther the frame is from the key frame, the smaller the correlation between the key frame and the key frame, that is, the key frame is difficult to provide effective reference information for distant multimedia frames. Therefore, the quality weight of the key frame is obtained based on the time interval between the current encoded multimedia frame and the current long-term reference frame (i.e., the key frame), and the quality weight is negatively correlated with the time interval. At the same time, the quality weight is in the range of (0,1). Exemplarily, the multimedia frame group has a total of 24 multimedia frames. If the currently encoded multimedia frame is the 4th frame, the quality weight can be set to 0.8; if the currently encoded multimedia frame is the 8th frame, the quality weight can be set to 0.6; if the currently encoded multimedia frame is the 12th frame, the quality weight can be set to 0.4; if the currently encoded multimedia frame is the 16th frame, the quality weight can be set to 0.2.

[0146] If the long-term reference frame is not a key frame, the process directly goes to step 511 .

[0147] Step 510: Generate a target coding quality of the long-term reference frame according to the quality weight and the coding quality of the long-term reference frame, wherein the target coding quality of the long-term reference frame is less than the coding quality of the long-term reference frame.

[0148] Specifically, the quality weight is multiplied by the encoding quality of the long-term reference frame (i.e., the key frame) at this time to obtain the target encoding quality of the long-term reference frame at this time. Since the quality weight is less than 1, the target encoding quality of the long-term reference frame is less than the encoding quality of the long-term reference frame. Assuming that the encoding quality of the key frame is 100 and the quality weight is 0.8, the target encoding quality of the key frame is 80.

[0149] Step 511: Update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0150] Step 511 is the same as step 380 in the above embodiment and will not be described again. If the long-term reference frame is not a key frame, the target encoding quality of the long-term reference frame is the same as the encoding quality of the long-term reference frame.

[0151] For example, Figure 6For example, the current long-term reference frame is the key frame 0 / 0, and the current encoded multimedia frame is frame 7 / 7. Assuming that frame 7 / 7 is a static multimedia frame, the encoding quality of frame 7 / 7 is 90, and the encoding quality of frame 0 / 0 is 100. If no quality weight is assigned to the key frame, the encoding quality of frame 7 / 7 is less than that of frame 0 / 0, and a new long-term reference frame cannot be generated. If a quality weight of 0.8 is assigned to the key frame, the target encoding quality of frame 0 / 0 is 80. At this time, the encoding quality of frame 7 / 7 is greater than that of frame 0 / 0, so frame 8 / 8 can be used as the updated long-term reference frame. When encoding frame 9 / 9 later, frame 7 / 7 is used as the long-term reference frame. The updated reference structure is as follows: Figure 7 shown.

[0152] In the technical solution provided in the embodiment of the present application, the key frame of the multimedia frame group is set as the long-term reference frame, and when updating, the key frame is assigned a quality weight that is negatively correlated with time, so as to avoid the situation where the long-term reference frame cannot be updated due to the encoding quality of subsequent multimedia frames being unable to be higher than the encoding quality of the key frame, thereby improving the accuracy of the long-term reference frame update.

[0153] The implementation process of the technical solution of this application will be described below with a specific embodiment.

[0154] Figure 8 A flowchart of a method for encoding multimedia data provided by an embodiment of the present application is schematically shown. This embodiment takes a low-latency live broadcast scenario using the H264 encoding protocol for video compression as an example, and the multimedia data is video data. A video includes multiple GOPs, a GOP includes at least one I frame, and the remaining frames can be P frames or B frames. Exemplarily, if the GOP is 24 and the frame rate of the current video is 24fps, then there will be an I frame every 1 second, and there will be 23 P frames or B frames between two I frames. For low-latency live video, there are only I frames and P frames in a GOP. If the GOP is 24, then a GOP consists of one I frame and twenty-three P frames.

[0155] like Figure 8 As shown, the multimedia data encoding method provided in the embodiment of the present application includes the following steps:

[0156] Step 801, set the first I frame as a long-term reference frame. That is, set the first I frame in the GOP as a long-term reference frame. In the H264 coding standard, the I frame contains a syntactic element "long_term_reference_flag", which is used to indicate whether the long-term reference frame mechanism is used. When long_term_reference_flag is 0, it indicates that the long-term reference frame mechanism is disabled; when long_term_reference_flag is 1, it indicates that the long-term reference frame mechanism is enabled, and the I frame automatically becomes a long-term reference frame. During the encoding process, set long_term_reference_flag to 1 and write it into the encoded bitstream to make the I frame a long-term reference frame.

[0157] Step 802: Encode the first I frame. The encoding of the I frame only refers to the already encoded part of itself.

[0158] Step 803, encode the P frame. The P frame encoding refers to the long-term reference frame and the short-term reference frame. The short-term reference frame is the frame or frames closest to the P frame. When the P frame is encoded for the first time, the long-term reference frame is the I frame. If the maximum number of reference frames is 2, and the t-th frame is currently being encoded, then the reference frame of the t-th frame is not only the t-1 frame, but also the 1st frame in the current GOP (sometimes also recorded as the 0th frame): the I frame. Since the I frame has a higher encoding quality than the P frame, the t-th frame that refers to the I frame and the t-1 frame will usually have a higher encoding quality than the reference to the t-1 and t-2 frames.

[0159] Step 804: Count the coding cost and the number of macroblocks with large motion.

[0160] During the encoding process, different methods are used to calculate the encoding cost depending on whether the pre-encoding process (lookahead) is enabled. During the encoding process, the encoder divides a coded frame into multiple coding blocks, called macroblocks. The encoder encodes each macroblock in order from left to right and from top to bottom. Figure 4As shown. After each macroblock is encoded, the corresponding original encoding cost will be obtained. The original encoding cost is multiplied by the quantization parameter when encoding this macroblock to obtain the target encoding cost of the macroblock. If the pre-coding process (lookahead) is disabled, the sum of the target encoding costs of all macroblocks in a coded frame is counted as the target encoding cost of the coded frame. If the pre-coding process (lookahead) is enabled, in addition to counting the encoding cost of disabling the pre-coding process (lookahead) (as the first encoding cost), the pre-coding cost of the coded frame obtained in the pre-coding process (lookahead) must also be counted. The pre-coding cost of the coded frame obtained by the pre-coding process (lookahead) is recorded as α, and the first encoding cost of the coded frame obtained by disabling the pre-coding process (lookahead) is recorded as β. The target encoding cost γ of a coded frame is calculated by weighting α and β. The size of the target encoding cost of the coded frame can reflect the quality of the coding to a certain extent. Generally speaking, the smaller the coding cost, the better the coding quality. This method of judging the quality of coding is relatively simple, consisting of only a few steps of multiplication and addition. It does not require additional time-consuming processing and will not significantly increase computing power.

[0161] The number of large motion macroblocks is used to determine whether the encoded frame is a static video frame. Generally speaking, the quality of static video frames is higher than that of dynamic video frames, and static video frames have richer information that can be used for reference. During the encoding process, macroblocks can perform inter-frame prediction and intra-frame prediction, and select the prediction method with the lowest encoding cost. For macroblocks that select inter-frame prediction, the macroblock will have one or more motion vectors to point to the area selected as a reference in the reference frame. The amplitude of each motion vector of the macroblock is counted, and the threshold TH_MV is set. If the maximum value of the amplitude of each motion vector of the macroblock exceeds TH_MV, the macroblock is considered to be a large motion macroblock.

[0162] Step 805, determine whether the preset number of P frames have been encoded. That is, the encoding quality is compared every preset number of frames, and the encoded frame with the best quality is selected. It can be seen that the preset number of frames actually represents the interval frame number between two adjacent encoding quality comparison operations, so the preset number of frames can be represented by Gap. The preset number of frames Gap is an integer greater than 0, and the maximum value is less than the number of frames contained in the GOP. If the preset number of frames (Gap) of P frames have been encoded, go to step 806. If the preset number of frames (Gap) of P frames have not been encoded, return to step 802 and continue encoding P frames.

[0163] Step 806: Select high-quality static P frames.

[0164] Static P frames refer to P frames that belong to static video frames. The percentage of large motion macroblocks in the coded frame to all macroblocks is counted, and a threshold TH_P is set. If the percentage of large motion macroblocks is less than TH_P, the coded frame is considered to be a static video frame. At the same time, the coding quality of the P frame is compared with the coding quality of the current long-term reference frame, and the frame with high quality is selected as the updated long-term reference frame. If the current long-term reference frame is an I frame, it is necessary to assign a quality weight to the I frame before comparing it with the coding quality of the P frame.

[0165] In one embodiment of the present application, in order to achieve more accurate selection, high-quality static P frames can be selected through a machine learning model. A machine learning model can be constructed, and multiple statistical characteristics of the encoded frame are used as input parameters of the machine learning model to select high-quality static P frames through the machine learning model.

[0166] Step 807, make the next frame after the high-quality static P frame as the long-term reference frame, and delete the previous long-term reference frame. The high-quality static P frame can be selected through the above steps. Due to the limitation that the writing of the syntax elements is earlier than the encoding, it is impossible to make the high-quality static P frame as the long-term reference frame after the encoding is completed. However, the quality and content of two adjacent frames are very similar, especially for static scenes. Therefore, the next frame of the high-quality static P frame that has not yet started encoding can be used as a long-term reference frame. Mark the syntax element adaptive_ref_pic_marking_mode_flag contained in the next frame of the high-quality static P frame (also a P frame) as 1, that is, allow it to be used as a long-term reference frame. Then, in the bitstream, the syntax element mmco is used to set the P frame as a long-term reference frame, and the I frame or P frame that was initially used as a long-term reference frame is removed from the reference frame queue. Exemplarily, Fig. 9 The long-term reference frame replacement process using the technical solution of the present application is schematically shown, where the long-term reference frame is replaced by the I frame with the t+1th frame.

[0167] The technical solution of the present application is aimed at the characteristics of low-delay live video, that is, the characteristics of mostly static scenes and short GOP length. On the one hand, it actively selects high-quality I frames as reference frames to improve the encoding quality of subsequent video frames. On the other hand, by analyzing the characteristics of the already encoded P frames, that is, the quality of the P frames and the video content of the P frames, high-quality static P frames are selected as reference frames, and such P frames are used to replace the I frames as reference frames, providing higher-quality references to subsequent P frames, thereby improving the encoding quality of subsequent video frames. Based on the above two aspects, the H264 encoder will be able to ensure that the reference frames always have high quality, thereby improving the PSNR (peak signal to noise ratio), SSIM (structural similarity index measurement) and VMAF (Video Multimethod Assessment Fusion) of the encoded video sequence BD-rate, BD-rate is an indicator for evaluating the performance of a video encoding algorithm, and a negative BD-rate indicates that the encoding performance of the optimized algorithm has been improved.

[0168] It has been verified through experiments that in a low-latency scenario, using the technical solution of this application, the average BD-rate benefits of the PSNR, SSIM and VMAF of the test sequence after H264 encoding are -0.35%, -0.4% and -0.48% respectively. It can be seen that the technical solution of this application can effectively improve the encoding quality of video data without increasing the delay or bit rate. At the same time, the calculation process steps involved in this application are simple and basically do not increase the computing power consumption.

[0169] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0170] The following introduces an apparatus embodiment of the present application, which can be used to execute the multimedia data encoding method in the above-mentioned embodiment of the present application. Fig.10 The structure block diagram of the multimedia data encoding device provided in the embodiment of the present application is schematically shown. Fig.10 As shown, the multimedia data encoding device provided in the embodiment of the present application includes:

[0171] The encoding module 1010 is used to encode the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame;

[0172] A coding quality calculation module 1020, configured to calculate the coding quality of the currently coded multimedia frame and the coding quality of the long-term reference frame;

[0173] A multimedia frame type calculation module 1030, configured to determine the type of the currently encoded multimedia frame according to a motion vector corresponding to a coding block in the currently encoded multimedia frame;

[0174] The reference frame updating module 1040 is used to update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

[0175] In one embodiment of the present application, the encoding quality calculation module 1020 includes:

[0176] A coding cost calculation unit, configured to calculate a target coding cost value of each coding block included in the currently encoded multimedia frame according to a coding mode corresponding to the currently encoded multimedia frame;

[0177] A statistical unit, configured to calculate a target coding cost value of the currently encoded multimedia frame according to a statistical value of a target coding cost value of each coding block;

[0178] The quality calculation unit is used to determine the encoding quality of the currently encoded multimedia frame according to the target encoding cost value of the currently encoded multimedia frame, and the target encoding cost value is negatively correlated with the encoding quality.

[0179] In one embodiment of the present application, the encoding cost calculation unit includes:

[0180] A first subunit is configured to calculate a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block if the currently encoded multimedia frame does not adopt pre-encoding processing during the encoding process;

[0181] The second subunit is configured to calculate a target encoding cost value of the encoding block according to an original encoding cost value and a pre-encoding cost value of the encoding block if the pre-encoding process is adopted in the encoding process of the currently encoded multimedia frame.

[0182] In one embodiment of the present application, the first subunit is specifically used for:

[0183] The original coding cost value of the coding block is multiplied by a quantization parameter corresponding to the coding block to obtain a target coding cost value of the coding block, wherein the quantization parameter is used to amplify the original coding cost value of the coding block.

[0184] In one embodiment of the present application, the second subunit is specifically used for:

[0185] Multiplying the original coding cost value of the coding block by the quantization parameter corresponding to the coding block to obtain a candidate coding cost value of the coding block;

[0186] A weighted sum is performed on the candidate coding cost values ​​and the pre-coding cost value of the coding block to obtain a target coding cost value of the coding block, wherein the weight of the candidate coding cost value is greater than the weight of the pre-coding cost value.

[0187] In one embodiment of the present application, the multimedia frame type calculation module 1030 includes:

[0188] a target coding block determining unit, configured to determine a target coding block having a motion vector amplitude greater than a preset amplitude in the current encoded multimedia frame according to a motion vector corresponding to the coding block in the current encoded multimedia frame;

[0189] The type determination unit is used to determine the type of the currently encoded multimedia frame according to the number of target coding blocks contained in the currently encoded multimedia frame.

[0190] In one embodiment of the present application, the target coding block determination unit is specifically used to:

[0191] Extracting a candidate coding block to be encoded based on the inter-frame prediction mode from a plurality of coding blocks included in the currently encoded multimedia frame;

[0192] Calculating the magnitude of the motion vector corresponding to the candidate coding block;

[0193] When the magnitude of the motion vector corresponding to the candidate coding block is greater than a preset magnitude, it is determined that the candidate coding block belongs to the target coding block.

[0194] In one embodiment of the present application, the type determination unit is specifically used to:

[0195] Calculating a ratio of the number of target coding blocks included in the current encoded multimedia frame to the total number of coding blocks included in the current encoded multimedia frame;

[0196] When the ratio is less than a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame;

[0197] When the ratio is greater than a preset threshold, it is determined that the currently encoded multimedia frame is a dynamic multimedia frame.

[0198] In one embodiment of the present application, the reference frame updating module 1040 is specifically used for:

[0199] When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as an updated long-term reference frame.

[0200] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; the reference frame updating module 1040 is specifically used for:

[0201] When the number of currently encoded frames reaches a preset number of frames, the currently encoded number of frames is cleared, and the long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame;

[0202] The preset frame number is an integer and is smaller than the total number of multimedia frames included in the multimedia frame group where the current multimedia frame to be encoded is located.

[0203] In one embodiment of the present application, the multimedia data includes a plurality of multimedia frame groups; the apparatus further includes:

[0204] The key frame setting module is used to use the first frame in the multimedia frame group as a long-term reference frame when encoding the multimedia frame to be encoded in the multimedia frame group for the first time, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

[0205] In one embodiment of the present application, the reference frame updating module 1040 is specifically used for:

[0206] If the long-term reference frame is a key frame, obtaining a quality weight of the long-term reference frame according to a time interval between the current encoded multimedia frame and the long-term reference frame, wherein the quality weight is negatively correlated with the time interval;

[0207] generating a target encoding quality of the long-term reference frame according to a quality weight and an encoding quality of the long-term reference frame, wherein the target encoding quality of the long-term reference frame is less than an encoding quality of the long-term reference frame;

[0208] The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame.

[0209] In one embodiment of the present application, the encoding module 1010 is specifically used for:

[0210] The number of short-term reference frames is obtained according to the difference between the total number of reference frames corresponding to the multimedia data and the number of long-term reference frames;

[0211] Using a plurality of encoded multimedia frames corresponding to the number of short-term reference frames and located before the current multimedia frame to be encoded as short-term reference frames;

[0212] The current multimedia frame to be encoded is encoded according to the short-term reference frame and the long-term reference frame.

[0213] The specific details of the multimedia data encoding device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0214] Fig.11 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0215] It should be noted that Fig.11 The computer system 1100 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0216] like Fig.11 As shown, the computer system 1100 includes a central processing unit 1101 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1102 (ROM) or the program loaded from the storage part 1108 to the random access memory 1103 (RAM). Various programs and data required for system operation are also stored in the random access memory 1103. The central processing unit 1101, the read-only memory 1102 and the random access memory 1103 are connected to each other through a bus 1104. An input / output interface 1105 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1104.

[0217] The following components are connected to the input / output interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read therefrom is installed into the storage section 1108 as needed.

[0218] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processor 1101, various functions defined in the system of the present application are executed.

[0219] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0220] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0221] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0222] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0223] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0224] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for encoding multimedia data, characterized in that: include: Encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data to obtain a current encoded multimedia frame; Calculating the encoding quality of the currently encoded multimedia frame and the encoding quality of the long-term reference frame; Determining the type of the current encoded multimedia frame according to the motion vector corresponding to the encoding block in the current encoded multimedia frame; The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

2. The method for encoding multimedia data according to claim 1, characterized in that: Calculating the encoding quality of the currently encoded multimedia frame, comprising: Calculating, according to the encoding mode corresponding to the currently encoded multimedia frame, a target encoding cost value of each encoding block included in the currently encoded multimedia frame; Calculating a target encoding cost value of the currently encoded multimedia frame according to the statistical value of the target encoding cost value of each encoding block; The encoding quality of the currently encoded multimedia frame is determined according to a target encoding cost value of the currently encoded multimedia frame, and the target encoding cost value is negatively correlated with the encoding quality.

3. The method for encoding multimedia data according to claim 2, characterized in that: Calculating, according to the encoding mode corresponding to the currently encoded multimedia frame, a target encoding cost value of each encoding block included in the encoded frame, including: If the currently encoded multimedia frame is not pre-encoded during the encoding process, calculating a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block; If the currently encoded multimedia frame adopts pre-encoding processing during the encoding process, the target encoding cost value of the encoding block is calculated according to the original encoding cost value and the pre-encoding cost value of the encoding block.

4. The method for encoding multimedia data according to claim 3, characterized in that: Calculating a target encoding cost value of the encoding block according to an original encoding cost value of the encoding block includes: The original coding cost value of the coding block is multiplied by a quantization parameter corresponding to the coding block to obtain a target coding cost value of the coding block, wherein the quantization parameter is used to amplify the original coding cost value of the coding block.

5. The method for encoding multimedia data according to claim 3, characterized in that: Calculating a target coding cost value of the coding block according to an original coding cost value and a pre-coding cost value of the coding block includes: Multiplying the original coding cost value of the coding block by the quantization parameter corresponding to the coding block to obtain a candidate coding cost value of the coding block; A weighted sum is performed on the candidate coding cost values ​​and the pre-coding cost value of the coding block to obtain a target coding cost value of the coding block, wherein the weight of the candidate coding cost value is greater than the weight of the pre-coding cost value.

6. The method for encoding multimedia data according to claim 1, characterized in that: Determining the type of the current encoded multimedia frame according to the motion vector corresponding to the encoding block in the current encoded multimedia frame includes: Determining a target coding block in the current coded multimedia frame whose motion vector amplitude is greater than a preset amplitude according to a motion vector corresponding to the coding block in the current coded multimedia frame; The type of the currently encoded multimedia frame is determined according to the number of target encoding blocks contained in the currently encoded multimedia frame.

7. The method for encoding multimedia data according to claim 6, characterized in that: Determining a target coding block having a motion vector amplitude greater than a preset amplitude in the current coded multimedia frame according to a motion vector corresponding to the coding block in the current coded multimedia frame, comprising: Extracting a candidate coding block to be encoded based on the inter-frame prediction mode from a plurality of coding blocks included in the currently encoded multimedia frame; Calculating the magnitude of the motion vector corresponding to the candidate coding block; When the magnitude of the motion vector corresponding to the candidate coding block is greater than a preset magnitude, it is determined that the candidate coding block belongs to the target coding block.

8. The method for encoding multimedia data according to claim 6, characterized in that: Determining the type of the currently encoded multimedia frame according to the number of target coding blocks included in the currently encoded multimedia frame includes: Calculating a ratio of the number of target coding blocks included in the current encoded multimedia frame to the total number of coding blocks included in the current encoded multimedia frame; When the ratio is less than a preset threshold, determining that the currently encoded multimedia frame is a static multimedia frame; When the ratio is greater than a preset threshold, it is determined that the currently encoded multimedia frame is a dynamic multimedia frame.

9. The method for encoding multimedia data according to claim 1, characterized in that: Updating the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, comprising: When the currently encoded multimedia frame is a static multimedia frame and the encoding quality of the currently encoded multimedia frame is higher than the encoding quality of the long-term reference frame, the next to-be-encoded multimedia frame of the currently encoded multimedia frame is used as an updated long-term reference frame.

10. The method for encoding multimedia data according to claim 1, characterized in that: The multimedia data includes a plurality of multimedia frame groups; updating the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, including: When the number of currently encoded frames reaches a preset number of frames, the currently encoded number of frames is cleared, and the long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame; The preset frame number is an integer and is smaller than the total number of multimedia frames included in the multimedia frame group where the current multimedia frame to be encoded is located.

11. The method for encoding multimedia data according to claim 1, characterized in that: The multimedia data includes a plurality of multimedia frame groups; Before encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, the method further includes: When encoding the to-be-encoded multimedia frame in the multimedia frame group for the first time, the first frame in the multimedia frame group is used as a long-term reference frame, wherein the first frame in the multimedia frame group is a key frame in the multimedia frame group.

12. The method for encoding multimedia data according to claim 1, characterized in that: Updating the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, comprising: If the long-term reference frame is a key frame, obtaining a quality weight of the long-term reference frame according to a time interval between the current encoded multimedia frame and the long-term reference frame, wherein the quality weight is negatively correlated with the time interval; generating a target encoding quality of the long-term reference frame according to a quality weight and an encoding quality of the long-term reference frame, wherein the target encoding quality of the long-term reference frame is less than an encoding quality of the long-term reference frame; The long-term reference frame is updated according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the target encoding quality of the long-term reference frame.

13. The method for encoding multimedia data according to any one of claims 1 to 12, characterized in that: Encoding the current multimedia frame to be encoded according to the long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, comprising: The number of short-term reference frames is obtained according to the difference between the total number of reference frames corresponding to the multimedia data and the number of long-term reference frames; Using a plurality of encoded multimedia frames corresponding to the number of short-term reference frames and located before the current multimedia frame to be encoded as short-term reference frames; The current multimedia frame to be encoded is encoded according to the short-term reference frame and the long-term reference frame.

14. A multimedia data encoding device, characterized in that: include: An encoding module, configured to encode the current multimedia frame to be encoded according to a long-term reference frame corresponding to the current multimedia frame to be encoded in the multimedia data, to obtain a current encoded multimedia frame; A coding quality calculation module, used for calculating the coding quality of the currently coded multimedia frame and the coding quality of the long-term reference frame; A multimedia frame type calculation module, configured to determine the type of the currently encoded multimedia frame according to a motion vector corresponding to a coding block in the currently encoded multimedia frame; A reference frame updating module is used to update the long-term reference frame according to the encoding quality of the currently encoded multimedia frame, the type of the currently encoded multimedia frame, and the encoding quality of the long-term reference frame, so as to encode the multimedia frame to be encoded in the multimedia data according to the updated long-term reference frame.

15. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for encoding multimedia data described in any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor executes the executable instructions so that the electronic device executes the multimedia data encoding method described in any one of claims 1 to 13.

17. A computer program product, characterized in that The computer program product includes computer instructions stored in a computer-readable storage medium; The processor of the computer device reads and executes the computer instructions from the computer-readable storage medium, so that the computer device executes the multimedia data encoding method according to any one of claims 1 to 13.

Images