Multimedia data processing method, apparatus, device, computer-readable storage medium, and computer program

Abstract

To provide a multimedia data processing method, apparatus, device, computer-readable storage medium, and computer program.SOLUTION: A method includes the steps of: acquiring attribute information of a to-be-decoded data block in a multimedia data frame; acquiring a first target reference data block matching the attribute information of the to-be-decoded data block, where the first target reference data block is selected from a first reference data block set, a second reference data block set, or a third reference data block set integrating the first reference data block set and the second reference data block set; and performing predictive coding on the to-be-decoded data block according to the first target reference data block. The method can effectively improve coding performance of media data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application claims priority from a Chinese patent application bearing application number 202111221890.5, filed with the China Patent Office on October 20, 2021, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of Internet, and in particular, but not exclusively, to a multimedia data processing method, apparatus, device, computer-readable storage medium and computer program. [Background technology]

[0003] In the process of multimedia data processing, predictive coding is usually performed on multimedia data frames using an intra-frame predictive coding method, which can improve the coding performance of the multimedia data frames. Therefore, the intra-frame predictive coding method has been widely applied in the process of multimedia data processing. However, in practice, it has been found that the coding performance of the multimedia data frames is currently low due to the limited reference data blocks for predictive coding in the intra-frame predictive coding method. Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the embodiments of the present application is to provide at least a multimedia data processing method, apparatus, device, computer-readable storage medium, and computer program that can improve the coding performance of multimedia data frames. [Means for solving the problem]

[0005] According to an embodiment of the present application, there is provided a multimedia data processing method, which includes: Obtaining attribute information of a data block to be coded in a multimedia data frame; Obtaining a first target reference data block that matches the attribute information of the data block to be coded, wherein the first target reference data block is selected from a first reference data block set, a second reference data block set, or a third reference data block set obtained by combining the first reference data block set and the second reference data block set, wherein the first reference data block set includes coded data blocks that do not have an adjacent relationship with the data block to be coded, and the second reference data block set includes coded data blocks that have an adjacent relationship with the data block to be coded; and The method includes performing predictive coding on the data block to be coded based on the first target reference data block.

[0006] According to an embodiment of the present application, there is provided a multimedia data processing method, which includes: Obtaining attribute information of a data block waiting to be decoded in a multimedia data frame; Obtaining a second target reference data block that matches the attribute information of the data block to be decoded, wherein the second target reference data block belongs to a first reference data block set, a second reference data block set, or a third reference data block set that is a combination of the first reference data block set and the second reference data block set, wherein the first reference data block set includes decoded data blocks that do not have an adjacent relationship with the data block to be decoded, and the second reference data block set includes decoded data blocks that have an adjacent relationship with the data block to be decoded; and The decoding step includes decoding the data block to be decoded based on the second target reference data block.

[0007] According to an embodiment of the present application, there is provided a multimedia data processing device, which comprises: a first obtaining module for obtaining attribute information of a data block to be coded in a multimedia data frame; a first matching module for obtaining a first target reference data block matching attribute information of the data block to be coded, the first target reference data block being selected from a first reference data block set, a second reference data block set, or a third reference data block set obtained by combining the first reference data block set and the second reference data block set, the first reference data block set including coded data blocks that do not have an adjacent relationship with the data block to be coded, and the second reference data block set including coded data blocks that have an adjacent relationship with the data block to be coded; and A coding module for performing predictive coding on the data block to be coded based on the first target reference data block is included.

[0008] According to an embodiment of the present application, there is provided a multimedia data processing device, which comprises: a second obtaining module for obtaining attribute information of a data block to be decoded in a multimedia data frame; a second matching module for obtaining a second target reference data block matching attribute information of the data block to be decoded, the second target reference data block belonging to a first reference data block set, a second reference data block set, or a third reference data block set obtained by combining the first reference data block set and the second reference data block set, the first reference data block set including decoded data blocks that do not have an adjacent relationship with the data block to be decoded, and the second reference data block set including decoded data blocks that have an adjacent relationship with the data block to be decoded; and A decoding module is included for decoding the data block to be decoded based on the second target reference data block.

[0009] According to an embodiment of the present application, a computer device is provided, which includes a processor and a memory, in which program instructions are stored in the memory, and the processor is configured to execute steps in the above-mentioned multimedia data processing method by calling the program instructions.

[0010] According to an embodiment of the present application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, are configured to perform steps in the multimedia data processing method described above.

[0011] According to an embodiment of the present application, a computer program product is provided, which includes a computer program or computer instructions, which, when executed by a processor, are configured to implement the steps of the multimedia data processing method described above. [Effects of the Invention]

[0012] The embodiments of the present application have at least the following advantageous effects: a first target reference data block matching the attribute information of a data block to be coded is obtained, and the first target reference data block is selected from the first set of reference data blocks, the second set of reference data blocks, or a third set of reference data blocks formed by combining the first set of reference data blocks and the second set of reference data blocks. In other words, the selectable reference data block sets include the first set of reference data blocks, the second set of reference data blocks, and the third set of reference data blocks. In this way, the selection range of reference data blocks is expanded, and adaptive selection of the first target reference data block based on the attribute information of the data block to be coded can be performed to perform predictive coding on the data block to be coded based on the first target reference data block. This improves the flexibility and accuracy of obtaining the first target reference data block and improves the coding performance of multimedia data frames. Furthermore, performing predictive coding on the data block to be coded based on the first target reference data block is advantageous to eliminating redundancy in multimedia data frames, improving the transmission efficiency of multimedia data frames, and saving transmission resources. [Brief explanation of the drawings]

[0013] In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. It is obvious that the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without any creative work. [Figure 1] 1 is a flowchart of video processing provided by an embodiment of the present application; [Figure 2] FIG. 2 is a flow diagram of a multimedia data processing method provided by an embodiment of the present application; [Figure 3] FIG. 2 illustrates a coding data block provided by an embodiment of the present application. [Figure 4] FIG. 2 illustrates a scenario of non-intraframe coding of multimedia data frames provided by an embodiment of the present application. [Figure 5] FIG. 2 illustrates a scenario of intra-frame block duplication (copying) of a multimedia data frame provided by an embodiment of the present application. [Figure 6] FIG. 2 illustrates a scenario of memory reuse of reference data blocks of a multimedia data frame provided by an embodiment of the present application. [Figure 7] FIG. 2 is a flow diagram of a multimedia data processing method provided by an embodiment of the present application. [Figure 8] FIG. 2 is a diagram illustrating the positional relationship between a first reference data set, a second reference data block set, and data blocks to be coded provided by an embodiment of the present application. [Figure 9] FIG. 10 illustrates another positional relationship between the first reference data set, the second reference data block set, and the data blocks to be coded provided by an embodiment of the present application. [Figure 10] FIG. 10 is a flow diagram of another multimedia data processing method provided by an embodiment of the present application. [Figure 11] 1 is a diagram showing the configuration of a multimedia data processing device provided by an embodiment of the present application; [Figure 12] FIG. 10 is a diagram showing the configuration of another multimedia data processing device provided by an embodiment of the present application. [Figure 13] FIG. 1 is a diagram showing the configuration of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0014] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Of course, the described embodiments are only some of the embodiments of the present application, not all of the embodiments. In addition, all other embodiments that a person skilled in the art can derive from the embodiments of the present application without any creative work also fall within the technical scope of the present application.

[0015] The embodiments of the present application relate to multimedia data processing technology. Multimedia data (also referred to as media data) refers to composite data formed by media data, such as text, graphics, images, audio, animation, and video, whose contents are associated with each other. The multimedia data referred to in the embodiments of the present application mainly includes image data consisting of images or video data consisting of images, audio, etc. The embodiments of the present application mainly use video data as an example to describe the data processing process of multimedia data in detail. When the multimedia data is image data, the embodiments of the present application can be referred to. The processing process for multimedia data in the embodiments of the present application mainly includes media data collection, media data coding, media data file encapsulation, media data file transmission, media data decoding, and final data display. When the multimedia data is video data, the complete processing process for video data can include, as shown in FIG. 1, S1: video collection; S2: video coding; S3: video file encapsulation; S4: video file transmission; S5: video file decapsulation; S6: video decoding; and S7: video display.

[0016] Video acquisition is used to convert analog video into digital video and then store it according to a digital video file format. In other words, video acquisition can convert a video signal into binary digital information, of which the binary information converted from the video signal is a binary data stream, also known as the code stream or bitstream of the video signal. Video coding refers to converting a file in an original video format into a file in another video format using compression technology. The generation of video media content referred to in the embodiments of this application includes capturing real scenes using a camera and generating screen content using a computer. From the perspective of video signal acquisition methods, video signals can be divided into two types: signals captured by a camera and signals generated by a computer. Due to different statistical characteristics, the compression coding methods corresponding to different video signals may also be different. At present, mainstream video coding technologies adopt a hybrid coding framework, as shown in Figure 2, to perform the following series of operations and processes on the input original video signal:

[0017] Block partition structure: A multimedia data frame (e.g., a video frame in video data) is input. Based on the size of the video frame, the video frame is divided into multiple non-overlapping processing units, and similar compression operations are performed on each processing unit. In one embodiment, the processing unit is called a coding tree unit (CTU) or a largest coding unit (LCU). The CTU can be further divided to obtain one or more basic coding units, which are called coding units (CUs), and each CU is the most basic element in a coding process. In another embodiment, the processing unit is also called a coding tile, which is a rectangular region of a multimedia data frame that can be independently decoded and coded. The coding tile can then be further divided to obtain one or more largest coding blocks (SBs, superblocks), which can then be further divided to obtain one or more data blocks (Bs, blocks). Each data block is the most basic element in a coding process. The following describes various coding methods that can be adopted for each data block, among which, the relationship between SB and B is as shown in Figure 3.

[0018] Predictive Coding: Includes methods such as intraframe prediction and interframe prediction. A residual video signal is obtained after the original video signal is predicted by a selected coded video signal. The coding side must select the most suitable one from many possible predictive coding modes for the current coding data block (i.e., the data block to be decoded) and notify the decoding side. Among them, intraframe prediction is the most common. Inter-picture prediction means that the predicted signal is from a coded and reconstructed region within the same picture. Inter-picture prediction means that the predicted signal is from another coded picture (also called a reference picture) that is different from the current picture.

[0019] Transform & Quantization: The residual video signal is transformed into a transform domain using transform operations such as the Discrete Fourier Transform (DFT) and Discrete Cosine Transform. The signal in the transform domain is then subjected to a lossy quantization operation, which makes the quantized signal more suitable for compressed representation.

[0020] Since some video coding standards allow multiple transform methods to be selected, the coding side also needs to select one of them for the current coding data block and notify the decoding side. The quantization resolution is usually determined by a quantization parameter (QP), and when the QP value is greater than a QP threshold, it means that coefficients larger than a predetermined numerical range are quantized to the same output, which usually results in distortion greater than a distortion rate threshold and a bit rate smaller than a bit rate threshold. Conversely, when the QP value is equal to or less than the QP threshold, it means that coefficients smaller than a predetermined numerical range are quantized to the same output, which usually results in distortion smaller than a distortion rate threshold and a bit rate larger than a bit rate threshold.

[0021] Entropy coding or statistical coding: The signal in the transform domain after quantization is statistically compressed and coded based on the frequency of each value, and finally a binary (0 or 1) compressed code stream is output. At the same time, other information from the coding, such as the selected mode, motion vector, etc., also needs to be entropy coded to reduce the bit rate.

[0022] Statistical coding is a lossless coding method that can effectively reduce the bit rate required to represent a similar signal. Common statistical coding methods are Variable Length Coding (VLC) or Content-Based Binarized Arithmetic Coding (CABAC). Adaptive Binary Arithmetic Coding.

[0023] Loop Filtering: After a coded image (i.e., a multimedia data frame) undergoes inverse quantization, inverse transform, and predictive compensation, a reconstructed decoded image can be obtained. Compared to the original image, the reconstructed image is affected by quantization, which can cause some information to differ from the original image, resulting in distortion. By performing filtering operations on the reconstructed image, such as deblocking, sample adaptive offset (SAO), and adaptive loop filtering (ALF), the degree of quantization-induced distortion can be effectively reduced. Because the reconstructed image after these filters is used as a reference for subsequent coded images and to predict future signals, the above filtering operations are also referred to as loop filtering and filtering operations within the coding loop.

[0024] Figure 2 shows the basic flow of a video coder. In Figure 2, k objective CUs (S k Here, k is a positive integer greater than or equal to 1 and less than or equal to the number of CUs in the current image being input. k [x, y] represents a pixel point with coordinates [x, y] in the k target CU, where x represents the horizontal coordinate of the pixel point, y represents the vertical coordinate of the pixel point, and S k [x,y] is a predicted signal after undergoing a suitable process such as motion compensation, intra-frame prediction, etc.

[0025]

number

[0026]

number

[0027]

number

[0028] As can be seen from the above coding process, for each data block, after the decoder obtains the compressed code stream, it first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transformation to obtain a residual signal. Furthermore, a prediction signal corresponding to the data block can be obtained based on the known coding mode information, and after adding the two, a coded data block can be obtained. Finally, the coded data block needs to undergo loop filtering to generate a final coded data block.

[0029] Currently, mainstream video coding standards, such as HEVC, VVC, AVS3, AV1, and AV2, all adopt a block-based hybrid coding framework. These video coding standards divide an original multimedia data frame into a series of coding data blocks and, in combination with video coding methods such as prediction, transform, and entropy coding, achieve video data compression. Among them, motion compensation is a commonly used predictive coding method in video coding. Motion compensation derives a predicted value of a current coding data block from a previously coded domain (i.e., a reference data block) based on redundancy in the temporal or spatial domain of video content. Such predictive coding methods include inter-frame prediction, intra-frame block duplication prediction, and intra-frame string duplication prediction, and these predictive coding methods can be used alone or in combination to achieve predictive coding. For coding data blocks using these predictive coding methods, one or more two-dimensional displacement vectors must usually be explicitly or implicitly coded in the codestream to indicate the displacement of the current coding data block (or its co-located blocks) relative to one or more reference data blocks.

[0030] In addition, under different predictive coding modes, the displacement vector may have different names, and in the embodiments of this application, we will unify the description according to the following methods: 1) the displacement vector during inter-frame prediction is called a motion vector (MV, Motion Vector); 2) the displacement vector during intra-frame block duplication (i.e., intra-frame predictive coding) is called a block vector (BV, Block Vector); and 3) the displacement vector during intra-frame string duplication is called a string vector (SV, String Vector). Below, we will introduce related technologies in inter-frame prediction and intra-frame block duplication prediction.

[0031] 1.1.Inter-frame prediction As shown in Figure 4, inter-frame prediction utilizes the temporal correlation of multimedia data frames and predicts pixels of the current image using adjacent pixels in the coded image, thereby achieving the purpose of effectively eliminating redundancy in the temporal domain of the multimedia data frame and effectively saving bits for coding residual data. Wherein, P is the current frame, Pr is the reference frame, B is the current data block to be decoded, and Br is the reference data block of B. B' and B have the same coordinate position in the image, with Br's coordinates being (xr,yr) and B's coordinates being (x,y). The displacement between the current coding data block and its reference data block is called the motion vector (MV), which can be expressed using the following formula (1):

[0032] MV=(x r -x,y r -y) (1) Considering that neighboring blocks in the temporal or spatial domain have high correlation, MV prediction techniques can be employed to further reduce the bits required for MV coding.

[0033] 2.1. Intra-frame Block Duplication Prediction Intra-frame block copy (IBC) is an intra-frame coding tool adopted in the HEVC Screen Content Coding (SCC) extension, which significantly improves the coding efficiency of screen content. AVS3, VVC, and AV1 also adopt IBC technology to improve screen content coding performance. IBC exploits the spatial correlation of screen content video and predicts pixels in data blocks to be decoded using pixels in previously coded data blocks in a multimedia data frame, thereby effectively saving bits required for pixel coding. Figure 5 shows a diagram of IBC technology, in which the displacement between a data block to be decoded and its reference data block is called the block vector (BV).

[0034] HEVC In the SCC extension, the IBC mode uses an implementation method similar to interframe prediction. If a multimedia data frame allows the use of the IBC mode, the currently coded but unfiltered picture is used as a special reference frame, and the coded data block derives the predicted value of the data block to be decoded from this reference frame using the interframe prediction-motion compensation method. This implementation method has the following characteristics: 1) the BV only uses integer precision; 2) a global reference data block set (i.e., a global reference range) is used, that is, all coded parts of the current multimedia data frame can be used as reference data blocks for the data block to be decoded, except for some areas that are prohibited from being used as reference data blocks due to parallel processing and chrominance interpolation; 3) in the decoding picture buffer, the currently coded but unfiltered picture can occupy a buffer the size of one picture; and 4) the currently coded but unfiltered picture is located at the end of the reference picture list.

[0035] In hardware implementations, computing cores can read data from on-chip memory or off-chip memory. On-chip memory is usually static and has very high read / write speeds, but due to its high cost, only small on-chip memory is generally used. Off-chip memory is usually quite large, but efficient access requires following certain rules, and there is usually a write-back delay when writing a recent coded data block back to the off-chip picture memory. In hardware implementations of motion compensation, the current coded image is written to off-chip memory for future reference after the loop filtering operation is completed. When motion compensation for a subsequent coded data block needs to reference a coded data block located in off-chip memory, the relevant data of the reference data block needs to be read from the off-chip memory. This process involves consuming memory bandwidth for writing and reading. In software implementations, the IBC scheme in HEVC-SCC has a small impact on software design among related technologies, but hardware implementations have the following two problems: That is, a) the reference image of IBC is a version that has not undergone loop filtering and is different from the final decoded image. Outputting this image off-chip may result in increased memory bandwidth. b) A currently coded or immediately reconstructed adjacent data block may be used as a reference for the next coded data block. This requires completing the process of decoding the reconstructed data block, outputting it to off-chip memory, and rewriting it back to off-chip memory within a limited number of clock cycles. This poses a significant challenge to hardware implementation.

[0036] To address the hardware implementation issues of IBC, subsequent standards, such as VVC, AVS3, and AV1, have adopted several IBC-related optimization schemes. The IBC scheme in the VVC and AVS3 standards uses a local reference data block set, i.e., it uses coded data blocks adjacent to the current coding data block as the reference data block for the current coding data block. In hardware implementation, the above problem can be solved by using reference data blocks in a local reference data block set stored in on-chip memory. Considering the hardware implementation cost and coding performance, IBC only uses reference data blocks in a local reference data block set stored in a memory the size of one CTU and adopts the reference memory reuse policy shown in Figure 6. In addition to the storage of the current 64x64 CU waiting for reconstruction, there are three more 64x64 CUs that can be used to store reconstructed data blocks. Therefore, IBC can only search for reference data blocks in these three 64x64 CUs and the reconstructed part of the current 64x64 CU.

[0037] In summary, the IBC scheme in the AV1 standard uses a global reference data block set, which allows a current coding data block to refer to previously coded data blocks in the current multimedia data frame. However, this scheme has two drawbacks: a) to meet the requirements of the write-back restriction condition, previously coded data blocks adjacent to the current coding data block are not allowed to be used as reference data blocks; and b) if the current multimedia data frame allows the use of IBC, the use of a loop filter is prohibited. However, because screen content sequences often have strong spatial correlation and neighboring data blocks have high reference value, allowing neighboring previously coded data blocks to be referenced is beneficial to improving coding performance. For some sequence content, prohibiting the use of a loop filter may also severely degrade the quality of multimedia data and impact coding performance. Although an IBC scheme based on a local reference data block set does not have the above drawbacks, due to the limited number of available reference data blocks in the local reference data block set, using only the local reference data block set may result in a degradation of the coding performance of the multimedia data frame. According to the characteristics of the sequence content, coding parameters such as global reference data block set, local reference data block set, loop filtering process, etc., have different proportions in impact on coding performance.Therefore, in view of the drawbacks of the global reference data block set-based IBC scheme and the local reference data block set-based IBC scheme, in the embodiments of the present application, selecting an appropriate predictive coding method based on the attribute information of the multimedia data frame is beneficial to improving the coding performance. In other words, in the present application, based on the attribute information of the current coding data block, the reference data block is adaptively selected from the first reference data block set (i.e., the global reference data block set), the second reference data block set (i.e., the local reference data block set), or the third data block set which is a combination of the second reference data block set and the first data block set, thereby expanding the selectable range of the reference data block, and solving the problem of the limited number of reference data blocks. Then, predictive coding is performed on the current coding data block based on the selected reference data block, thereby improving the coding performance of the multimedia data frame.

[0038] This application may be executed by a terminal, a server, or both a terminal and a server. In this application, devices for executing the method may be collectively referred to as computer devices. The server may be an independent physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms. The terminal may be, but is not limited to, a smart vehicle terminal, a smart camera, a smartphone, a tablet computer, a laptop, a desktop computer, a smart speaker, a speaker with a screen, a smart watch, a smart TV, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication. The number of terminals and servers may be one or at least two, but this application is not limited thereto.

[0039] It should be noted that the embodiments of the present application can be applied to various scenarios, such as, but not limited to, cloud technology, artificial intelligence, intelligent transportation, and driver assistance. For example, in an intelligent transportation scenario, the multimedia data frame is a video frame or an image frame captured by an imaging device of a road, and the imaging device may refer to an in-vehicle terminal or a roadside imaging device. Performing predictive coding on the multimedia data frame according to the embodiments of the present application is advantageous to improving the coding performance of the multimedia data frame, and using the coded multimedia data frame can enable real-time observation of road conditions, etc., thereby improving vehicle driving safety.

[0040] 7 is a flow diagram of a multimedia data processing method provided by an embodiment of the present application. As shown in FIG. 7, the method includes at least the following S101-S103.

[0041] S101: Obtain attribute information of a data block to be coded in a multimedia data frame.

[0042] In an embodiment of the present application, a computer device can obtain a multimedia data frame, which can be captured by the computer device or downloaded from a network by the computer device. The multimedia data frame can refer to a video frame to be coded in video data or a frame image to be coded in image data, the video data can refer to live broadcast video data, movie and television video data, daily life video data, road surveillance video data, etc., and the image data can refer to image data of people, building image data, product image data, etc. Attribute information of a data block to be coded in the multimedia data frame can be obtained, which can refer to the currently coded data block mentioned above, and the data block to be coded can refer to a data block to be coded among the uncoded data blocks in the multimedia data frame, i.e., the data block to be coded can refer to the uncoded data block in the multimedia data frame that is closest to a coded data block.

[0043] The attribute information of the data block to be coded includes one or more of the media type of the multimedia data frame to which the data block to be coded belongs, the reference flag of the multimedia data frame, and the clarity of the multimedia data frame to which the data block to be coded belongs. The media type includes intraframe and non-intraframe. The intraframe type may refer to a multimedia data frame with a representative meaning, and the intraframe type includes a full intraframe coded frame, an intraframe, an I-frame, and a key frame. An intraframe type frame is only allowed to use intraframe coding technology and does not need to be coded depending on other frames. For example, the intraframe type may refer to a multimedia data frame that can reflect the main meaning of video data or image data. The non-intraframe type may refer to a multimedia data frame other than the intraframe type in video data or image data, and the non-intraframe type may be called an interframe coded frame, a P-frame, a B-frame, or a non-key frame. Non-intraframe type frames are allowed to use interframe coding and intraframe coding techniques, and when interframe coding is used, coding usually needs to be performed after the reference frame coding is completed. The reference flag of the multimedia data frame is used to reflect the selection range of the reference data block of the data block to be coded.

[0044] S102: Obtain a first target reference data block matching the attribute information of the data block to be coded, wherein the first target reference data block is selected from the first reference data block set, the second reference data block set, or a third reference data block set obtained by combining the first and second reference data block sets, wherein the first reference data block set includes coded data blocks that have no adjacent relationship with the data block to be coded, and the second reference data block set includes coded data blocks that have an adjacent relationship with the data block to be coded. The third data block set includes data blocks in the first data block set and the second data block set, i.e., the third data block set includes coded data blocks that have no adjacent relationship with the data block to be coded and coded data blocks that have an adjacent relationship with the data block to be coded.

[0045] In an embodiment of the present application, in a multimedia data frame, coded data blocks that are not adjacent to the data block to be coded and coded data blocks that are adjacent to the data block to be coded are both used as reference data blocks, and the computer device adaptively selects a first target reference data block from among the coded data blocks that are adjacent to the data block to be coded and the coded data blocks that are not adjacent to the data block to be coded based on the attribute information of the data block to be coded, thereby improving the flexibility and accuracy of obtaining the reference data block.

[0046] In addition, the coded data block in the embodiments of the present application may also be referred to as a coding-side reconstructed data block, and the reconstructed data block refers to the data block after the coded data block has undergone reconstruction processing (e.g., inverse quantization, inverse transform, prediction compensation operations, etc.).

[0047] In addition, the first target reference data block may refer to a coded data block that is adjacent to the data block to be coded, or the first target reference data block may refer to a coded data block that is not adjacent to the data block to be coded, or the first region of the first target reference data block may belong to a coded data block that is adjacent to the data block to be coded, and the second region of the first target reference data block may belong to a coded data block that is not adjacent to the data block to be coded.

[0048] S103: Perform predictive coding on the data block to be coded based on the first target reference data block.

[0049] In an embodiment of the present application, a computer device can determine a block vector between the first target reference data block and the data block to be coded based on the position information of the first target reference data block and the data block to be coded in each multimedia data frame, and then obtain a predicted signal of the data block to be coded by predicting based on the block vector and the first target reference data block, and then obtain a residual signal of the data block to be coded based on the actual signal and the predicted signal of the data block to be coded, that is, the residual signal of the data block to be coded is a predicted coding value of the data block to be coded, and performing predictive coding on the data block to be coded is advantageous to removing redundancy in the multimedia data frame, improving the transmission efficiency of the multimedia data frame, and saving transmission resources.

[0050] Note that, typically, video data includes multiple multimedia data frames, or image data includes multiple multimedia data frames, but the above-mentioned data block to be coded and the first target reference data block both belong to the same multimedia data frame.

[0051] In an embodiment of the present application, a computer device obtains a first target reference data block that matches the attribute information of a data block to be coded, and the first target reference data block is selected from a first set of reference data blocks, a second set of reference data blocks, or a third set of reference data blocks that is a combination of the first set of reference data blocks and the second set of reference data blocks. In other words, the selectable reference data block sets include the first set of reference data blocks, the second set of reference data blocks, and the third set of reference data blocks. In this way, the selection range of reference data blocks is expanded, and the first target reference data block is adaptively selected based on the attribute information of the data block to be coded, and predictive coding is performed on the data block to be coded based on the first target reference data block, thereby improving the flexibility and accuracy of obtaining the first target reference data block and improving the coding performance of the multimedia data frame. In addition, predictive coding of the data block to be coded based on the first target reference data block is also advantageous to removing redundancy in the multimedia data frame, improving the transmission efficiency of the multimedia data frame, and saving transmission resources.

[0052] In some embodiments, the computer device can obtain a first target reference data block that matches the attribute information of the data block to be coded by one or a combination of the following three matching methods:

[0053] Matching method 1: if the media type of the multimedia data frame is an intra-frame type, obtain a first target reference data block that matches the attribute information of the data block to be coded from the first reference data block set or the third reference data block set, and if the media type of the multimedia data frame is a non-intra-frame type, obtain a first target reference data block that matches the attribute information of the data block to be coded from the second reference data block set. In other words, the computer device adaptively selects a reference data block set according to the attribute information of the data block to be coded, and then determines a first target reference data block from the selected reference data set, thereby improving the flexibility and accuracy of obtaining reference data blocks.

[0054] Typically, the number of data blocks in the first reference data block set is greater than the number of data blocks in the second reference data block set, and the first reference data block set and the second reference data block set are stored in a first storage space and a second storage space, respectively, which may be in the same computer device or in different computer devices. The first storage space and the second storage space may refer to a magnetic disk, a storage medium array, etc. When the first storage space and the second storage space are in the same computer device, the first storage space may refer to the off-chip memory of the computer device and the second storage space may refer to the on-chip memory of the computer device, or the first storage space may refer to the on-chip memory of the computer device and the second storage space may refer to the off-chip memory of the computer device.

[0055] In matching method 1, if the media type of the multimedia data frame is an intra-frame type, it means that the importance of the multimedia data frame is high, and the first and second reference data block sets (i.e., third reference data block sets) are searched for a data block (i.e., an optimal data block) that has the highest correlation with the data block to be coded, and this data block can be determined as a first target reference data block that matches the attribute information of the data block to be coded, and this first target reference data block is determined by the first and second reference data block sets. That is, any one data block in the first and second reference data block sets can be used as a reference data block, thereby expanding the selection range of reference data blocks and improving the coding performance of the multimedia data frame. If the media type of the multimedia data frame is a non-intra-frame type, it means that the importance of the multimedia data frame is relatively low, and in this case, the second reference data block set is searched for a data block that has the highest correlation with the data block to be coded, and this data block can be determined as a first target reference data block that matches the attribute information of the data block to be coded. Adaptively selecting a reference data block that matches a data block waiting to be loaded based on the media type of a multimedia data frame is advantageous in improving the accuracy and flexibility of obtaining the reference data block.

[0056] where the first target reference data block is a data block determined based on the first reference data block set and the second reference data block set, and has one of the following meanings: first, the first target reference data block is selected from the first reference data block set; second, the first target reference data block is selected from the second reference data block set; and third, a first region of the first target reference data block may belong to the first reference data block set and a second region of the first target reference data block may belong to the second reference data block set, that is, a part of the first target reference data block belongs to the second reference data block set and a part belongs to the first data block set. Wherein, the correlation between the data block to be coded and the coded data block may be determined based on the similarity between the pixels in the data block to be coded and the pixels in the coded data block.

[0057] In matching method 1, if the media type of the multimedia data frame is an intra-frame type, the first set of reference data blocks may be searched for a data block having the highest correlation with the data block to be coded, and the data block may be set as a first target reference data block matching the attribute information of the data block to be coded. If the media type of the multimedia data frame is a non-intra-frame type, the second set of reference data blocks may be searched for a data block having the highest correlation with the data block to be coded, and the data block may be set as a first target reference data block matching the attribute information of the data block to be coded. Adaptively selecting reference data blocks matching the data block to be coded based on the media type of the multimedia data frame is advantageous to improving the accuracy and flexibility of reference data block acquisition.

[0058] In addition, the computer's searching for a data block having the highest correlation with the data block to be coded from the first reference data block set and the second reference data block set, and determining it as a first target reference data block matching the attribute information of the data block to be coded, may also be searching for a pixel having the highest correlation with a pixel in the data block to be coded from an area consisting of data blocks in the first reference data block set and the second reference data block set, and determining the area containing the pixel having the highest correlation with the pixel in the data block to be coded as the first target reference data block matching the attribute information of the data block to be coded.

[0059] Matching method 2: When the attribute information of the data block to be coded includes a reference flag of the multimedia data frame to which the data block to be coded belongs, the computer device obtains a first target reference data block that matches the attribute information of the data block to be coded by: if the reference flag of the multimedia data frame is a first reference flag, searching for a data block that has the highest correlation with the data block to be coded from the first reference data block set or the third reference data block set, and determining the first target reference data block that matches the attribute information of the data block to be coded; if the reference flag of the multimedia data frame is a second reference flag, searching for a data block that has the highest correlation with the data block to be coded from the second reference data block set, and determining the first target reference data block that matches the attribute information of the data block to be coded. The reference flag may be preset by the sender (or receiver) of the multimedia data frame. Therefore, adaptively selecting a first target reference data block based on the reference flag is equivalent to adaptively selecting a first target reference data block according to user coding needs, which is advantageous to improving the flexibility and accuracy of the selected reference data block.

[0060] The first reference flag is used to indicate that the selection range of reference data blocks for the data block to be coded is the first reference data block set or the third reference data block set, and the second reference flag is used to indicate that the selection range of reference data blocks for the data block to be coded is the second reference data block set. The first reference flag and the second reference flag may be preset by the sender (or receiver) of the multimedia data frame, and the first reference flag and the second reference flag may consist of one or more of numbers, letters, and Chinese characters. For example, suppose the first reference flag (ibc_ref_flag) is 1 and the second reference flag is 0. When the reference flag of a multimedia data frame in the multimedia data frame is 1, the computer device can obtain a first target reference data block that matches the attribute information of the data block to be coded from the first reference data block set, and when the reference flag of a multimedia data frame in the multimedia data frame is 0, the computer device can obtain a first target reference data block that matches the attribute information of the data block to be coded from the second reference data block set.

[0061] Furthermore, when the media type of the multimedia data frame is an intra-frame type, the header of the multimedia data frame includes a reference flag; when the media type of the multimedia data frame is a non-intra-frame type, the header of the multimedia data frame may not include a reference flag; or when the multimedia data frame allows the use of the IBC coding method, the header of the multimedia data frame includes a reference flag; when the multimedia data frame does not allow the use of the IBC coding method, the header of the multimedia data frame may not include a reference flag. In this case, the computer device can determine a first target reference data block that matches the attribute information of the data block to be coded based on whether the header of the multimedia data frame includes a reference flag.

[0062] In matching method 2, if the reference flag of the multimedia data frame is a first reference flag, a data block having the highest correlation with the data block to be coded is searched from the first reference data block set, and is determined as a first target reference data block that matches the attribute information of the data block to be coded, where the first target reference data block refers to any one of the first reference data blocks in the first reference data block set; if the reference flag of the multimedia data frame is a second reference flag, a data block having the highest correlation with the data block to be coded is searched from the second reference data block set, and is determined as a first target reference data block that matches the attribute information of the data block to be coded, where the first target reference data block may refer to any one of the second reference data blocks in the second reference data block set. If the reference flag of the multimedia data frame is a first reference flag, a data block having the highest correlation with the data block to be coded is searched from the first reference data block set and the second reference data block set, and is determined as a first target reference data block matching the attribute information of the data block to be coded, where the first target reference data block is determined based on the first reference data block set and the second reference data block set; if the reference flag of the multimedia data frame is a second reference flag, a data block having the highest correlation with the data block to be coded is searched from the second reference data block set, and is determined as a first target reference data block matching the attribute information of the data block to be coded, where the first target reference data block may refer to any one of the second reference data blocks in the second reference data block set. Adaptively selecting a reference data block matching the data block to be coded based on the reference flag of the multimedia data frame is advantageous to improving the accuracy and flexibility of obtaining reference data blocks.

[0063] Matching method 3: When the attribute information of the data block to be coded includes the clarity of the multimedia data frame to which the data block to be coded belongs, if the clarity of the multimedia data frame to which the data block to be coded belongs is greater than the clarity threshold, it means that the user's quality requirement for the multimedia data frame is high, and a first target reference data block matching the attribute information of the data block to be coded can be obtained from the first set of reference data blocks or the third set of reference data blocks.If the clarity of the multimedia data frame to which the data block to be coded belongs is less than the clarity threshold, it means that the user's quality requirement for the multimedia data frame is low, and a first target reference data block matching the attribute information of the data block to be coded can be obtained from the second set of reference data blocks. Since the clarity of the multimedia data frame is used to reflect the user's quality requirements for the multimedia data frame, adaptively selecting the first target reference data block based on the clarity of the multimedia data frame is equivalent to adaptively selecting the first target reference data block according to the user's quality needs for the multimedia data frame, which is advantageous in improving the flexibility and accuracy of the selected reference data block.

[0064] Furthermore, when the media type of the multimedia data frame is a non-intra-frame type, the reference flag of the multimedia data frame is the second reference flag, or the clarity of the multimedia data frame is below the clarity threshold, the computer device can obtain a first target reference data block that matches the attribute information of the data block to be coded from the second set of reference data blocks. At this time, the computer device only needs to store the second set of reference data blocks and does not need to store the first set of reference data blocks, thereby saving the storage space of the computer device, narrowing the search range for the reference data blocks, and improving the search efficiency for the reference data blocks.

[0065] In some embodiments, the computer device may select one of the following two methods to obtain the first set of reference data blocks based on the coding method of the multimedia data frame:

[0066] Manner 1: The first reference data block set includes one or more first reference data blocks, and when the coding method of the multimedia data frame is a serial coding method, the first reference data blocks are coded data blocks in the multimedia data frame whose parameter association relationship satisfies a write-back restriction condition and belong to the same coding tile as the data block to be coded, the parameter association relationship is used to reflect the association relationship between first position information, size information of the largest coding block, and second position information, the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame, and the write-back restriction condition is used to reflect the restriction delay of writing the coded data block to the first reference data block set. In other words, the serial coding method refers to predictive coding of only one data block in a multimedia data frame at a time, and when the coding method of the multimedia data frame is a serial coding method, the serial coding method does not affect the reference data block determination process. However, writing the coded data block to the first reference data block set may result in a large write-back delay, i.e., it takes a long time to write the coded data block to the first reference data block set, so the coded data block can be used as the reference data block after waiting for at least a predetermined time length, where the predetermined time length is determined based on a limit delay for writing the coded data block to the first reference data block set, and the limit delay may refer to one of an average delay, a maximum delay, a minimum delay, etc. for writing the coded data block to the first reference data block set.As can be seen, when the coding method of the multimedia data frame is a serial coding method, the first reference data block in the first reference data block set must satisfy the following two conditions: a) it belongs to the same coding tile as the data block to be coded; and b) the parameter association relationship satisfies the write-back restriction condition.

[0067] In the implementation process, the computer device can obtain the first reference data block set by performing the following steps: when the coding method of the multimedia data frame is a serial coding method, obtain first position information, size information of the largest coding block, and second position information, respectively, where the first position information and second position information refer to the position information of the coded data block and the data block to be coded in the multimedia data frame, respectively; obtain a write-back constraint condition, where the write-back constraint condition is used to reflect the limit delay of writing the coded data block to the first reference data block set; select from the multimedia data frame a coded data block whose parameter association relationship satisfies the write-back constraint condition and belongs to the same coding tile as the data block to be coded, as the first reference data block, where the parameter association relationship is used to reflect the association relationship between the first position information, size information of the largest coding block, and second position information; and add the first reference data block to the first reference data block set.

[0068] In an embodiment of the present application, when the coding method of the multimedia data frame is a serial coding method, the computer device can obtain first position information, size information of the largest coding block, and second position information, respectively. The first position information and second position information refer to position information of the coded data block and the data block to be coded in the multimedia data frame, respectively. Wherein, the first position information and second position information may refer to the coordinates of the upper left corner pixel of the coded data block and the upper left corner pixel of the data block to be coded in the multimedia data frame, and the size information of the largest coding block may refer to the width and height of the largest coding block. In addition, the computer device can obtain a write-back restriction condition, and select from the multimedia data frame a coded data block whose parameter association relationship satisfies the write-back restriction condition and belongs to the same coding tile as the data block to be coded, as a first reference data block, and add the first reference data block to the first reference data block set. Obtaining the first reference data block by selecting based on the write-back restriction conditions, the first position information, the second position information and the size information of the largest coding block can ensure that the first reference data block and the data block to be coded have high spatial correlation, and also ensure the availability of the first reference data block.

[0069] In some embodiments, the selection of the first reference data block may be realized in the following manner: first, based on the first position information and the second position information, a coded data block belonging to the same coding tile is selected from the multimedia data frame to be set as the first candidate reference data block; then, under the write-back restriction condition, a limit number of interval data blocks between the coded data block and the data block to be coded in the multimedia data frame is determined; then, based on the first position information, the size information of the largest coding block and the second position information, the number of interval data blocks between the first candidate reference data block and the data block to be coded is counted; finally, a first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks is determined as the first reference data block.

[0070] where the first position information and the second position information refer to the coordinates of the pixels of the coded data block and the pixels of the data block to be coded in the multimedia data frame, the position information of the coded data block is (ref_x, ref_y), and the position information of the data block to be coded is (cur_x, cur_y). The computer device selects a coded data block belonging to the same coding tile as the data block to be coded from the multimedia data frame as a first candidate reference data block, and determines a limit number of interval data blocks between the coded data block and the data block to be coded in the multimedia data frame under the write-back limit condition; The interval data block limit number may refer to the minimum number of data blocks to be placed between the coded data block and the data block to be coded in the multimedia data frame when the write-back limit condition is satisfied. Then, based on the first position information, the size information of the largest coding block, and the second position information, the number of interval data blocks between the first candidate reference data block and the data block to be coded is counted, where the interval data block number refers to the number of data blocks actually present between the first candidate reference data block and the data block to be coded. The first candidate reference data block whose interval data block number is greater than the interval data block limit number can be determined as the first reference data block. Obtaining the first reference data block set based on the position information of the coded data block, the position information of the data block to be coded, the write-back delay limit condition, etc., can ensure that the first reference data block and the data block to be coded have a high spatial correlation, and can also ensure the availability of the first reference data block. Furthermore, since additional hardware costs are not required, costs can be saved.

[0071] For example, the computer device can determine the first candidate reference data block that satisfies the following formula (2) as the first reference data block.

[0072]

number

[0073] In some embodiments, counting the number of interval data blocks between the first candidate reference data block and the data blocks to be coded may be realized in the following manner: first, determine the row indicator (ID) and column indicator of the first candidate reference data block based on the first position information and the size information of the largest coding block; then, determine the row indicator and column indicator of the data block to be coded based on the second position information and the size information of the largest coding block; then, determine the number of data blocks in each row in the multimedia data frame based on the size information of the data blocks to be coded; and finally, determine the number of interval data blocks between the first candidate reference data block and the data blocks to be coded based on the number of data blocks in each row, the row indicator and column indicator of the first candidate reference data block, and the row indicator and column indicator of the data blocks to be coded.

[0074] Here, the row indicator and column indicator of the first candidate reference data block are ref_y_num and ref_x_num respectively, and the row indicator and column indicator of the data block to be coded are cur_y_num and cur_x_num respectively. The computer device can respectively obtain the row indicator and column indicator of the first candidate reference data block and the row indicator and column indicator of the data block to be coded by adopting the following formula (3):

[0075]

number

[0076] The computer device determines the number of data blocks in each row in the multimedia data frame based on the size information of the data blocks to be coded; and determines the number of a first candidate reference data block based on the number of data blocks in each row and the row indicator and column indicator of the first candidate reference data block, and determines the number of a data block to be coded based on the number of data blocks in each row and the row indicator and column indicator of the data block to be coded, and determines the difference between the number of the first candidate reference data block and the number of a data block to be coded as the number of interval data blocks between the first candidate reference data block and the data block to be coded. For example, the computer device can use the following formula (4) to determine the number of data blocks in each row in the multimedia data frame, the number of the first candidate reference data block, and the number of a data block to be coded.

[0077]

number

[0078] In addition, when the coding method of the multimedia data frame is a serial coding method, the computer device may further obtain a first reference data block set in the following manner: determine an interval distance between the coded data block and the data block to be coded according to the first position information, the second position information and the size information of the largest coding block; determine an interval limit distance between the coded data block and the data block to be coded under a write-back delay condition, where the interval limit distance refers to the minimum interval distance between the coded data block and the data block to be coded when the write-back delay condition is satisfied; and select a coded data block that belongs to the same coding tile as the data block to be coded in the multimedia data frame and has an interval distance greater than the interval limit distance as the first reference data block, and add the first reference data block to the first reference data block set.

[0079] In some embodiments, the first set of reference data blocks includes one or more second reference data blocks, and when the coding method of the multimedia data frame is a parallel coding method or a serial coding method, the second reference data blocks are coded data blocks that belong to the same coding tile as the data blocks to be coded, and whose parameter association relationship in the multimedia data frame satisfies a write-back restriction condition and a coding process condition, the parameter association relationship is used to reflect the association relationship between the position information of the coded data blocks and the data blocks to be coded in the multimedia data frame and the size information of the largest coding block, and the write-back restriction condition is used to reflect the limit delay for writing the coded data blocks to the first set of reference data blocks. In other words, a parallel coding method refers to predictive coding of multiple data blocks in a multimedia data frame at the same time, and when the coding method of the multimedia data frame is a parallel coding method, the coding method may affect the reference data block determination process. In addition, writing the coded data blocks to the first set of reference data blocks may result in a large write-back delay. It can be seen that when the coding method of the multimedia data frame is a parallel coding method, the second reference data block in the first set of reference data blocks must satisfy three conditions: a) belong to the same coding tile as the data block to be coded; b) the parameter association relationship satisfies the write-back restriction condition; and c) the parameter association relationship satisfies the coding process condition. In some embodiments, when the coding method of the multimedia data frame is a serial coding method or a parallel coding method, both can be achieved by the above-mentioned method of obtaining the first set of reference data blocks. The coding process condition here may refer to the parallel coding process condition.

[0080] In an embodiment of the present application, a computer device can perform the following steps to obtain a first set of reference data blocks: obtain first location information, size information of the largest coding block, and second location information, where the first location information and the second location information respectively refer to location information of coded data blocks and data blocks to be coded in a multimedia data frame; then obtain a write-back restriction condition and a coding process condition, where the write-back restriction condition is used to reflect a limit delay for writing the coded data blocks into the first set of reference data blocks; then select, from the multimedia data frame, a coded data block whose parameter association relationship satisfies the write-back restriction condition and the coding process condition and which belongs to the same coding tile as the data blocks to be coded, as a second reference data block, where the parameter association relationship is used to reflect the association relationship between the first location information, size information of the data blocks to be coded, and second location information; and finally add the second reference data block to the first set of reference data blocks.

[0081] wherein the first position information and the second position information may refer to the coordinates of the upper left corner pixel of the coded data block and the upper left corner pixel of the data block to be coded in the multimedia data frame, and the size information of the largest coding block may refer to the width and height of the largest coding block. The write-back constraint condition is used to reflect a limit delay for writing the coded data block to the first set of reference data blocks, the coding process condition is used to reflect a limit distance between the reference data block and the data block to be coded when coding is performed in parallel on the multimedia data frame, or the coding process condition is used to reflect a limit number of interval data blocks between the reference data block and the data block to be coded when coding is performed in parallel on the multimedia data frame. From the multimedia data frame, a coded data block whose parameter association relationship satisfies the write-back constraint condition and the coding process condition and which belongs to the same coding tile as the data block to be coded is selected as a second reference data block, and the second reference data block is added to the first set of reference data blocks, thereby ensuring high spatial correlation between the second reference data block and the data block to be coded and ensuring the availability of the second reference data block.

[0082] In some embodiments, the selection of the second reference data block may be realized in the following manner: based on the first position information and the second position information, select a coded data block from the multimedia data frame that belongs to the same coding tile as the data block to be coded, as the first candidate reference data block; then, from among the first candidate reference data blocks, select a first candidate reference data block whose parameter association relationship satisfies a write-back restriction condition, as the second candidate reference data block; then, determine a first data block restriction distance between the coded data block and the data block to be coded in the multimedia data frame under the write-back restriction condition, and also determine a second data block restriction distance between the coded data block and the data block to be coded in the multimedia data frame under the coding process condition; finally, based on the first position information, the size information of the largest coding block, the second position information, the first data block restriction distance, and the second data block restriction distance, determine a second candidate reference data block from among the second candidate reference data blocks that satisfies the coding process condition as the second reference data block.

[0083] Here, the first position information and the second position information refer to the coordinates of the pixels of the coded data block and the pixels of the data block to be coded in the multimedia data frame, the position information of the coded data block is (ref_x, ref_y), and the position information of the data block to be coded is (cur_x, cur_y), and the computer device can select from the multimedia data frame a coded data block that belongs to the same coding tile as the data block to be coded, and set it as the first candidate reference data block. In an embodiment of the present application, the computer device selects, from the first candidate reference data blocks, a first candidate reference data block whose coordinates satisfy the above formula (2) as a second candidate reference data block; determines a first data block limit distance between the coded data blocks and the data blocks to be coded in the multimedia data frame under a write-back limit condition, where the first data block limit distance refers to the minimum distance that must be maintained between the coded data blocks and the data blocks to be coded in the multimedia data frame when the write-back delay limit condition is satisfied; determines a second data block limit distance between the coded data blocks and the data blocks to be coded in the multimedia data frame under a coding processing condition, where the second data block limit distance refers to the minimum distance that must be maintained between the coded data blocks and the data blocks to be coded in the multimedia data frame when the coding processing condition is satisfied; and then, from the second candidate reference data blocks, determines, as the second reference data block, a second candidate reference data block that satisfies the coding processing condition based on the first position information, the size information of the largest coding block, the second position information, the first data block limit distance and the second data block limit distance.

[0084] In some embodiments, determining the second candidate reference data block that satisfies the coding processing conditions as the second reference data block from the above-mentioned second candidate reference data blocks may further be realized as follows: determining row indicators and column indicators of the second candidate reference data block according to the first position information and the size information of the largest coding block; determining row indicators and column indicators of the data block to be coded according to the second position information and the size information of the largest coding block; then selecting the second candidate reference data block whose row indicator is equal to or smaller than the row indicator of the data block to be coded from the second candidate reference data blocks as the third candidate reference data block; and then determining the third candidate reference data block. determine a row distance between a third candidate reference data block and the data block to be coded based on the row indicator of the data block and the row indicator of the data block to be coded; determine a column distance between the third candidate reference data block and the data block to be coded based on the column indicator of the third candidate reference data block and the column indicator of the data block to be coded; finally, determine a third candidate reference data block among the third candidate reference data blocks whose column distance is smaller than a first distance threshold as a second reference data block, where the first distance threshold is determined based on the first data block limit distance, the second data block limit distance, and the row distance between the third candidate reference data block and the data block to be coded.

[0085] Here, the computer device can respectively use the above formula (3) to obtain the row indicator and column indicator of the second candidate reference data block and the row indicator and column indicator of the data block to be coded, and then select the second candidate reference data block whose row indicator is less than or equal to the row indicator of the data block to be coded as the third candidate reference data block. In an embodiment of the present application, the difference between the row indicator of the third candidate reference data block and the row indicator of the data block to be coded is determined as the row distance between the third candidate reference data block and the data block to be coded, and the difference between the column indicator of the third candidate reference data block and the column indicator of the data block to be coded is determined as the column distance between the third candidate reference data block and the data block to be coded. Furthermore, among the third candidate reference data blocks, the third candidate reference data block whose column distance is less than the first distance threshold can be determined as the second reference data block. For example, the computer device can determine the second candidate reference data block that satisfies the following formula (5) as the first reference data block:

[0086]

number

[0087] In addition, when the coding method of the multimedia data frame is a parallel coding method or a serial coding method, the computer device may further obtain a first reference data block set in the following manner: determine the number of interval data blocks between the coded data block and the data block to be coded based on the first position information, the second position information and the size information of the largest coding block; determine a target interval limit data block number between the coded data block and the data block to be coded under the write-back delay conditions and the coding processing conditions, where the target interval limit data block number refers to the minimum interval data block number between the coded data block and the data block to be coded when the write-back delay conditions and the coding processing conditions are satisfied; and add coded data blocks that belong to the same coding tile as the data block to be coded in the multimedia data frame and have a number of interval data blocks greater than the target interval limit data block number to the first reference data block set.

[0088] In some embodiments, the computer device may select one of the following two methods or a combination of at least two of them to obtain the second set of reference data blocks:

[0089] Scheme 1: The second set of reference data blocks includes one or more third reference data blocks, which are coded data blocks located in the same coding processing unit as the data blocks to be coded in the multimedia data frame. The coding processing unit may be referred to as a basic unit of on-chip memory (VPDU, Virtual Pipeline Data Units), and the size information of the VPDU may be determined based on the coding processing capability of the coder. For example, the size of the VPDU is VPDU_W*VPDU_H, and the values ​​of VPDU_W and VPDU_H can be 4, 8, 16, 32, 64, 128, etc. In other words, the coded data block whose coordinates in the multimedia data frame satisfy ref_x / VPDU_W=cur_x / VPDU_W and ref_y / VPDU_H=cur_y / VPDU_H is the second reference data block. Also, for example, if the size of the VPDU is 64x64, all coded data blocks in the VPDU 64x64 to which the data block to be coded belongs can be used as reference data blocks for the data block to be coded. Alternatively, if the size of the VPDU is 64x64, a coded data block in the multimedia data frame whose coordinates satisfy Floor(ref_x / 64)=Floor(cur_x / 64) and Floor(ref_y / 64)=Floor(cur_y / 64) is used as the third reference data block.

[0090] In some embodiments, each pixel of a reference data block of a data block to be coded of a multimedia data frame may be located in the same VPDU.

[0091] Manner 2: The second set of reference data blocks includes one or more fourth reference data blocks, which are coded data blocks that are adjacent to the data blocks to be coded in the multimedia data frame and belong to the same coding tile as the data blocks to be coded. The adjacent relationship between the fourth reference data block and the data blocks to be coded may include one or more combinations of the following three cases: a) The adjacent relationship between the fourth reference data block and the data blocks to be coded refers to the fact that the fourth reference data block belongs to the same largest coding block as the data blocks to be coded; b) The fourth reference data block and the data block to be coded have an adjacent relationship when the fourth reference data block is among the N largest coding blocks adjacent to the data block to be coded, and the size of the largest coding block is smaller than a size threshold, where N is determined based on size information of the largest coding block. For example, when the coding direction of a multimedia data frame is from left to right, the size threshold may be set dynamically based on application scenarios or according to user needs, and may be 128x128, for example. In this case, the fourth reference data block and the data block to be coded have an adjacent relationship when the fourth reference data block is among the N largest coding blocks to the left of the data block to be coded, and the size of the largest coding block is smaller than 128x128. N may be determined based on formula (6) as follows:

[0092]

number

[0093] c) The fourth reference data block and the data block to be coded have an adjacent relationship means that the fourth reference data block is among the N largest coding blocks adjacent to the data block to be coded, and the size of the largest coding block is equal to or greater than a size threshold. In a multimedia data frame, after the fourth reference data block moves one largest coding block in a first direction (e.g., moves to the right), the data block located at the top corner (e.g., the top left corner) in the second direction in the area where the fourth reference data block is located after the movement is an unreconstructed data block. Here, the first direction and the second direction are two opposite directions, and the first direction refers to the movement direction of the fourth reference data block, which is the same as the coding direction of the data block to be coded. In other words, for a row of data blocks to be coded arranged horizontally, if the left or right direction is the coding direction of the data block to be coded, then left to right is the first direction, and right to left is the second direction. When the fourth reference data block moves a distance of one largest coding block (the moving direction is the same as the coding direction), its position information is different from that of the data block to be coded, and N is determined based on the size information of the largest coding block. In other words, when the size of the largest coding block is equal to or larger than a size threshold, for example, when the size of the largest coding block is 128x128, the fourth reference data block needs to satisfy the following three conditions: 1) it is a coded data block that belongs to the same coding tile as the data block to be coded; 2) if the coordinates of the fourth reference data block after moving 128 pixels to the right are (ref_x+128, ref_y), then the data block at the upper left corner of the area with coordinates (ref_x+128, ref_y) and size of 64x64 in the multimedia data frame is an unreconstructed data block; and 3) when the fourth reference data block moves a distance of one largest coding block to the right, its position information is different from that of the data block to be coded, and these conditions can be expressed by the following formula (7):

[0094]

number

[0095] In the embodiment of the present application, the distances between the first target reference data block and the data blocks to be coded are all smaller than the second distance threshold, and the second distance threshold can be dynamically set according to the application scenario or user needs, for example, the second distance threshold can be set to 2 14 and the first target reference data block is located at an integer pixel position of the multimedia data frame.

[0096] For example, if the coding method of a multimedia data frame is a parallel coding method, the size of the largest coding block is 128*128, the number of interval-restricted data blocks between the coded data blocks and the data blocks to be coded under the write-back delay restriction condition is 4, and the number of interval-restricted data blocks between the coded data blocks and the data blocks to be coded under the coding process condition is 2, the method of obtaining the second reference data block set is Scheme 1 described above, and when the size of the VPDU is 64x64, the relationship between the first reference data block set, the second reference data block set, and the data blocks to be coded is as shown in Figure 8. In Figure 8, the size of one smallest square (i.e., the small square drawn with dotted lines) is 64x64, the size of the small square drawn with solid lines is 128x128, and the size of the data block is 32x32. Therefore, in Figure 8, the coded data block in the 19th square of the first row in the multimedia data frame belongs to the second reference data block set, and this second reference data block set is used to code the data block to be coded in the 19th square of the first row. In the multimedia data frame, coded data blocks that are eight data blocks away from the data blocks to be coded in the first and second rows belong to a first reference data block set, which is used to code the data block to be coded in the 19th square in the first row. As can be seen from Figure 8, the number of reference data blocks in the first reference data block set is greater than the number of reference data blocks in the second reference data block set, and the reference data blocks in the first reference data block set do not have an adjacent relationship with the data blocks to be coded, while the reference data blocks in the second reference data block set do have an adjacent relationship with the data blocks to be coded.

[0097] For example, if the coding method for a multimedia data frame is a parallel coding method, the size of the largest coding block is 128*128, the number of interval-restricted data blocks between the coded data blocks and the data blocks to be coded under the write-back delay restriction condition is 4, and the number of interval-restricted data blocks between the coded data blocks and the data blocks to be coded under the coding process condition is 2, the method for obtaining the second reference data block set is the above-mentioned method 2, and when the size of the VPDU is 64x64, the relationship between the first reference data block set, the second reference data block set, and the data blocks to be coded is as shown in Figure 9. In Figure 9, the size of one smallest square (i.e., the small square drawn with dotted lines) is 64x64, the size of the small square drawn with solid lines is 128x128, and the size of the data block is 32x32. 9, the coded data block in the 19th square of the first row of the multimedia data frame and the coded data block in the largest coding block to the left of the data block to be coded belong to the second set of reference data blocks, which is used to code the data block to be coded in the 19th square of the first row. The coded data blocks in the first and second rows of the multimedia data frame that are eight data blocks away from the data blocks to be coded belong to the first set of reference data blocks, which is used to code the data block to be coded in the 19th square of the first row. As can be seen from Fig. 9, the number of reference data blocks in the first set of reference data blocks is greater than the number of reference data blocks in the second set of reference data blocks, and the reference data blocks in the first set of reference data blocks are not adjacent to the data blocks to be coded, while the reference data blocks in the second set of reference data blocks are adjacent to the data blocks to be coded.

[0098] Furthermore, when the chrominance components of the data block to be coded are obtained by predictive coding based on the luminance components of the first target reference data block, and the first target reference data block is the first reference data block, both the luminance and chrominance components of the first target reference data block must satisfy the conditions for the first reference data block, and the conditions for the first reference data block may refer to the conditions for determining the first reference data block set described above. When the chrominance components of the data block to be coded are obtained by predictive coding based on the luminance components of the first target reference data block, and the first target reference data block is the second reference data block, both the luminance and chrominance components of the first target reference data block must satisfy the conditions for the second reference data block, and the conditions for the second reference data block may refer to the conditions for determining the second reference data block set described above.

[0099] In some embodiments, an adjacent relationship between a data block to be coded and a coded data block may refer to the coding order of the data block to be coded being adjacent to the coding order of the coded data block, and no adjacent relationship between a data block to be coded and a coded data block may refer to the coding order of the data block to be coded not being adjacent to the coding order of the coded data block.

[0100] In some embodiments, the number of data blocks to be coded in the multimedia data frame is plural, attribute information of each data block to be coded in the multimedia data frame is obtained, and when it is determined based on the attribute information of each data block to be coded in the multimedia data frame that first target reference data blocks corresponding to all data blocks to be coded in the multimedia data frame all belong to the second reference data block set, a filtering process is performed on the data blocks to be coded in the multimedia data frame, and when it is determined based on the attribute information of each data block to be coded in the multimedia data frame that first target reference data blocks corresponding to the data blocks to be coded in the multimedia data frame are present in the multimedia data frame, the filtering process on the data blocks to be coded in the multimedia data frame is suspended, and when it is determined based on the attribute information of each data block to be coded in the multimedia data frame that second target reference data blocks corresponding to the data blocks to be coded in the multimedia data frame, the first region of the second target reference data block belongs to the first reference data block set and the second region of the second target reference data block belongs to the second reference data block set, the filtering process on the data blocks to be coded in the multimedia data frame is suspended. In other words, on the coding side, when all data blocks in the multimedia data frame are coded using the reference data blocks in the second reference data block set, it is allowed to use a loop filter to perform filtering processing on the multimedia data frame, which is beneficial to improving the quality of the multimedia data frame.When a data block in a multimedia data frame is coded using a reference data block in a first reference data block set, or when a data block in the multimedia data frame is coded using a first target reference data block, part of which belongs to the first reference data block set and part of which belongs to a second reference data block set, the loop filter is prohibited from filtering the multimedia data frame, and prohibiting the loop filter from processing the multimedia data frame can avoid an increase in the need for additional image storage, which is advantageous for saving storage space. Loop filtering may include, but is not limited to, deblocking filtering (DBF), sample adaptive offset (SAO), adaptive loop filtering (ALF), constrained directional enhancement filtering (CDEF), loop restoration filtering (LR), cross component sample offset (CCSO), etc.

[0101] 10 is a flow diagram of a multimedia data processing method provided by an embodiment of the present application. As shown in FIG. 10, the method includes at least the following S201-S203.

[0102] S201: Obtain attribute information of a data block to be decoded in a multimedia data frame.

[0103] In an embodiment of the present application, the computer device can obtain attribute information of the data block to be decoded in the multimedia data frame, and the data block to be decoded may refer to the currently coded data block mentioned above, and the data block to be decoded may refer to the data block to be decoded among the undecoded data blocks in the multimedia data frame, that is, the data block to be decoded may refer to the undecoded data block that is closest to the decoded data block in the multimedia data frame.

[0104] The attribute information of the data block to be decoded may include one or more of the media type of the multimedia data frame to which the data block to be decoded belongs, the reference flag of the multimedia data frame, the clarity of the multimedia data frame to which the data block to be decoded belongs, etc. The media type includes intraframe and non-intraframe. The intraframe type may refer to a multimedia data frame with a representative meaning, and the intraframe type includes all intraframe coded frames, Intra frames, I frames, and key frames. An intraframe type frame is only allowed to use intraframe coding technology and does not need to be coded depending on other frames. For example, the intraframe type may refer to a multimedia data frame that can represent the main meaning of video data or image data. The non-intraframe type may refer to a multimedia data frame other than the intraframe type in video data or image data, and the non-intraframe type may be referred to as an interframe coded frame, P frame, B frame, or non-key frame. A non-intraframe type frame is allowed to use both interframe coding technology and intraframe coding technology. Generally, when using interframe coding technology, coding can be performed after reference frame coding is completed. The reference flags of a multimedia data frame are used to reflect the selection of reference data blocks for a data block waiting to be decoded.

[0105] S202: Obtain a second target reference data block that matches the attribute information of the data block to be decoded, where the second target reference data block belongs to the first reference data block set, the second reference data block set, or a third reference data block set that is a combination of the first reference data block set and the second reference data block set, where the first reference data block set includes decoded data blocks that are not adjacent to the data block to be decoded, and the second reference data block set includes decoded data blocks that are adjacent to the data block to be decoded.

[0106] In an embodiment of the present application, in a multimedia data frame, both decoded data blocks that are not adjacent to the data block to be decoded and decoded data blocks that are adjacent to the data block to be decoded can be used as reference data blocks, and the computer device adaptively selects a second target reference data block from the decoded data blocks that have an adjacent relationship and the decoded data blocks that do not have an adjacent relationship based on the attribute information of the data block to be decoded, thereby improving the flexibility and accuracy of obtaining the reference data block.

[0107] In addition, the decoded data block in the embodiments of the present application may be referred to as the decoding-side reconstructed data block, and the reconstructed data block refers to the decoded data block after the data block has been restored. The second target reference data block may refer to the decoded data block adjacent to the data block to be decoded, or the second target reference data block may refer to the decoded data block not adjacent to the data block to be decoded, or the second region of the second target reference data block may belong to the decoded data block adjacent to the data block to be decoded, and the second region of the second target reference data block may belong to the decoded data block not adjacent to the data block to be decoded.

[0108] S203: Decode the data block to be decoded based on the second target reference data block.

[0109] In an embodiment of the present application, the computer device determines position information of the second target reference data block and the data block to be decoded in each multimedia data frame, determines a block vector between the second target reference data block and the data block to be decoded based on the position information, and then performs prediction based on the block vector and the second target reference data block to obtain a predicted signal of the data block to be decoded, and adds the predicted signal to the residual signal of the data block to be decoded to obtain the original signal of the data block to be decoded. Decoding the data block to be decoded is advantageous for removing the multimedia data frame and performing reconstruction processing, thereby improving the quality of the multimedia data frame.

[0110] Generally, video data includes multiple multimedia data frames, or image data includes multiple multimedia data frames, but the above-mentioned data block to be decoded and the second target reference data block all belong to the same multimedia data frame.

[0111] In an embodiment of the present application, a computer device obtains a second target reference data block that matches the attribute information of the data block to be decoded, and the second target reference data block belongs to the first reference data block set, the second reference data block set, or a third reference data block set that is a combination of the first reference data block set and the second reference data block set. In other words, the reference data block sets available for selection include the first reference data block set, the second reference data block set, and the third reference data block set. Thus, the selection range of reference data blocks is expanded. Furthermore, by adaptively selecting the second target reference data block according to the attribute information of the data block to be decoded and decoding the data block to be decoded based on the second target reference data block, the flexibility and accuracy of obtaining the second target reference data block can be improved, and the decoding performance of the multimedia data frame can be improved. Furthermore, decoding the data block to be decoded based on the second target reference data block is advantageous to the reconstruction process of the multimedia data frame and the quality of the multimedia data frame.

[0112] In some embodiments, the computer device can obtain a second target reference data block that matches the attribute information of the data block to be decoded by one or a combination of the following three matching methods:

[0113] Matching method 1: if the media type of the multimedia data frame is an intra-frame type, a second target reference data block matching the attribute information of the data block to be decoded is obtained from the first reference data block set or the third reference data block set, and the second target reference data block is determined based on the first reference data block and the second reference data block; if the media type of the multimedia data frame is a non-intra-frame type, a second target reference data block matching the attribute information of the data block to be decoded is obtained from the second reference data block set. The adjacent relationship between the data block to be decoded and the decoded data block may refer to the decoding order of the data block to be decoded being adjacent to the decoding order of the decoded data block; and the lack of adjacent relationship between the data block to be decoded and the decoded data block may refer to the decoding order of the data block to be decoded not being adjacent to the decoding order of the decoded data block.

[0114] The first reference data block set and the second reference data block set are stored in a first memory space and a second memory space, respectively, and the first memory space and the second memory space may be located in the same computer device or in different computer devices.

[0115] In matching method 1, if the media type of the multimedia data frame is an intra-frame type, a data block having the highest correlation with the data block to be decoded is searched from the first reference data block set, and can be set as a second target reference data block that matches the attribute information of the data block to be decoded. If the media type of the multimedia data frame is a non-intra-frame type, a data block having the highest correlation with the data block to be decoded is searched from the second reference data block set, and can be set as a second target reference data block that matches the attribute information of the data block to be decoded. Adaptively selecting a reference data block that matches the data block to be decoded based on the media type of the multimedia data frame is advantageous to improving the accuracy and flexibility of reference data block acquisition.

[0116] Matching method 2: when the attribute information of the data block to be decoded includes a reference flag of the multimedia data frame to which the data block to be decoded belongs, the computer device obtaining a second target reference data block that matches the attribute information of the data block to be decoded includes: if the reference flag of the multimedia data frame is the first reference flag, obtaining a second target reference data block that matches the attribute information of the data block to be decoded from the first reference data block set or the third reference data block set; if the reference flag of the multimedia data frame is the second reference flag, obtaining a second target reference data block that matches the attribute information of the data block to be decoded from the second reference data block set. Since the reference flag may be preset by the sender (or receiver) of the multimedia data frame, adaptively selecting a second target reference data block based on the reference flag corresponds to adaptively selecting a second target reference data block according to user coding needs, which is advantageous to improving the flexibility and accuracy of the selected reference data block.

[0117] The first reference flag is used to indicate that the selected range of reference data blocks for the data block to be decoded is the first reference data block set or the third reference data block set, and the second reference flag is used to indicate that the selected range of reference data blocks for the data block to be decoded is the second reference data block set. The first reference flag and the second reference flag may be preset by the sender (or receiver) of the multimedia data frame, and the first reference flag and the second reference flag may be composed of one or more of numbers, letters, and Chinese characters.

[0118] In matching method 2, if the reference flag of the multimedia data frame is the first reference flag, the first reference data block set and the second reference data block set (i.e., the third reference data block set) are searched for a data block that has the highest correlation with the data block to be decoded, and this data block is determined as a second target reference data block that matches the attribute information of the data block to be decoded, where the second target reference data block is determined based on the second reference data block set and the second reference data block set, and if the reference flag of the multimedia data frame is the second reference flag, the second reference data block set is searched for a data block that has the highest correlation with the data block to be decoded, and this data block is determined as a second target reference data block that matches the attribute information of the data block to be decoded. Adaptively selecting the reference data block that matches the data block to be decoded based on the reference flag of the multimedia data frame is advantageous to improving the accuracy and flexibility of obtaining reference data blocks.

[0119] In matching method 2, if the reference flag of the multimedia data frame is a first reference flag, a data block having the highest correlation with the data block to be decoded is searched from the first reference data block set, and the data block is determined as a second target reference data block that matches the attribute information of the data block to be decoded, where the second target reference data block may refer to any one of the reference data blocks in the first reference data block set; if the reference flag of the multimedia data frame is a second reference flag, a data block having the highest correlation with the data block to be decoded is searched from the second reference data block set, and the second target reference data block is determined as a second target reference data block that matches the attribute information of the data block to be decoded, where the second target reference data block may refer to any one of the reference data blocks in the second reference data block set. Adaptively selecting reference data blocks that match the data blocks to be decoded based on the reference flag of the multimedia data frame is advantageous to improving the accuracy and flexibility of reference data block acquisition.

[0120] Matching method 3: When the attribute information of the data block to be decoded includes the clarity of the multimedia data frame to which the data block to be decoded belongs, if the clarity of the multimedia data frame to which the data block to be decoded belongs is greater than the clarity threshold, it means that the user's quality requirement for the multimedia data frame is high, and a second target reference data block matching the attribute information of the data block to be decoded can be obtained from the first set of reference data blocks or the third set of reference data blocks.If the clarity of the multimedia data frame to which the data block to be decoded belongs is less than the clarity threshold, it means that the user's quality requirement for the multimedia data frame is low, and a second target reference data block matching the attribute information of the data block to be decoded can be obtained from the second set of reference data blocks. Since the clarity of a multimedia data frame can be used to reflect a user's quality requirements for the multimedia data frame, adaptively selecting a second target reference data block based on the clarity of the multimedia data frame is equivalent to adaptively selecting a second target reference data block according to a user's quality needs for the multimedia data frame, which is advantageous in improving the flexibility and accuracy of the selected reference data block.

[0121] In some embodiments, the computer device can select one of the following two methods to obtain the first set of reference data blocks based on the decoding method of the multimedia data frame: Method 1: When the decoding method of the multimedia data frame is a serial decoding method, the first reference data blocks in the first set of reference data blocks are decoded data blocks in the multimedia data frame whose parameter association relationship satisfies the write-back restriction condition and belong to the same decoding tile as the data blocks to be decoded, the parameter association relationship is used to reflect the association relationship between first position information, size information of the largest decoding block, and second position information, the first position information and second position information respectively refer to the position information of the decoded data blocks and the data blocks to be decoded in the multimedia data frame, and the write-back restriction condition is used to reflect the restriction delay for writing the decoded data blocks to the first set of reference data blocks. In other words, the serial decoding method refers to decoding only one data block in a multimedia data frame at a time, and when the decoding method of the multimedia data frame is a serial decoding method, the serial decoding method does not affect the reference data block determination process. However, writing the decoded data block to the first reference data block set may incur a large write-back delay, i.e., it takes a long time to write the decoded data block to the first reference data block set. Therefore, the decoded data block can be used as a reference data block after waiting for at least a predetermined time period, where the predetermined time period is determined based on a limit delay for writing the decoded data block to the first reference data block set, and the limit delay may refer to one of an average delay, a maximum delay, a minimum delay, etc. for writing the decoded data block to the first reference data block set.As can be seen, when the decoding method of the multimedia data frame is a serial decoding method, the first reference data block in the first reference data block set must satisfy the following two conditions: a) it belongs to the same decoding tile as the data block waiting to be decoded; and b) the parameter association relationship satisfies the write-back restriction condition.

[0122] Method 2: When the decoding method of the multimedia data frame is a parallel decoding method or a serial decoding method, the first reference data block of the first reference data block set is a decoded data block that belongs to the same decoding tile as the data block to be decoded, and whose parameter association relationship in the multimedia data frame satisfies a write-back restriction condition and satisfies a decoding process condition, where the parameter association relationship is used to reflect the association relationship between the position information of the decoded data block and the data block to be decoded in the multimedia data frame, and the size information of the largest decoding block, and the write-back restriction condition is used to reflect the limit delay for writing the decoded data block to the first reference data block set. In other words, the parallel decoding method refers to performing predictive decoding on multiple data blocks in a multimedia data frame at the same time. When the decoding method of the multimedia data frame is a parallel decoding method, the decoding method may affect the reference data block determination process. In addition, writing the decoded data block to the first reference data block set may result in a large write-back delay. It can be seen that when the decoding method of the multimedia data frame is a parallel decoding method, the first reference data block in the first set of reference data blocks must satisfy the following three conditions: a) belong to the same decoding tile as the data block to be decoded; b) the parameter association relationship satisfies the write-back restriction condition; and c) the parameter association relationship satisfies the decoding processing condition. In particular, when the decoding method of the multimedia data frame is a serial decoding method, Scheme 2 can be adopted to obtain the first set of reference data blocks. The decoding processing condition here can also refer to the parallel decoding processing condition.

[0123] The method of obtaining the first reference data block set can refer to the method of obtaining the first reference data block set in the coding stage, and a redundant description will be omitted here.

[0124] In some embodiments, the computer device can obtain the second set of reference data blocks by adopting one of the following two methods or a combination of at least two of them: Method 1: The second reference data blocks of the second set of reference data blocks are decoded data blocks located in the same decoding processing unit as the data blocks to be decoded in the multimedia data frame; Method 2: The second reference data blocks of the second set of reference data blocks are decoded data blocks that are adjacent to the data blocks to be decoded in the multimedia data frame and belong to the same decoding tile as the data blocks to be decoded. Wherein, the existence of an adjacent relationship between the second reference data block and the data block waiting to be decoded may be one or a combination of the following cases: a) the existence of an adjacent relationship between the second reference data block and the data block waiting to be decoded may refer to the second reference data block belonging to the same largest decoding block as the data block waiting to be decoded; and b) the existence of an adjacent relationship between the second reference data block and the data block waiting to be decoded may refer to the second reference data block being within the N largest decoding blocks adjacent to the data block waiting to be decoded, and the size of the largest decoding block is smaller than a size threshold, where N is determined according to the size information of the largest decoding block.

[0125] In some embodiments, the distances between the first reference data block, the second reference data block and the data block to be decoded are all less than a second distance threshold, and the second distance threshold may be dynamically set according to application scenarios or user needs, for example, the second distance threshold is 2 14and the first and second reference data blocks are located at integer pixel positions of the multimedia data frame. The number of data blocks to be decoded in the multimedia data frame may be plural, and attribute information of each data block to be decoded in the multimedia data frame is obtained, and when it is determined based on the attribute information of each data block to be decoded in the multimedia data frame that second target reference data blocks corresponding to all data blocks to be decoded in the multimedia data frame all belong to the second reference data block set, a filtering process is performed on the data blocks to be decoded in the multimedia data frame; and when it is determined based on the attribute information of each data block to be decoded in the multimedia data frame that second target reference data blocks corresponding to all data blocks to be decoded that belong to the first reference data block set exist in the multimedia data frame, the filtering process on the data blocks to be decoded in the multimedia data frame is suspended. When it is determined based on the attribute information of each data block awaiting decoding in the multimedia data frame that the multimedia data frame contains a second target reference data block corresponding to the data block awaiting decoding, the first region of the second target reference data block belonging to the first reference data block set and the second region of the second target reference data block belonging to the second reference data block set, the filtering process for the data blocks awaiting decoding in the multimedia data frame is temporarily suspended.

[0126] 11 is a diagram showing the configuration of a multimedia data processing device provided by an embodiment of the present application. The multimedia data processing device may be a computer program (including program code) executed on a computer device, for example, the multimedia data processing device may be application software, and the device may be used to perform corresponding steps in the method provided by the embodiment of the present application. As shown in FIG. 11, the multimedia data processing device may include a first acquisition module 111, a first match module 112, and a coding module 113.

[0127] The first acquiring module is configured to acquire attribute information of a data block to be coded in a multimedia data frame; the first matching module is configured to acquire a first target reference data block matching the attribute information of the data block to be coded, where the first target reference data block is selected from a first reference data block set, a second reference data block set, or a third reference data block set obtained by combining the first reference data block set and the second reference data block set, where the first reference data block set includes coded data blocks that do not have an adjacent relationship with the data block to be coded, and the second reference data block set includes coded data blocks that have an adjacent relationship with the data block to be coded; and the coding module is configured to perform predictive coding on the data block to be coded based on the first target reference data block.

[0128] In some embodiments, the attribute information of the data block to be coded includes a media type of a multimedia data frame to which the data block to be coded belongs, and the first matching module is further configured to, when the media type of the multimedia data frame is an intra-frame type, obtain a first target reference data block matching the attribute information of the data block to be coded from the first reference data block set or the third reference data block set, and when the media type of the multimedia data frame is a non-intra-frame type, obtain a first target reference data block matching the attribute information of the data block to be coded from the second reference data block set.

[0129] In some embodiments, the attribute information of the data block to be coded includes a reference flag of a multimedia data frame to which the data block to be coded belongs, and the first matching module is further configured to, when the reference flag of the multimedia data frame is a first reference flag, obtain a first target reference data block that matches the attribute information of the data block to be coded from the first reference data block set or the third reference data block set, and when the reference flag of the multimedia data frame is a second reference flag, obtain a first target reference data block that matches the attribute information of the data block to be coded from the second reference data block set.

[0130] In some embodiments, the first reference data block set includes one or more first reference data blocks, and when the coding method of the multimedia data frame is a serial coding method, the first reference data blocks are coded data blocks in the multimedia data frame whose parameter association relationship satisfies a write-back restriction condition and which belong to the same coding tile as the data block to be coded, the parameter association relationship is used to reflect the association relationship among first position information, size information of the largest coding block, and second position information, the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame, and the write-back restriction condition is used to reflect the restriction delay of writing the coded data block to the first reference data block set.

[0131] In some embodiments, when the coding method of the multimedia data frame is a serial coding method, the first obtaining module is further configured to obtain first position information, size information of the largest coding block, and second position information, respectively, where the first position information and the second position information refer to position information of the coded data block and the data block to be coded in the multimedia data frame; obtain a write-back restriction condition, where the write-back restriction condition is used to reflect a limit delay for writing the coded data block to the first reference data block set; select from the multimedia data frame a coded data block whose parameter association relationship satisfies the write-back restriction condition and belongs to the same coding tile as the data block to be coded, as the first reference data block; where the parameter association relationship is used to reflect an association relationship among the first position information, size information of the largest coding block, and second position information; and add the first reference data block to the first reference data block set.

[0132] In some embodiments, the first acquisition module is further configured to: select a coded data block belonging to the same coding tile as the data block to be coded from the multimedia data frame based on the first position information and the second position information, as a first candidate reference data block; determine a limit number of interval data blocks between the coded data block in the multimedia data frame and the data block to be coded under the write-back limit condition; count the number of interval data blocks between the first candidate reference data block and the data block to be coded based on the first position information, size information of the largest coding block, and the second position information; and determine a first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks as a first reference data block.

[0133] In some embodiments, the first acquisition module is further configured to determine row and column indicators of the first candidate reference data block based on first position information and size information of the largest coding block, determine row and column indicators of the data block to be coded based on the second position information and size information of the largest coding block, determine the number of data blocks in each row in the multimedia data frame based on the size information of the data blocks to be coded, and count the number of interval data blocks between the first candidate reference data block and the data block to be coded based on the number of data blocks in each row, the row and column indicators of the first candidate reference data block, and the row and column indicators of the data block to be coded.

[0134] In some embodiments, the first reference data block set includes one or more second reference data blocks, which belong to the same coding tile as the data blocks to be coded, and which are coded data blocks in the multimedia data frame whose parameter association relationship satisfies a write-back restriction condition and whose parameter association relationship satisfies a coding process condition, the parameter association relationship is used to reflect an association relationship between position information of the coded data blocks and the data blocks to be coded in the multimedia data frame, and size information of the largest coding block, and the write-back restriction condition is used to reflect a limit delay for writing the coded data blocks to the first reference data block set.

[0135] In some embodiments, the first obtaining module further obtains first position information, size information of the largest coding block, and second position information, respectively, where the first position information and the second position information refer to position information of the coded data block and the data block to be coded in the multimedia data frame; obtains a write-back restriction condition and a coding process condition, where the write-back restriction condition is used to reflect a limit delay for writing the coded data block to the first reference data block set; selects from the multimedia data frame a coded data block whose parameter association relationship satisfies the write-back restriction condition and the coding process condition and which belongs to the same coding tile as the data block to be coded, as a second reference data block; the parameter association relationship is used to reflect an association relationship between the first position information, the size information of the data block to be coded, and the second position information; and is used to add the second reference data block to the first reference data block set.

[0136] In some embodiments, the first obtaining module is further configured to: select, from the multimedia data frame, a coded data block belonging to the same coding tile as the data block to be coded based on the first position information and the second position information, as a first candidate reference data block; select, from the first candidate reference data blocks, a first candidate reference data block whose parameter association relationship satisfies a write-back restriction condition, as a second candidate reference data block; determine a first data block restriction distance between the coded data block and the data block to be coded in the multimedia data frame under the write-back restriction condition; determine a second data block restriction distance between the coded data block and the data block to be coded in the multimedia data frame under the coding process condition; and select, from the second candidate reference data blocks, a second candidate reference data block that satisfies the coding process condition, as a second reference data block, based on the first position information, the size information of the largest coding block, the second position information, and the first data block restriction distance and the second data block restriction distance.

[0137] and determining a row indicator and a column indicator of the data block to be coded based on the second location information and the size information of the largest coding block; selecting a second candidate reference data block from the second candidate reference data blocks, whose row indicator is equal to or smaller than the row indicator of the data block to be coded, as a third candidate reference data block; determining a row distance between the third candidate reference data block and the data block to be coded based on the row indicator of the third candidate reference data block and the row indicator of the data block to be coded; determining a column distance between the third candidate reference data block and the data block to be coded based on the column indicator of the third candidate reference data block and the column indicator of the data block to be coded; and determining a third candidate reference data block from the third candidate reference data blocks, whose column distance is smaller than a first distance threshold, as a second reference data block, wherein the first distance threshold is determined based on the first data block limit distance, the second data block limit distance, and the row distance between the third candidate reference data block and the data block to be coded.

[0138] In some embodiments, the second set of reference data blocks includes one or more third reference data blocks, which are coded data blocks located in the same coding processing unit as the data blocks to be coded in the multimedia data frame, or the second set of reference data blocks includes one or more fourth reference data blocks, which are coded data blocks that are adjacent to the data blocks to be coded in the multimedia data frame and belong to the same coding tile as the data blocks to be coded.

[0139] In some embodiments, the fourth reference data block and the data block to be coded have an adjacent relationship, which means that the fourth reference data block and the data block to be coded belong to the same largest coding block; or the fourth reference data block is among the N largest coding blocks adjacent to the data block to be coded, and the size of the largest coding block is less than or equal to a size threshold, and N is determined based on size information of the largest coding block; or the fourth reference data block is among the N largest coding blocks adjacent to the data block to be coded, and the size of the largest coding block is greater than or equal to a size threshold, and the data block located in the upper left corner of the area in the multimedia data frame where the fourth reference data block is located after moving one largest coding block to the right is the unreconstructed data block, and the position information of the fourth reference data block when it moves one largest coding block is different from the position information of the data block to be coded, and N is determined based on size information of the largest coding block.

[0140] In some embodiments, the distances between the first target reference data block and the data blocks to be coded are all less than a second distance threshold, and the first target reference data block is at an integer pixel position in the multimedia data frame.

[0141] In some embodiments, the number of data blocks to be coded in the multimedia data frame is plural, and the first acquiring module further acquires attribute information of each data block to be coded in the multimedia data frame; and, when determining, based on the attribute information of each data block to be coded in the multimedia data frame, that first target reference data blocks corresponding to all data blocks to be coded in the multimedia data frame all belong to the second reference data block set, performs a filtering process on the data blocks to be coded in the multimedia data frame; and, based on the attribute information of each data block to be coded in the multimedia data frame, assigns the first reference data block to the multimedia data frame. the filtering process for the data blocks to be coded in the multimedia data frame is temporarily suspended when it is determined that a first target reference data block corresponding to the data block to be coded, selected from a set, exists; and the filtering process for the data blocks to be coded in the multimedia data frame is temporarily suspended when it is determined that a first target reference data block corresponding to the data block to be coded, selected from a set, exists in the multimedia data frame based on attribute information of each data block to be coded, the first region of the first target reference data block belonging to the first reference data block set and the second region of the first target reference data block belonging to the second reference data block set, based on attribute information of each data block to be coded in the multimedia data frame.

[0142] According to one embodiment of the present application, the steps in the multimedia data processing method shown in Fig. 7 may be performed by each module in the multimedia data processing device shown in Fig. 11. For example, step S101 shown in Fig. 7 may be performed by the first obtaining module 111 in Fig. 11, step S102 shown in Fig. 7 may be performed by the first matching module 112 in Fig. 11, and step S103 shown in Fig. 7 may be performed by the coding module 113 in Fig. 11.

[0143] 12 is a diagram showing the configuration of another multimedia data processing device provided by an embodiment of the present application. The multimedia data processing device may be a computer program (including program code) executed on a computer device, for example, application software, which can be used to perform corresponding steps in the method provided by an embodiment of the present application. As shown in FIG. 12, the multimedia data processing device may include a second acquisition module 121, a second matching module 122, and a decoding module 123.

[0144] the second acquiring module is configured to acquire attribute information of the data blocks to be decoded in the multimedia data frame; the second matching module is configured to acquire second target reference data blocks matching the attribute information of the data blocks to be decoded, wherein the second target reference data blocks belong to a first reference data block set, a second reference data block set, or a third reference data block set formed by combining the first reference data block set and the second reference data block set; the first reference data block set includes decoded data blocks that do not have an adjacent relationship with the data blocks to be decoded; and the second reference data block set includes decoded data blocks that have an adjacent relationship with the data blocks to be decoded; and the decoding module is configured to decode the data blocks to be decoded based on the second target reference data blocks.

[0145] In some embodiments, the attribute information of the data block to be decoded includes a media type of a multimedia data frame to which the data block to be decoded belongs, and the second matching module is further configured to, if the media type of the multimedia data frame is an intra-frame type, obtain a second target reference data block matching the attribute information of the data block to be decoded from a first set of reference data blocks, and if the media type of the multimedia data frame is a non-intra-frame type, obtain a second target reference data block matching the attribute information of the data block to be decoded from the first set of reference data blocks or a third set of reference data blocks.

[0146] In some embodiments, the attribute information of the data block to be decoded includes a reference flag of a multimedia data frame to which the data block to be decoded belongs, and the second matching module is further configured to, when the reference flag of the multimedia data frame is a first reference flag, obtain a second target reference data block that matches the attribute information of the data block to be decoded from the first reference data block set or the third reference data block set, and when the reference flag of the multimedia data frame is a second reference flag, obtain a second target reference data block that matches the attribute information of the data block to be decoded from the second reference data block set.

[0147] In some embodiments, the first reference data block set includes one or more first reference data blocks, and when the decoding method of the multimedia data frame is a serial decoding method, the first reference data blocks are decoded data blocks in the multimedia data frame whose parameter association relationship satisfies a write-back restriction condition and belong to the same decoding tile as the data block to be decoded, the parameter association relationship is used to reflect an association relationship between first position information, size information of the largest decoding block, and second position information, the first position information and the second position information respectively refer to the position information of the decoded data block and the data block to be decoded in the multimedia data frame, and the write-back restriction condition is used to reflect the restriction delay for writing the decoded data block to the first reference data block set.

[0148] In some embodiments, the first reference data block set includes one or more second reference data blocks, and the second reference data blocks are decoded data blocks that belong to the same decoding tile as the data blocks waiting to be decoded, and whose parameter association relationship in the multimedia data frame satisfies a write-back restriction condition and whose parameter association relationship satisfies a decoding process condition, the parameter association relationship is used to reflect an association relationship between position information of the decoded data blocks and the data blocks waiting to be decoded in the multimedia data frame, and size information of the largest decoding block, and the write-back restriction condition is used to reflect a limit delay for writing the decoded data blocks to the first reference data block set.

[0149] In some embodiments, the second set of reference data blocks includes one or more third reference data blocks, which are decoded data blocks in the same decoding processing unit as the data blocks awaiting decoding in the multimedia data frame, or the second set of reference data blocks includes one or more fourth reference data blocks, which are decoded data blocks that are adjacent to the data blocks awaiting decoding in the multimedia data frame and belong to the same decoding tile as the data blocks awaiting decoding.

[0150] In some embodiments, the number of data blocks to be decoded in the multimedia data frame is plural, and the second acquiring module further acquires attribute information of each data block to be decoded in the multimedia data frame; and, when determining, based on the attribute information of each data block to be decoded in the multimedia data frame, that second target reference data blocks corresponding to all data blocks to be decoded in the multimedia data frame all belong to the second reference data block set, performs a filtering process on the data blocks to be decoded in the multimedia data frame; and, based on the attribute information of each data block to be decoded in the multimedia data frame, adds a second target reference data block corresponding to the data block to be decoded to the multimedia data frame. The filtering process for the data blocks awaiting decoding in the multimedia data frame is temporarily suspended when it is determined that there are two target reference data blocks, where the second target reference data block belongs to the first reference data block set, and the filtering process for the data blocks awaiting decoding in the multimedia data frame is temporarily suspended when it is determined that there is a second target reference data block in the multimedia data frame corresponding to the data blocks awaiting decoding, where a first region of the second target reference data block belongs to the first reference data block set and a second region of the second target reference data block belongs to the second reference data block set, based on attribute information of each data block awaiting decoding in the multimedia data frame.

[0151] According to one embodiment of the present application, the steps in the multimedia data processing method shown in Figure 10 may be performed by each module in the multimedia data processing device shown in Figure 12. For example, step S201 shown in Figure 10 may be performed by the second acquisition module 121 in Figure 12, step S202 shown in Figure 10 may be performed by the second match module 122 in Figure 12, and step S203 shown in Figure 10 may be performed by the decoding module 123 in Figure 12.

[0152] According to one embodiment of the present application, each or all of the modules in the multimedia data processing device shown in Figures 11 and 12 may be integrated into one or more units, or one or more of the units may be further divided into at least two smaller sub-units in terms of functionality to perform the same operations, without affecting the realization of the technical effects of the embodiment of the present application. The above-mentioned modules are divided based on logical functions, and in actual applications, the function of one module may be realized by at least two units, or the functions of at least two modules may be realized by one unit. In other embodiments of the present application, the data processing device may include other units, and in actual applications, these functions may be realized in cooperation with other units and may be realized by cooperation of at least two units.

[0153] According to one embodiment of the present application, a computer program (including program code) capable of executing the steps of the methods shown in Figures 7 and 10 on a general-purpose computer device, such as a computer, including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM) may be executed to form the multimedia data processing device shown in Figures 11 and 12, and the multimedia data processing method of the embodiment of the present application. The computer program may be stored in a computer-readable storage medium, for example, and may be installed on the computer device by the computer-readable storage medium and executed on the computer device.

[0154] FIG. 13 illustrates the configuration of a computer device provided by an embodiment of the present application. As shown in FIG. 13, the computer device 1000 includes a processor 1001, a network interface 1004, and a storage device 1005. The computer device 1000 may further include a media content interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to connect and communicate between these components. The media content interface 1003 may include a display and a keyboard. Optionally, the media content interface 1003 may further include a standard-compliant wired interface or a wireless interface. Optionally, the network interface 1004 includes a standard-compliant wired interface or a wireless interface (e.g., a Wi-Fi interface). The storage device 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one magnetic disk memory. Optionally, the storage device 1005 may further include at least one storage device separate from the processor 1001. 13, a storage device 1005 serving as a computer-readable storage medium includes an operating system, a network communication module, a media content interface module, and a device control application. In the computer device 1000 shown in FIG. 13, a network interface 1004 can provide a network communication function, and a media content interface 1003 is mainly used to provide an interface for inputting media content.

[0155] Although the computer device 1000 described in the embodiments of the present application can execute the steps of the multimedia data processing method described above in the embodiments corresponding to Figures 7 and 10, and can also execute the processing of the multimedia data processing device described above in the embodiments corresponding to Figures 11 and 12, detailed descriptions thereof will be omitted here, and descriptions of the advantageous effects of adopting the same methods will also be omitted.

[0156] In addition, the embodiments of the present application further provide a computer-readable storage medium, which stores a computer program executed by the multimedia data processing device, the computer program including program instructions, and when the processor executes the program instructions, it can perform the steps of the multimedia data processing method in the embodiments corresponding to Figures 7 and 10, so detailed descriptions thereof will be omitted here. Also, descriptions of advantageous effects of adopting the same method will be omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium of the present application, please refer to the descriptions of the embodiments of the method of the present application.

[0157] As an example, the above-described program instructions may be located and executed on one computer device, or may be located and executed on at least two computer devices at one location, or may be executed by at least two computer devices distributed across at least two locations and connected to each other by a communication network, and the at least two computer devices distributed across at least two locations and connected to each other by a communication network can constitute a blockchain network.

[0158] The computer-readable storage medium may be an internal storage unit of the multimedia data processing device provided by any one of the above-mentioned embodiments or the computer device described above, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash card provided in the computer device. The computer-readable storage medium may also include both an internal storage unit and an external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also temporarily store data that has been output or that needs to be output.

[0159] An embodiment of the present application further provides a computer program product, which includes a computer program or computer instructions, which, when executed by a processor, can realize the steps of the multimedia data processing method described above in the embodiment corresponding to Figures 7 and 10, and therefore a detailed description thereof will be omitted here. Also, a description of the advantageous effects of adopting the same method will be omitted. For technical details not disclosed in the embodiment of the computer program product of the present application, please refer to the description of the embodiment of the method of the present application.

[0160] It should be noted that the terms "first," "second," etc. in the present specification and claims and in the drawings are used to distinguish between similar objects and are not intended to limit the specific order or sequence of events. It should be understood that the terms so used may be interchanged, where appropriate, to cause the embodiments of the present invention described herein to be performed in an order other than that illustrated or described herein. Furthermore, the terms "comprise" and "have" and any variations thereof are intended to cover a non-exclusive inclusiveness; for example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the explicitly listed steps or units, but may also include other steps or units not explicitly listed or inherent in the explicitly listed process, method, product, or apparatus.

[0161] As can be understood by those skilled in the art, the units and algorithm steps in the various examples described in the embodiments disclosed herein can be realized by electronic hardware, computer software, or a combination of both. To clearly explain the interchangeability of hardware and software, the above description has already detailed the configurations and steps of each example according to their functions. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may implement the above functions in different ways for specific applications, and such implementations also fall within the scope of this application.

[0162] The methods and related devices provided by the embodiments of the present application are described based on diagrams illustrating flowcharts and / or configurations of the methods provided by the embodiments of the present application. Specifically, each flow and / or block in the flowchart and / or configuration diagrams, and combinations of flows and / or blocks in the flowchart and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a device, such that the instructions executed by the processor of the computer or other programmable data processing device implement the function(s) specified in one or more blocks in the flowchart and / or configuration diagrams. These computer program instructions may be stored in a computer-readable storage device that can direct a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable storage device generate a product including an instruction device that implements the function(s) specified in one or more blocks in the flowchart and / or configuration diagrams. These computer program instructions can be installed on a computer or other programmable data processing device and can execute a series of operational steps on the computer or other programmable device to generate a computer-implemented process, whereby the instructions executing on the computer or other programmable device can provide steps to implement the function specified in one or more flows of the flowchart and / or one or more blocks of the diagrams illustrating the configuration.

[0163] Although the preferred embodiment of the present application has been described above, the present application is not limited to this embodiment, and any modification to the present application falls within the technical scope of the present application as long as it does not depart from the spirit of the present application.

Claims

[Claim 1] 1. A method for processing multimedia data, implemented by a computing device, comprising: obtaining attribute information of a data block to be coded in a multimedia data frame; obtaining a first target reference data block matching attribute information of the data block to be coded, the first target reference data block being selected from a first reference data block set, a second reference data block set, or a third reference data block set obtained by combining the first reference data block set and the second reference data block set, the first reference data block set including coded data blocks that have no adjacent relationship with the data block to be coded, and the second reference data block set including coded data blocks that have an adjacent relationship with the data block to be coded; and A method comprising: performing predictive coding on the to-be-coded data block based on the first target reference data block.

Images