Multimedia data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a multimedia data processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a global reference data block set and a local reference data block set which are related to a to-be-coded data block in a multimedia data frame; the global reference data block comprises a coded data block which does not have an adjacent relationship with the to-be-coded data block in the multimedia data frame, and the local reference data block set comprises a coded data block which has an adjacent relationship with the to-be-coded data block in the multimedia data frame; and selecting a reference data block for performing predictive coding on the to-be-coded data block from the global reference data block set and the local reference data block set. According to the invention, the coding performance of the media data can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a multimedia data processing method, apparatus, device and storage medium. Background Technology

[0002] In multimedia data processing, intra-frame predictive coding is commonly used to predictively code multimedia data frames, which improves the coding performance. Therefore, intra-frame predictive coding has been widely applied in multimedia data processing. However, in practice, it has been found that existing intra-frame predictive coding methods suffer from poor coding performance due to the limited number of reference data blocks available for predictive coding. Summary of the Invention

[0003] The technical problem to be solved by the embodiments of this application is to provide a multimedia data processing method, apparatus, device and storage medium that can improve the encoding performance of multimedia data frames.

[0004] One embodiment of this application provides a multimedia data processing method, including: Obtain a global reference data block set and a local reference data block set associated with the data block to be encoded in a multimedia data frame; the global reference data block set includes encoded data blocks in the multimedia data frame that are not adjacent to the data block to be encoded, and the local reference data block set includes encoded data blocks in the multimedia data frame that are adjacent to the data block to be encoded. From the global reference data block set and the local reference data block set, a reference data block is selected for predictive coding of the data block to be encoded.

[0005] One embodiment of this application provides a multimedia data processing method, including: Obtain a global reference data block set and a local reference data block set associated with the data block to be decoded in the multimedia data frame; the global reference data block set includes decoded data blocks in the multimedia data frame that are not adjacent to the data block to be decoded, and the local reference data block set includes decoded data blocks in the multimedia data frame that are adjacent to the data block to be decoded. From the global reference data block set and the local reference data block set, a reference data block is selected for decoding the data block to be decoded.

[0006] One embodiment of this application provides a multimedia data processing apparatus, including: An acquisition module is used to acquire a global reference data block set and a local reference data block set associated with the data block to be encoded in a multimedia data frame; the global reference data block set includes encoded data blocks in the multimedia data frame that are not adjacent to the data block to be encoded, and the local reference data block set includes encoded data blocks in the multimedia data frame that are adjacent to the data block to be encoded. The selection module is used to select a reference data block from the global reference data block set and the local reference data block set for predictive coding of the data block to be encoded.

[0007] One embodiment of this application provides a multimedia data processing apparatus, including: An acquisition module is used to acquire a global reference data block set and a local reference data block set associated with the data block to be decoded in a multimedia data frame; the global reference data block set includes decoded data blocks in the multimedia data frame that are not adjacent to the data block to be decoded, and the local reference data block set includes decoded data blocks in the multimedia data frame that are adjacent to the data block to be decoded; The selection module is used to select a reference data block from the global reference data block set and the local reference data block set for decoding the data block to be decoded.

[0008] This application provides a computer device, including: a processor and a memory; The aforementioned memory is used to store computer programs, and the aforementioned processor is used to invoke the aforementioned computer programs to execute the steps in the aforementioned method.

[0009] One embodiment of this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, perform the steps in the method described above.

[0010] One embodiment of this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described method.

[0011] In this application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for the encoding terminal to adaptively select reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and to perform predictive encoding on the data blocks to be encoded in the multimedia data frame according to the reference data blocks; the selection range of reference data blocks is expanded, which can improve the encoding performance of multimedia data frames. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a video processing method provided in this application; Figure 2 This is a flowchart illustrating the multimedia data processing method provided in this application; Figure 3 This is a schematic diagram of an encoded data block provided in this application; Figure 4 This application provides a schematic diagram of a scenario for inter-frame coding of multimedia data frames; Figure 5 This application provides a schematic diagram of a scenario for intra-frame block copying of a multimedia data frame; Figure 6 This application provides a schematic diagram of a scenario where a reference data block of a multimedia data frame is reused in memory. Figure 7 This is a flowchart illustrating a multimedia data processing method provided in this application; Figure 8 This application provides a schematic diagram illustrating the positional relationship between a global reference data set, a local reference data block set, and a data block to be encoded; Figure 9 This application provides a schematic diagram illustrating the positional relationship between a global reference data set, a local reference data block set, and a data block to be encoded; Figure 10 This is a flowchart illustrating a multimedia data processing method provided in this application; Figure 11 This is a schematic diagram of the structure of a multimedia data processing device provided in this application; Figure 12 This is a schematic diagram of the structure of a multimedia data processing device provided in this application; Figure 13 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0015] This application relates to multimedia data processing technology. Multimedia data (or media data) refers to composite data formed by interrelated media data such as text, graphics, images, sound, animation, and moving images. In this application, multimedia data primarily includes image data composed of images or video data composed of images and sound. This application primarily uses video data as an example to describe the multimedia data processing process in detail. When the multimedia data is image data, please refer to the embodiments of this invention. The multimedia data processing process involved in this application mainly includes media data acquisition, media data encoding, media data file encapsulation, media data file transmission, media data decoding, and final data presentation. When the multimedia data is video data, the complete processing process for video data can be as follows: Figure 1 As shown, the specific processes include: video capture, video encoding, video file encapsulation, video file transfer, video file decapsulation, video decoding, and final video rendering.

[0016] Video capture is used to convert analog video into digital video and save it in the format of a digital video file. In other words, video capture can convert video signals into binary digital information. The binary information converted from the video signal is a binary data stream, which can also be called the bitstream of the video signal. Video encoding is the process of converting the original video format file into another video format file through compression technology. The generation of video media content mentioned in this application includes real-world scenes captured by cameras and computer-generated screen content scenes. From the perspective of video signal acquisition methods, video signals can be divided into two types: those captured by cameras and those generated by computers. Due to differences in statistical characteristics, their corresponding compression coding methods may also differ. Modern mainstream video coding technologies, taking international video coding standards HEVC (High Efficiency Video Coding, HEVC / H.265), VVC (versatile video coding, VVC / H.266), and AVS (Audio Video Coding Standard), or AVS3 (the third-generation video coding standard introduced by the AVS standards group), as examples, employ a hybrid coding framework. The input raw video signal undergoes a series of operations and processing, as detailed below. Figure 2 As shown: ① The image to be processed is divided into blocks, i.e., a block partition structure. The input multimedia data frame (such as a video frame in video data) is divided into several non-overlapping processing units based on its size. Each processing unit performs a similar compression operation. In one embodiment, this processing unit is called a CTU (Coding Tree Unit) or LCU (Largest Coding Unit). The CTU can be further subdivided into one or more basic coding units, called CUs (Coding Units). Each CU is the most basic element in a coding process. In another embodiment, this processing unit is also called a tile (a rectangular area of ​​a multimedia data frame that can be independently decoded and encoded). The tile can be further subdivided into one or more superblocks (SB: Superblock, the starting point of block partitioning, which can be further divided into multiple subblocks). Then, the superblocks are further subdivided into one or more data blocks (B). Each data block is the most basic element in a coding process. The following describes the various encoding methods that can be used for each data block, where the relationship between SB and B can be as follows: Figure 3 As shown.

[0017] ② Predictive Coding: This includes intra-frame prediction and inter-frame prediction. The original video signal is predicted by the selected reconstructed video signal to obtain the residual video signal. The encoder needs to determine the most suitable predictive coding mode from among many possible modes for the current coded data block (i.e., the data block to be encoded) and inform the decoder.

[0018] a. Intra (picture) Prediction: The predicted signal comes from a region within the same image that has already been encoded and reconstructed.

[0019] b. Inter-frame prediction: The predicted signal comes from other encoded images that are different from the current image (called reference images).

[0020] ③ Transform & Quantization: The residual video signal undergoes transformation operations such as DFT (Discrete Fourier Transform) and DCT (Discrete Cosine Transform, a subset of DFT) to convert the signal into the transform domain, which are called transform coefficients. The signal in the transform domain is then further subjected to lossy quantization, losing some information, making the quantized signal more suitable for compression.

[0021] In some video coding standards, there may be more than one transform option to choose from. Therefore, the encoder needs to select one of the transforms for the current encoded data block and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually results in greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually results in less distortion and a higher bitrate.

[0022] ④ Entropy Coding or Statistical Coding: The quantized transform domain signal is statistically compressed and encoded based on the frequency of each value, finally outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected mode and motion vectors, also requires entropy coding to reduce the bit rate.

[0023] Statistical coding is a lossless coding method that can effectively reduce the bit rate required to represent the same signal. Common statistical coding methods include variable length coding (VLC) or context-adaptive binary arithmetic coding (CABAC).

[0024] ⑤ Loop Filtering: The encoded image (i.e., multimedia data frame) undergoes inverse quantization, inverse transform, and prediction compensation operations (the reverse of operations ②~④ above) to obtain a reconstructed decoded image. Compared to the original image, the reconstructed image differs in some information due to the influence of quantization, resulting in distortion. Filtering the reconstructed image, such as deblocking, SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter), can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent encoded images to predict future signals, the above filtering operations are also called loop filtering, or filtering operations within the coding loop.

[0025] Figure 2 The document illustrates the basic process of a video encoder. Figure 2 The k-th CU (labeled as) Let's take k as an example, where k is a positive integer greater than or equal to 1 and less than or equal to the number of CUs in the current input image. Let x represent the pixel with coordinates [x, y] in the k-th CU, where x represents the x-coordinate of the pixel and y represents the y-coordinate of the pixel. The predicted signal is obtained after a preferred processing method, such as motion compensation or intra-frame prediction. , and Subtraction yields the residual signal Then, the residual signal After transformation and quantization, the quantized output data goes to two different places: one is sent to an entropy encoder for entropy encoding, and the encoded bitstream is output to a buffer for storage, waiting to be transmitted; the other application is to perform inverse quantization and inverse transformation to obtain the signal. The signal and The addition yields a new prediction signal. and will Save it to the buffer of the current image. f( is obtained through intra-image prediction) ), After loop filtering, the result is obtained and will It is sent to the decoded image buffer for storage, and used to generate the reconstructed video. After obtaining motion-compensation prediction , Indicates a reference block. and These represent the horizontal and vertical components of the motion vector, respectively.

[0026] As can be seen from the above encoding process, at the decoding end, for each data block, after the decoder obtains the compressed bitstream, it first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transform to obtain the residual signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to the data block can be obtained. After adding the two, the encoded data block is obtained. Finally, the encoded data block needs to undergo a loop filtering operation to produce the final encoded data block.

[0027] Currently, mainstream video coding standards such as HEVC, VVC, AVS3, AV1, and AV2 all employ block-based hybrid coding frameworks. They divide the original multimedia data frames into a series of coded data blocks, combining prediction, transform, and entropy coding methods to achieve video data compression. Motion compensation is a commonly used predictive coding method in video coding. Based on the redundancy characteristics of video content in the temporal or spatial domains, motion compensation derives the predicted value of the current coded data block from the already coded region (i.e., the reference data block). These predictive coding methods include inter-frame prediction, intra-frame block copy prediction, and intra-frame string copy prediction. In specific predictive coding implementations, these methods may be used individually or in combination. For coded data blocks using these predictive coding methods, one or more two-dimensional displacement vectors are typically explicitly or implicitly encoded in the bitstream to indicate the displacement of the current coded data block (or its sibling block) relative to one or more of its reference data blocks.

[0028] It should be noted that the displacement vector may have different names in different predictive coding modes. This application uniformly describes it as follows: 1) The displacement vector in inter-frame prediction is called the motion vector (MV); 2) The displacement vector in intra-frame block copying (i.e., intra-frame predictive coding) is called the block vector (BV); 3) The displacement vector in intra-frame string copying is called the string vector (SV). The following describes the related technologies in inter-frame prediction and intra-frame block copying prediction.

[0029] 1.1 Inter-frame prediction. For example... Figure 4As shown, inter-frame prediction utilizes the temporal correlation of multimedia data frames, using pixels from neighboring encoded images to predict pixels in the current image. This effectively removes temporal redundancy in multimedia data frames, saving bits of encoding residual data. Here, P is the current frame, Pr is the reference frame, B is the current data block to be encoded, and Br is the reference data block for B. B' and B have the same coordinate position in the image, and the coordinate of Br is (x... r ,y r The coordinates of B' are (x, y). The displacement between the current encoded data block and its reference data block is called the motion vector (MV), which can be represented by the following formula (1): MV=(x r -x,y r -y)(1) Considering the strong correlation between neighboring blocks in the temporal or spatial domains, MV prediction techniques can be used to further reduce the bits required to encode MVs. In H.265 / HEVC, inter-frame prediction includes two MV prediction techniques: Merge and AMVP.

[0030] 2.1 Intra Block Copy Prediction. Intra Block Copy (IBC) is an intra-frame coding tool adopted in the HEVC Screen Content Coding (SCC) extension, which significantly improves the coding efficiency of screen content. AVS3, VVC, and AV1 also employ IBC technology to enhance the performance of screen content coding. IBC utilizes the spatial correlation of screen content video, using the pixels of already encoded data blocks in multimedia data frames to predict the pixels of the data block to be encoded, effectively saving the bits required for encoding pixels. Figure 5 A schematic diagram of IBC technology, such as Figure 5 In a multimedia data frame, the data block to be encoded (i.e., the current block) and the reference data block (i.e., the reference block) are both located in the same multimedia data frame. The reference data block is located in the encoded region, and the data block to be encoded is located in the uncoded region. The displacement between the data block to be encoded and its reference data block is called the block vector (BV).

[0031] In the HEVC SCC extension, the IBC mode employs an implementation similar to inter-frame prediction. If the multimedia data frame allows the use of IBC mode, the currently decoded but unfiltered image is used as a special reference frame. The predicted values ​​of the data blocks to be encoded are derived from this reference frame using a motion compensation method similar to inter-frame prediction. This implementation has the following characteristics: 1) BV uses only integer precision; 2) A global reference data block set (i.e., a global reference range) is adopted, which means that all reconstructed parts of the current multimedia data frame can be used as reference data blocks, except for certain areas which are prohibited from being used as reference data blocks due to considerations of parallel processing and chroma interpolation. 3) In the decoded image buffer, the currently decoded but unfiltered image will occupy a buffer of one image size; 4) The currently decoded but unfiltered image will be placed at the end of the reference image list.

[0032] In hardware implementation, the computing core can read and write data from either on-chip memory or off-chip memory. On-chip memory is typically static memory, offering very fast read and write speeds, but due to its relatively high cost, only a small amount of on-chip memory is usually used. Off-chip memory, on the other hand, is typically much larger, but efficient access requires adherence to certain rules, often resulting in a writing-back delay when writing recently encoded data blocks back to off-chip image memory. In a typical hardware implementation of motion compensation, the currently decoded image is written to off-chip memory after completing the loop filtering operation for future reference. When motion compensation for subsequent encoded data blocks requires reference to the encoded data blocks located in off-chip memory, the relevant data for that reference data block needs to be read from off-chip memory. This process involves memory bandwidth consumption for both writing and reading. While the IBC scheme in HEVC-SCC has a relatively small impact on existing software designs in software implementation, it presents the following two problems in hardware implementation: a. The reference image for the IBC is a version without loop filtering, which differs from the final decoded image output. Outputting this image off-chip will increase memory bandwidth.

[0033] b. A currently decoded or recently reconstructed adjacent data block may be used as a reference for the next encoded data block. This requires completing the decoding of the reconstructed data block, outputting it to off-chip memory, and then reading it into on-chip memory within a limited clock cycle. This poses a significant challenge to hardware implementation.

[0034] To address the hardware implementation issues of IBC, subsequent standards such as VVC, AVS3, and AV1 adopted several IBC-related optimization schemes. The IBC schemes in VVC and AVS3 standards use a local reference data block set, that is, using the already encoded data block adjacent to the current encoded data block as the reference data block for the current encoded data block. In specific hardware implementations, reference data blocks from the local reference data block set stored in on-chip memory can be used to solve the aforementioned problems. Considering hardware implementation cost and encoding performance, IBC uses only one CTU-sized memory to store reference data blocks from the local reference data block set, and employs methods such as... Figure 6The reference memory reuse strategy is shown. Besides storing the currently reconstructed 64x64 CU, there are three other 64x64 CUs available to store reconstructed data blocks. Therefore, IBC can only search for reference data blocks in these three 64x64 CUs and the reconstructed portion of the current 64x64 CU.

[0035] In summary, the IBC scheme in the AV1 standard uses a global reference block set, meaning the current encoded data block is allowed to reference already encoded data blocks in the current multimedia data frame. However, this scheme has two drawbacks: a) to meet write-back constraints, adjacent already encoded data blocks are not allowed to be used as reference blocks; b) if the current multimedia data frame allows IBC, the loop filter will be disabled. However, screen content sequences often have strong spatial correlation, and adjacent data blocks have high reference value. Allowing adjacent already encoded data blocks to be referenced can improve coding performance. Disabling the loop filter in some sequence content will also lead to a significant degradation in multimedia data quality and affect coding performance. Although the IBC scheme based on a local reference block set does not have the above drawbacks, the limited number of usable reference blocks in the local reference block set may degrade the coding performance of the multimedia data frame. Depending on the characteristics of the sequence content, the global reference block set, the local reference block set, and loop filtering processing have different proportions of influence on coding performance. Therefore, addressing the shortcomings of IBC schemes based on global and local reference data block sets, this application selects a suitable predictive coding method based on the attribute information of the multimedia data frame, which is beneficial to improving coding performance. In other words, this application provides both global and local reference data block sets, which allows the coding terminal to adaptively select reference data blocks from these sets based on the attribute information of the current coding data block. This expands the range of selectable reference data blocks and solves the problem of a limited number of reference data blocks. Furthermore, predictive coding of the current coding data block based on the selected reference data block can improve the coding performance of the multimedia data frame.

[0036] It should be noted that this application can be executed by a terminal, a server, or both. The device used to execute this method in this application can be collectively referred to as a computer device. The server can be a single physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal can be a smart vehicle terminal, smart camera, smartphone, tablet, laptop, desktop computer, smart speaker, speaker with screen, smartwatch, smart TV, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication. The number of terminals and servers can be one or at least two; this application does not impose any restrictions.

[0037] It should be noted that the embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. For example, in a smart transportation scenario, multimedia data frames are video frames or image frames obtained by a camera device capturing images of the road. This camera device can refer to an in-vehicle terminal or a roadside camera device. By performing predictive encoding on multimedia data frames through this application, the encoding performance of multimedia data frames can be improved. The encoded multimedia data frames can then be used to observe road conditions in real time, thereby improving vehicle driving safety.

[0038] Further, please see Figure 7 This is a flowchart illustrating a multimedia data processing method provided in an embodiment of this application. Figure 7 As shown, the method may include at least the following S101-S102: S101. Obtain a global reference data block set and a local reference data block set associated with the data block to be encoded in the multimedia data frame; the global reference data block set includes encoded data blocks in the multimedia data frame that are not adjacent to the data block to be encoded, and the local reference data block set includes encoded data blocks in the multimedia data frame that are adjacent to the data block to be encoded.

[0039] In this application, the computer device can acquire multimedia data frames; these multimedia data frames can be those captured by the computer device or downloaded by the computer device from a network. The multimedia data frame refers to a video frame to be encoded in video data or an image frame to be encoded in image data. Video data can refer to live video data, film and television video data, daily life video data, road detection video data, etc.; image data can refer to images of people, buildings, objects, etc. Furthermore, the computer device can acquire a global reference data block set associated with the multimedia data frame based on at least one of the following: encoding method, write-back delay, correlation between encoded data blocks and data blocks to be encoded; and can acquire both a global reference data block set and a local reference data block set associated with the multimedia data frame based on at least one of the following: storage space size for storing local reference data block sets, correlation between encoded data blocks and data blocks to be encoded.

[0040] It should be noted that the global reference data block set and the local reference data block set are stored in the first storage space and the second storage space, respectively. The first storage space and the second storage space can be located in the same computer device or in different computer devices. Specifically, the first storage space and the second storage space can refer to a disk, or a storage media array, etc., and this application does not limit this. In particular, when the first storage space and the second storage space are located in the same computer device, the first storage space can refer to the off-chip memory of the computer device, and the second storage space can refer to the on-chip memory of the computer device; or, the first storage space can refer to the on-chip memory of the computer device, and the second storage space can refer to the off-chip memory of the computer device; this application does not limit this.

[0041] S102. Select reference data blocks from the local reference data block set and the global reference data block set for predictive coding of the data block to be coded.

[0042] In this application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for adaptively selecting reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and performing predictive coding on the data block to be encoded in the multimedia data frame according to the reference data blocks; it expands the selection range of reference data blocks and can improve the coding performance of multimedia data frames.

[0043] For example, a computer device can acquire attribute information of a data block to be encoded. Based on this attribute information, it selects reference data blocks from the local reference data block set and the global reference data block set for predictive coding of the data block to be encoded. The attribute information of the data block to be encoded includes one or more of the following: the media type of the multimedia data frame to which the data block belongs, the reference flag of the multimedia data frame, and the resolution of the multimedia data frame to which the data block belongs. The media type includes intra-frame and inter-frame types. Intra-frame types can refer to representative multimedia data frames, such as fully intra-coded frames, Intra frames, I-frames, and keyframes. Frames of this type are only allowed to use intra-frame coding techniques, including intra-block copy mode or intra-string copy mode, and do not rely on other frames for encoding. For example, an intra-frame type can refer to a multimedia data frame that embodies the main meaning of video or image data; an inter-frame type can refer to multimedia data frames in video or image data other than intra-frame types, such as inter-coded frames, P-frames, B-frames, and non-keyframes. This type of frame allows the use of inter-frame coding and intra-frame coding techniques. Typically, when using inter-frame coding, encoding can only begin after the reference frame has been encoded. The reference flag in a multimedia data frame reflects the range of reference data blocks selected for the data block to be encoded (i.e., which set of reference data blocks is chosen).

[0044] In this application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for the encoding terminal to adaptively select reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and to perform predictive encoding on the data blocks to be encoded in the multimedia data frame according to the reference data blocks; the selection range of reference data blocks is expanded, which can improve the encoding performance of multimedia data frames.

[0045] Optionally, the computer device may select one of the following two methods to obtain the global reference data block set based on the encoding method of the multimedia data frame: Method 1: The global reference data block set includes one or more first reference data blocks. When the encoding method of the multimedia data frame is serial encoding, the first reference data block in the global reference data block set is an encoded data block in the multimedia data frame whose parameter association relationship satisfies the write-back constraint and belongs to the same coding slice as the data block to be encoded. The parameter association relationship is used to reflect the association between the first position information, the size information of the largest coding block, and the second position information. The first position information and the second position information refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame, respectively. The write-back constraint is used to reflect the time limit for writing the encoded data block to the global reference data block set. That is to say, serial encoding means that only one data block in the multimedia data frame is predicted and encoded at the same time. If the encoding method of the multimedia data frame is serial encoding, this encoding method will not affect the determination process of the reference data block. However, writing encoded data blocks to the global reference data block set introduces significant write-back latency, meaning it takes a considerable amount of time. Therefore, encoded data blocks must wait at least a preset duration before they can be used as reference data blocks. This preset duration is determined based on a time limit for writing the encoded data blocks to the global reference data block set. This time limit can refer to one of the average latency, maximum latency, or minimum latency for writing the encoded data blocks to the global reference data block set. Thus, when the multimedia data frame is encoded using serial encoding, the first reference data block in the global reference data block set must meet the following two conditions: a) it belongs to the same coded segment as the data block to be encoded; b) its parameter association satisfies the write-back constraint.

[0046] Specifically, the computer device can perform the following steps s11~s14 to obtain the global reference data block set: s11. When the encoding method of the multimedia data frame is serial encoding, the first position information, the size information of the largest encoded block, and the second position information are obtained respectively; the first position information and the second position information refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame, respectively.

[0047] s12. Obtain write-back constraints; these write-back constraints reflect the time limit for writing the encoded data block to the global reference data block set.

[0048] s13. Select from the multimedia data frame an encoded data block whose parameter association relationship satisfies the write-back constraint and belongs to the same coding slice as the data block to be encoded, and use it as the first reference data block; the parameter association relationship is used to reflect the relationship between the first position information, the size information of the largest coding block and the second position information.

[0049] s14. Add the first reference data block to the global reference data block set.

[0050] In steps s11-s14, when the multimedia data frame is encoded using serial encoding, the computer device can acquire first position information, the size information of the maximum encoding block, and second position information. The first and second position information refer to the position information of the encoded data block and the data block to be encoded within the multimedia data frame, respectively. More specifically, the first and second position information can refer to the coordinates of the pixels of the encoded data block and the pixels of the data block to be encoded within the multimedia data frame. The size information of the maximum encoding block can refer to the width and height of the maximum encoding block. Further, the computer device can acquire write-back constraints; it can filter out encoded data blocks from the multimedia data frame whose parameter relationships satisfy the write-back constraints and which belong to the same encoding slice as the data block to be encoded, and use these as first reference data blocks, adding them to the global reference data block set. By filtering the first reference data block according to the write-back constraints, the first position information, the second position information, and the size information of the maximum encoding block, it can ensure that the first reference data block has a strong spatial correlation with the data block to be encoded and ensure the availability of the first reference data block.

[0051] Optionally, step s13 above may include the following steps s21~s24: s21. Based on the first location information and the second location information, select encoded data blocks belonging to the same coded slice from the multimedia data frame as first candidate reference data blocks.

[0052] s22. Determine the number of data blocks that are spaced between the encoded data blocks in the multimedia data frame and the data blocks to be encoded under the write-back constraint conditions.

[0053] s23. Based on the first position information, the size information of the largest coding block, and the second position information, count the number of interval data blocks between the first candidate reference data block and the data block to be encoded.

[0054] s24. The first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks is determined as the first reference data block.

[0055] In steps s21-s24, assuming the first and second position information can refer to the coordinates of the pixels of the encoded data block and the pixels of the data block to be encoded within the multimedia data frame, and the position information of the encoded data block is (ref_x, ref_y), and the position information of the data block to be encoded is (cur_x, cur_y), the computer device can filter out encoded data blocks belonging to the same coding slice as the data block to be encoded from the multimedia data frame as first candidate reference data blocks. Further, under the write-back constraint, the number of data block intervals between the encoded data blocks in the multimedia data frame and the data block to be encoded is determined. This number of data block intervals can refer to the minimum number of data blocks that should be spaced between the encoded data blocks in the multimedia data frame and the data block to be encoded under the write-back constraint. Then, based on the first location information, the size information of the largest coded block, and the second location information, the number of interval data blocks between the first candidate reference data block and the data block to be encoded is counted. This number of interval data blocks refers to the actual number of data blocks between the first candidate reference data block and the data block to be encoded. The first candidate reference data block whose number of interval data blocks is greater than the limit is determined as the first reference data block. By obtaining the location information of the encoded data blocks, the location information of the data block to be encoded, and the write-back latency limit, a global reference data block set can be obtained, ensuring a strong spatial correlation between the first reference data block and the data block to be encoded, and ensuring the availability of the first reference data block. Simultaneously, no additional hardware costs are required, saving costs.

[0056] For example, a computer device may use a first candidate reference data block that satisfies the following formula (2) as the first reference data block: (2) In formula (2) The number of interval data blocks between the first candidate reference data block and the data block to be encoded is indicated by cur_num and src_num, which are respectively the number of the first candidate reference data block and the number of the data block to be encoded. D is the limit number of the interval data blocks that meet the write-back constraint. cur_num and src_num can be calculated by formulas (3) and (4) below.

[0057] Optionally, step s23 above may include the following steps s31~s34: s31. Determine the row identifier and column identifier of the first candidate reference data block based on the first position information and the size information of the largest coding block.

[0058] s32. Determine the row identifier and column identifier of the data block to be encoded based on the second position information and the size information of the largest encoding block.

[0059] s33. Determine the number of data blocks in each line of the multimedia data frame based on the size information of the data block to be encoded.

[0060] s34. Based on the number of data blocks in each row, the row identifier and column identifier of the first candidate reference data block, and the row identifier and column identifier of the data block to be encoded, count the number of interval data blocks between the first candidate reference data block and the data block to be encoded.

[0061] In steps s31~s34, the row identifier and column identifier of the first candidate reference data block are ref_y_num and ref_x_num, respectively, and the row identifier and column identifier of the data block to be encoded are cur_x_num and cur_y_num, respectively. The computer device can obtain the row identifier and column identifier of the first candidate reference data block and the row identifier and column identifier of the data block to be encoded by the following formula (3): (3) In formula (3), Floor() represents rounding down. W and H can be determined using any of the following four methods: 1. W and H are determined based on the size information of the maximum coding block; 2. W and H are determined based on the size information of VPDU (Virtual Pipeline Data Units, basic units of on-chip memory); 3. H can be determined by the size information of the maximum coding block, such as 128x128 for the maximum coding block and H being 128, or 64x64 for the maximum coding block and H being 64. W can be a default value, such as 64. 4. W and H are both preset values, which can be 4, 64, 128, etc. A maximum coding block can include one or more data blocks. The size of the maximum coding block and the size of the data blocks are specifically determined according to the segmentation method of the multimedia data frame.

[0062] Then, the computer device can determine the number of data blocks in each row of the multimedia data frame based on the size information of the data block to be encoded; determine the number of the first candidate reference data block according to the number of data blocks in each row, the row identifier and column identifier of the first candidate reference data block, and determine the number of the data block to be encoded according to the number of data blocks in each row and the row identifier and column identifier of the data block to be encoded; and determine the difference between the number of the first candidate reference data block and the number of the data block to be encoded as the number of interval data blocks between the first candidate reference data block and the data block to be encoded. For example, the computer device can use the following formula (4) to determine the number of data blocks in each row of the multimedia data frame, the number of the first candidate reference data block, and the number of the data block to be encoded: (4) In formula (4), stride refers to the number of data blocks in each row of the multimedia data frame, cur_num and src_num refer to the numbers of the first candidate reference data block and the data block to be encoded, respectively, Ceil() means rounding up, and tile_w represents the width of the encoded slice. The encoded slice includes one or more maximum encoded blocks. Therefore, the width of the encoded slice can be determined based on the size information of the maximum encoded block. For example, when the size of the maximum encoded block is 128x128, the encoded slice can be 1024x720. For example, the first location information and the second location information are the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively; when the encoding method of the multimedia data frame is serial encoding, the size of the maximum encoding block is 128x128, the width and height of the encoding slice are 1024 and 720 respectively, the values ​​of W and H are both equal to 64, and the number of interval data blocks between the encoded data block and the data block to be encoded is limited to 4 under the write-back constraint, the first reference data block in the global reference data block set is the encoded data block that belongs to the same encoding slice as the data block to be encoded and whose coordinates satisfy the following formula (5). The number of interval data blocks can refer to the minimum number of data blocks that should be spaced between the encoded data block and the data block to be encoded in the multimedia data frame under the write-back constraint.

[0063] (5) In formula (5), (ref_x, ref_y) and (cur_x, cur_y) represent the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively, and Floor() represents rounding down.

[0064] For example, the first location information and the second location information are the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively; when the encoding method of the multimedia data frame is serial encoding, the size of the maximum encoding block is 64x64, the width and height of the encoding slice are 1024 and 720 respectively, the values ​​of W and H are both equal to 64, and the number of interval data blocks between the encoded data block and the data block to be encoded is limited to 4 under the write-back restriction condition, the first reference data block in the global reference data block set is the encoded data block that belongs to the same encoding slice as the data block to be encoded and whose coordinates satisfy the following formula (6).

[0065] (6) Where (ref_x, ref_y) and (cur_x, cur_y) represent the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively, and Floor() represents rounding down.

[0066] It should be noted that when the encoding method of the multimedia data frame is serial encoding, the computer device can also obtain the global reference data block set in the following way: Based on the first position information, the second position information, and the size information of the largest coded block, determine the interval distance between the encoded data block and the data block to be encoded, and determine the interval limit distance between the encoded data block and the data block to be encoded under the write-back delay condition. The interval limit distance refers to the minimum interval distance between the encoded data block and the data block to be encoded under the write-back delay condition. The encoded data block in the multimedia data frame that belongs to the same coded segment as the data block to be encoded and whose interval distance is greater than the interval limit distance is taken as the first reference data block and added to the global reference data block set.

[0067] Optionally, having an adjacency relationship between the data block to be encoded and the encoded data block can mean that the encoding order of the data block to be encoded is adjacent to the encoding order of the encoded data block; not having an adjacency relationship between the data block to be encoded and the encoded data block can mean that the encoding order of the data block to be encoded is not adjacent to the encoding order of the encoded data block.

[0068] Method 2: The global reference data block set includes one or more second reference data blocks. When the multimedia data frame is encoded using parallel encoding or serial encoding, the second reference data blocks in the global reference data block set are encoded data blocks that belong to the same coding slice as the data block to be encoded, whose parameter association in the multimedia data frame satisfies the write-back constraint, and whose parameter association satisfies the encoding processing condition. The parameter association reflects the relationship between the position information of the encoded data block and the data block to be encoded in the multimedia data frame, and the size information of the largest coding block. The write-back constraint reflects the time limit for writing the encoded data block to the global reference data block set. In other words, parallel encoding refers to predictively encoding multiple data blocks in the multimedia data frame at the same time. If the multimedia data frame is encoded using parallel encoding, this encoding method will affect the determination process of the reference data blocks. Meanwhile, writing encoded data blocks to the global reference data block set introduces significant write-back latency. Therefore, when the multimedia data frame is encoded using parallel encoding, the second reference data block in the global reference data block set must meet the following three conditions: a) it belongs to the same coding slice as the data block to be encoded; b) the parameter association satisfies the write-back constraint; c) the parameter association satisfies the encoding processing conditions. Specifically, when the multimedia data frame is encoded using serial encoding, method two can also be used to obtain the global reference data block set. Here, the encoding processing conditions can refer to the parallel encoding processing conditions.

[0069] Specifically, the computer device can perform the following steps s41~s44 to obtain the global reference data block set: s41. Obtain the first position information, the size information of the largest encoded block, and the second position information respectively; the first position information and the second position information refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame respectively.

[0070] s42. Obtain write-back constraints and encoding processing conditions; the write-back constraints are used to reflect the time limit for writing the encoded data block to the global reference data block set.

[0071] s43. Select encoded data blocks from the multimedia data frames whose parameter association relationships satisfy the write-back restriction conditions and the encoding processing conditions, and which belong to the same encoding slice as the data block to be encoded, as the second reference data blocks; the parameter association relationship is used to reflect the association relationship between the first position information, the size information of the data block to be encoded, and the second position information.

[0072] s44. Add the second reference data block to the global reference data block set.

[0073] In steps s41-s44, the first position information and the second position information can refer to the coordinates of the pixels of the encoded data block and the pixels of the data block to be encoded in the multimedia data frame. The size information of the maximum encoded block can refer to the width and height of the maximum encoded block. The write-back constraint is used to reflect the time limit for writing the encoded data block to the global reference data block set, and the encoding processing condition is used to reflect the distance limit between the reference data block and the data block to be encoded when encoding the multimedia data frame in parallel; or, the encoding processing condition is used to reflect the number of interval data blocks between the reference data block and the data block to be encoded when encoding the multimedia data frame in parallel. By selecting encoded data blocks from the multimedia data frame whose parameter correlation satisfies the write-back constraint and the encoding processing condition, and which belong to the same coding slice as the data block to be encoded, as the second reference data block, and adding the second reference data block to the global reference data block set, it can be ensured that the second reference data block and the data block to be encoded have a strong spatial correlation, and the availability of the second reference data block can be ensured.

[0074] Optionally, step s43 above may include the following steps s51~s55: s51. Based on the first location information and the second location information, select encoded data blocks from the multimedia data frames that belong to the same coding slice as the data block to be encoded, and use them as second candidate reference data blocks.

[0075] s52. Select the first candidate reference data block from the first candidate reference data block whose parameter association relationship meets the write-back restriction condition, and use it as the second candidate reference data block.

[0076] s53. Determine the first data block limit distance between the encoded data block in the multimedia data frame under the write-back constraint and the data block to be encoded.

[0077] s54. Determine the second data block limitation distance between the encoded data block in the multimedia data frame under the encoding processing conditions and the data block to be encoded.

[0078] s55. Based on the first position information, the size information of the maximum coding block and the second position information, the first data block limit distance and the second data block limit distance, determine a second candidate reference data block that satisfies the coding processing conditions from the second candidate reference data blocks, and use it as the second reference data block.

[0079] In steps s51-s55, assuming the first position information and the second position information can refer to the coordinates of the pixels of the encoded data block and the pixels of the data block to be encoded in the multimedia data frame, the position information of the encoded data block is (ref_x, ref_y), and the position information of the data block to be encoded is (cur_x, cur_y). The computer device can filter out the encoded data block that belongs to the same coding slice as the data block to be encoded from the multimedia data frame as the first candidate reference data block. Further, the computer device can filter out the first candidate reference data block whose coordinates satisfy the above formula (4) as the second candidate reference data block. Furthermore, a first data block constraint distance can be determined between the encoded data block in the multimedia data frame under the write-back constraint condition and the data block to be encoded. This first data block constraint distance can refer to the minimum distance that should be maintained between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back delay constraint condition. A second data block constraint distance can also be determined between the encoded data block in the multimedia data frame and the data block to be encoded under the encoding processing condition. This second data block constraint distance refers to the minimum distance that should be maintained between the encoded data block in the multimedia data frame and the data block to be encoded under the encoding processing condition. Then, based on the first position information, the size information of the maximum encoded block, the second position information, the first data block constraint distance, and the second data block constraint distance, a second candidate reference data block that meets the encoding processing condition can be determined from the second candidate reference data blocks and used as the second reference data block.

[0080] Optionally, step s55 above may include the following steps s61~s66: s61. Determine the row identifier and column identifier of the second candidate reference data block based on the first position information and the size information of the largest coding block.

[0081] s62. Determine the row identifier and column identifier of the data block to be encoded based on the second position information and the size information of the largest encoded block.

[0082] s63. Select a second candidate reference data block from the second candidate reference data block whose line identifier is less than or equal to the line identifier of the data block to be encoded, and use it as the third candidate reference data block.

[0083] s64. Determine the row distance between the third candidate reference data block and the data block to be encoded based on the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded.

[0084] s65. Determine the column distance between the third candidate reference data block and the data block to be encoded based on the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded.

[0085] s66. The third candidate reference data block whose column distance is less than the first distance threshold is determined as the second reference data block; the first distance threshold is determined based on the first data block limit distance, the second data block limit distance, and the row distance between the third candidate reference data block and the data block to be encoded.

[0086] In steps s61 to s66, the computer device can obtain the row and column identifiers of the second candidate reference data block and the row and column identifiers of the data block to be encoded using the formula (5) above. Then, it can select a second candidate reference data block from the second candidate reference data block whose row identifier is less than or equal to the row identifier of the data block to be encoded, and use it as the third candidate reference data block. Further, the difference between the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded can be determined as the row distance between the third candidate reference data block and the data block to be encoded, and the difference between the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded can be determined as the column distance between the third candidate reference data block and the data block to be encoded; and the third candidate reference data block whose column distance is less than the first distance threshold can be determined as the second reference data block. For example, the computer device can determine the second candidate reference data block whose coordinates satisfy the following formula (7) as the second reference data block: (7) In formula (7), ref_y_num and ref_x_num refer to the row and column identifiers of the second candidate reference data block, respectively; cur_y_num and cur_x_num refer to the row and column identifiers of the data block to be encoded, respectively; P refers to the first data block restriction distance between the encoded data block in the multimedia data frame under the write-back restriction condition and the data block to be encoded; and E refers to the second data block restriction distance between the encoded data block in the multimedia data frame under the encoding processing condition and the data block to be encoded. Specifically, the first data block restriction distance can be determined based on the number of interval restriction data blocks between the encoded data block in the multimedia data frame under the write-back restriction condition and the data block to be encoded. For example, P and D can be numerically the same. The second data block restriction distance is determined based on the number of interval restriction data blocks between the encoded data block in the multimedia data frame under the encoding processing condition and the data block to be encoded. This represents the column distance between the third candidate reference data block and the data block to be encoded. This represents the line distance between the third candidate reference data block and the data block to be encoded.

[0087] For example, the first location information and the second location information are the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively; when the encoding method of the multimedia data frame is parallel encoding or serial encoding, the size of the maximum encoding block is 128x128, the width and height of the encoding slice are 1024 and 720 respectively, the values ​​of W and H are both equal to 64, and the interval data block restriction distance between the encoded data block and the data block to be encoded is 4 under the write-back restriction condition, and the interval data block restriction distance between the encoded data block and the data block to be encoded is 2 under the encoding processing condition, the second reference data block in the global reference data block set is the encoded data block that belongs to the same encoding slice as the data block to be encoded and whose coordinates satisfy the following formula (8): (8) Where (ref_x, ref_y) and (cur_x, cur_y) represent the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively, and Floor() represents rounding down.

[0088] For example, the first location information and the second location information are the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively; when the encoding method of the multimedia data frame is parallel encoding or serial encoding, the size of the maximum encoding block is 64x64, the width and height of the encoding slice are 1024 and 720 respectively, the values ​​of W and H are both equal to 64, and the interval data block limit distance between the encoded data block and the data block to be encoded is 4 under the write-back restriction condition, and the interval data block limit distance between the encoded data block and the data block to be encoded is 1 under the encoding processing condition, the second reference data block in the global reference data block set is the encoded data block that belongs to the same encoding slice as the data block to be encoded and whose coordinates satisfy the following formula (9): (9) Where (ref_x, ref_y) and (cur_x, cur_y) represent the coordinates of the encoded data block and the data block to be encoded in the multimedia data frame, respectively, and Floor() represents rounding down.

[0089] It should be noted that when the multimedia data frame is encoded in parallel or serial encoding, the computer device can also obtain the global reference data block set in the following way: Based on the first position information, the second position information, and the size information of the largest coded block, determine the number of interval data blocks between the encoded data block and the data block to be encoded, and determine the target interval limit data block number between the encoded data block and the data block to be encoded under write-back delay conditions and encoding processing conditions. The target interval limit data block number refers to the minimum number of interval data blocks between the encoded data block and the data block to be encoded under the write-back delay conditions and encoding processing conditions. The encoded data block in the multimedia data frame that belongs to the same coded segment as the data block to be encoded and whose number of interval data blocks is greater than the target interval limit data block number is taken as the second reference data block and added to the global reference data block set.

[0090] Optionally, the computer device may select one or at least a combination of the following two methods to obtain the local reference data block set: Method 1: The local reference data block set includes one or more third reference data blocks. The third reference data block in this local reference data block set is the encoded data block in the multimedia data frame that is located in the same encoding processing unit as the data block to be encoded. This encoding processing unit can also be called a Virtual Pipeline Data Unit (VPDU). The size information of the VPDU can be determined according to the encoding processing capability of the encoder. For example, the size of the VPDU is VPDU_WxVPDU_H, and the values ​​of VPDU_W and VPDU_H can be 4, 8, 16, 32, 64, 128, etc. That is, the encoded data block in the multimedia data frame whose coordinates satisfy ref_x / VPDU_W = cur_x / VPDU_W and ref_y / VPDU_H = cur_y / VPDU_H is used as the second reference data block. For example, if the size of the VPDU is 64x64, then all the encoded data blocks in the 64x64 VPDU to which the data block to be encoded belongs can be used as reference data blocks for the data block to be encoded. Alternatively, if the VPDU size is 64x64, the encoded data block in the multimedia data frame whose coordinates satisfy Floor(ref_x / 64) = Floor(cur_x / 64) and Floor(ref_y / 64) = Floor(cur_y / 64) is used as the second reference data block.

[0091] Method 2: The local reference data block set includes one or more fourth reference data blocks; the fourth reference data block in the local reference data block set is an encoded data block in the multimedia data frame that is adjacent to the data block to be encoded and belongs to the same coding slice as the data block to be encoded. The adjacency relationship between the fourth reference data block and the data block to be encoded can include one or more combinations of the following three cases: a. The adjacency relationship between the fourth reference data block and the data block to be encoded means that the fourth reference data block and the data block to be encoded belong to the same maximum coding block.

[0092] b. The adjacency relationship between the fourth reference data block and the data block to be encoded means that the fourth reference data block is located within the N largest coding blocks adjacent to the data block to be encoded, and the size of the largest coding block is less than the size threshold; N is determined based on the size information of the largest coding block. For example, the encoding direction of the multimedia data frame is from left to right, and the size threshold can be dynamically set according to the application scenario or user needs. For example, the size threshold can be 128x128. Then, the adjacency relationship between the fourth reference data block and the data block to be encoded means that the fourth reference data block is located within the N largest coding blocks to the left of the data block to be encoded, and the size of the largest coding block is less than 128x128. N can be determined according to the following formula (10): (10) Here, log2_sb_size refers to an intermediate variable, and sb_size refers to the size of the maximum coding block.

[0093] c. The adjacency relationship between the fourth reference data block and the data block to be encoded means that: the fourth reference data block is located within N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is greater than or equal to a size threshold; the data block located in the upper left corner of the region where the fourth reference data block is located after moving one maximum coding block to the right in the multimedia data frame is an unreconstructed data block; the position information of the fourth reference data block when it moves (moving in the same direction as the encoding direction) the distance of one maximum coding block is different from the position information of the data block to be encoded; N is determined based on the size information of the maximum coding block. In other words, when the size of the maximum coding block is greater than or equal to the size threshold, such as when the size of the maximum coding block is 128x128, the fourth reference data block needs to meet the following three conditions: 1. It is an encoded data block belonging to the same coding slice as the data block to be encoded; 2. After the fourth reference data block moves 128 pixels to the right, its coordinates are (ref_x + 128, ref_y), then the data block with coordinates (ref_x + 128, ref_y) in the multimedia data frame is an unreconstructed data block; 3. The position information of the fourth reference data block when it moves to the right by the distance of one maximum coding block is different from the position information of the data block to be encoded. This condition can be expressed by formula (11): (11) In formula (11), ! = means not equal to.

[0094] Optionally, the distances between the first reference data block, the second reference data block, the third reference data block, and the fourth reference data block and the data block to be encoded are all less than a second distance threshold; the second distance threshold can be dynamically set according to the application scenario or user needs, such as the second distance threshold being 2. 14 Meanwhile, the first reference data block, the second reference data block, the third reference data block, and the fourth reference data block are located at integer pixel positions in the multimedia data frame.

[0095] For example, when the multimedia data frame is encoded using parallel encoding, the maximum code block size is 128x128, the interval between the encoded data block and the data block to be encoded is limited to 4 blocks under write-back delay constraints, and the interval between the encoded data block and the data block to be encoded is limited to 2 blocks under encoding processing conditions, and the local reference data block set is obtained using the first method described above, with a VPDU size of 64x64, the relationship between the global reference data block set, the local reference data block set, and the data block to be encoded is as follows: Figure 8 As shown. Figure 8The smallest square (the one drawn with dashed lines) has a size of 64x64, the smallest square drawn with solid lines has a size of 128x128, and the size of the data block is 32x32. Therefore, Figure 8 In a multimedia data frame, the encoded data block located in the 19th cell of the first row belongs to a local reference data block set, which is used to encode the data block to be encoded in the 19th cell of the first row. The encoded data blocks in the first and second rows of the multimedia data frame, located 8 data blocks away from the data block to be encoded, belong to a global reference data block set, which is also used to encode the data block to be encoded in the 19th cell of the first row. Figure 8 It can be seen that the number of reference data blocks in the global reference data block set is greater than the number of reference data blocks in the local reference data block set. The reference data blocks in the global reference data block set are not adjacent to the data blocks to be encoded, while the data blocks in the local reference data block set are adjacent to the data blocks to be encoded.

[0096] For example, when the multimedia data frame is encoded using parallel encoding, the maximum code block size is 128x128, the interval between the encoded data block and the data block to be encoded is limited to 4 blocks under write-back delay constraints, and the interval between the encoded data block and the data block to be encoded is limited to 2 blocks under encoding processing conditions. The local reference data block set is obtained using the second method described above. When the VPDU size is 64x64, the relationship between the global reference data block set, the local reference data block set, and the data block to be encoded is as follows: Figure 9 As shown. Figure 9 The smallest square (the one drawn with dashed lines) has a size of 64x64, the smallest square drawn with solid lines has a size of 128x128, and the size of the data block is 32x32. Therefore, Figure 9 In a multimedia data frame, the encoded data block located in the 19th cell of the first row, and the encoded data block in the largest encoded block to the left of the data block to be encoded, belong to the local reference data block set. This local reference data block set is used to encode the data block to be encoded in the 19th cell of the first row. The encoded data blocks in the first and second rows of the multimedia data frame, located 8 data blocks away from the data block to be encoded, belong to the global reference data block set. This global reference data block set is used to encode the data block to be encoded in the 19th cell of the first row. Figure 9 It can be seen that the number of reference data blocks in the global reference data block set is greater than the number of reference data blocks in the local reference data block set. The reference data blocks in the global reference data block set are not adjacent to the data blocks to be encoded, while the data blocks in the local reference data block set are adjacent to the data blocks to be encoded.

[0097] Optionally, for any multimedia data frame, the computer device can select reference data blocks from a global reference data block set and a local reference data block set for encoding the data block to be encoded. Alternatively, the computer device can adaptively select a set of reference data blocks based on the encoding mode of the data block to be encoded, and determine the reference data blocks for encoding the data block to be encoded from the selected set. Specifically, the computer device can obtain the encoding mode of the data block to be encoded; if the encoding mode of the data block to be encoded is an intra-block copy mode or an intra-string copy mode, then a reference data block for predictive encoding of the data block to be encoded is selected from the global reference data block set and the local reference data block set. The encoding mode of the data block to be encoded can be determined based on the media type of the multimedia data frame, or it can be determined based on the encoding flag (or reference flag) of the multimedia data frame. In particular, the encoding mode of the data block to be encoded can refer to the predictive mode of the data block to be encoded.

[0098] Optionally, if the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, then the above-mentioned selection of reference data blocks for predictive coding of the data block to be encoded from the global reference data block set and the local reference data block set includes: if the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, then since the local reference data blocks and the data block to be encoded have an adjacency relationship, and the second reference data block in the local reference data blocks has a strong correlation with the data block to be encoded, the computer device can prioritize searching the local reference data block set. If a reference data block that meets the conditions is found in the local reference data block set, it is not necessary to search the global reference data block set. If a reference data block that meets the conditions is not found in the local reference data block set, then the global reference data block set is searched. This helps to improve the efficiency of searching for reference data blocks and avoid unnecessary waste of resources. For example, firstly, the computer device can obtain the pixel difference between the second reference data block in the local reference data block set and the data block to be encoded. Here, the pixel difference can refer to the sum of the mean square error or absolute difference between the pixels of the second reference data block and the pixels of the data block to be encoded. If there is a target data block in the local reference data block set with a pixel difference less than the pixel difference threshold, it indicates that there is a strong correlation between the target data block and the data block to be encoded. Then, the target data block is used as a reference data block for predictive encoding of the data block to be encoded. If there is no target data block in the local reference data block set with a pixel difference less than the pixel difference threshold, it indicates that the correlation between the second reference data block in the local reference data block set and the data block to be encoded is relatively weak. Then, a reference data block for predictive encoding of the data block to be encoded is selected from the global reference data block set.

[0099] Optionally, if the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, then selecting a reference data block from the global reference data block set and the local reference data block set for predictive coding of the data block to be encoded includes: if the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, the computer device can determine the reference data block based on a target encoded data block that has a spatial adjacency relationship with the data block to be encoded. Here, the target encoded data block with a spatial adjacency relationship can refer to all encoded data blocks that have a positional adjacency relationship with the data block to be encoded, or, the target encoded data block with a spatial adjacency relationship can refer to an encoded data block that has a positional adjacency relationship with the data block to be encoded and is strongly correlated with the pixels of the data block to be encoded. Specifically, the computer device can acquire target encoded data blocks that have spatial adjacency with the data block to be encoded. If the reference data block of the target encoded data block belongs to the global reference data block set and the local reference data block set, then a refined search is performed in the regions corresponding to the global reference data block set and the local reference data block set according to the movement parameter to obtain reference data blocks for predictive coding of the data block to be encoded. At this time, the reference data block of the data block to be encoded belongs to the global reference data block set or the local reference data block set, or, the first region of the reference data block of the data block to be encoded belongs to the global reference data block set, and the second region of the reference data block of the data block to be encoded belongs to the local reference data block set, and the data block in the first region and the data block in the second region have a positional adjacency relationship. By searching for reference data blocks according to a movement parameter, it is beneficial to find the optimal reference data block and improve coding performance. If all reference data blocks of the target encoded data block belong to the global reference data block set, then the local reference data block set is skipped, and a fine-grained search is performed in the region corresponding to the global reference data block set according to the movement parameters to obtain reference data blocks for predictive coding of the data block to be encoded; if all reference data blocks of the target encoded data block belong to the local reference data block set, then the global reference data block set is skipped, and a fine-grained search is performed in the region corresponding to the local reference data block set according to the movement parameters to obtain reference data blocks for predictive coding of the data block to be encoded.

[0100] The movement parameters include a movement window and a movement step size. The size of the movement window is the same as the size of the data block to be encoded, and the movement step size is smaller than the width of the movement window.

[0101] Further, please see Figure 10 This is a flowchart illustrating a multimedia data processing method provided in an embodiment of this application. Figure 10As shown, the method may include at least the following S201-S202: S201. Obtain a global reference data block set and a local reference data block set associated with the data block to be decoded in the multimedia data frame; the global reference data block set includes decoded data blocks in the multimedia data frame that are not adjacent to the data block to be decoded, and the local reference data block set includes decoded data blocks in the multimedia data frame that are adjacent to the data block to be decoded.

[0102] In this application, the computer device can acquire multimedia data frames; these multimedia data frames can be those captured by the computer device or downloaded by the computer device from a network. The multimedia data frame refers to a video frame to be decoded in video data or an image frame to be decoded in image data. Video data can refer to live video data, film and television video data, daily life video data, road detection video data, etc.; image data can refer to images of people, buildings, objects, etc. Furthermore, the computer device can acquire a global reference data block set associated with the multimedia data frame based on at least one of the following: decoding method, write-back delay, correlation between decoded data blocks and data blocks to be decoded; and can acquire both a global reference data block set and a local reference data block set associated with the multimedia data frame based on at least one of the following: storage space size for storing local reference data block sets, correlation between decoded data blocks and data blocks to be decoded.

[0103] It should be noted that the global reference data block set and the local reference data block set are stored in the first storage space and the second storage space, respectively. The first storage space and the second storage space can be located in the same computer device or in different computer devices. Specifically, the first storage space and the second storage space can refer to a disk, or a storage media array, etc., and this application does not limit this. In particular, when the first storage space and the second storage space are located in the same computer device, the first storage space can refer to the off-chip memory of the computer device, and the second storage space can refer to the on-chip memory of the computer device; or, the first storage space can refer to the on-chip memory of the computer device, and the second storage space can refer to the off-chip memory of the computer device; this application does not limit this.

[0104] S202. Select a reference data block from the local reference data block set and the global reference data block set to decode the data block to be decoded.

[0105] In this application, the computer device can adaptively select reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and decode the data block to be decoded in the multimedia data frame according to the reference data blocks; this expands the selection range of reference data blocks and can improve the decoding performance of multimedia data frames.

[0106] For example, a computer device can acquire attribute information of a data block to be decoded. Based on this attribute information, it selects a reference data block from the local reference data block set and the global reference data block set to decode the data block to be decoded. The attribute information of the data block to be decoded includes one or more of the following: the media type of the multimedia data frame to which the data block belongs, the reference flag of the multimedia data frame, and the resolution of the multimedia data frame to which the data block belongs. The media type includes intra-frame and inter-frame types. Intra-frame types can refer to representative multimedia data frames, such as fully intra-decoded frames, Intra frames, I-frames, and keyframes. Frames of this type only allow intra-frame decoding and do not require reliance on other frames for decoding. For example, an intra-frame type can refer to a multimedia data frame that embodies the main meaning of video or image data; an inter-frame type can refer to multimedia data frames in video or image data other than intra-frame types, such as inter-decoded frames, P-frames, B-frames, and non-keyframes. This type of frame allows for the use of inter-frame decoding and intra-frame decoding techniques. Typically, when using inter-frame decoding, decoding can only proceed after the reference frame has been decoded. The reference flag of a multimedia data frame is used to reflect the range of reference data blocks selected for the data block to be decoded (i.e., which set of reference data blocks to choose).

[0107] In this application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for adaptively selecting reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and decoding the data block to be decoded in the multimedia data frame according to the reference data blocks; it expands the selection range of reference data blocks and can improve the decoding performance of multimedia data frames.

[0108] Optionally, the computer device may select one of the following two methods to obtain the global reference data block set based on the decoding method of the multimedia data frame: Method 1: The global data block set includes one or more first reference data blocks. When the multimedia data frame is decoded using serial decoding, the first reference data block in the global reference data block set is a decoded data block whose parameter association satisfies the write-back constraint and belongs to the same decoding chip as the data block to be decoded. The parameter association reflects the relationship between the first position information, the size information of the largest decoded block, and the second position information. The first position information and the second position information refer to the position information of the decoded data block and the data block to be decoded in the multimedia data frame, respectively. The write-back constraint reflects the time limit for writing the decoded data block to the global reference data block set. In other words, serial decoding means that only one data block in the multimedia data frame is decoded at a time. If the multimedia data frame is decoded using serial decoding, this decoding method will not affect the determination process of the reference data block. However, writing decoded data blocks to the global reference data block set introduces significant write-back latency, meaning it takes a considerable amount of time. Therefore, decoded data blocks must wait at least a preset duration before they can be used as reference data blocks. This preset duration is determined based on a time limit for writing the decoded data blocks to the global reference data block set. This time limit can refer to one of the average latency, maximum latency, or minimum latency for writing the decoded data blocks to the global reference data block set. Thus, when the multimedia data frame is decoded serially, the first reference data block in the global reference data block set must meet the following two conditions: a) it belongs to the same decoding chip as the data block to be decoded; b) the parameter association satisfies the write-back constraint.

[0109] Method 2: The global data block set includes one or more second reference data blocks. When the multimedia data frame is decoded in parallel or serial mode, the second reference data block in the global reference data block set is a decoded data block belonging to the same decoding chip as the data block to be decoded, whose parameter association in the multimedia data frame satisfies the write-back constraint, and whose parameter association satisfies the decoding processing condition. The parameter association reflects the relationship between the position information of the decoded data block and the data block to be decoded in the multimedia data frame, and the size information of the largest decoded block. The write-back constraint reflects the time limit for writing the decoded data block to the global reference data block set. In other words, parallel decoding refers to predictively decoding multiple data blocks in the multimedia data frame at the same time. If the multimedia data frame is decoded in parallel mode, this decoding method will affect the determination process of the reference data block. Meanwhile, writing decoded data blocks to the global reference data block set introduces significant write-back latency. Therefore, when the multimedia data frame is decoded using parallel decoding, the second reference data block in the global reference data block set must meet the following three conditions: a) it belongs to the same decoding chip as the data block to be decoded; b) the parameter association satisfies the write-back constraint; and c) the parameter association satisfies the decoding processing conditions. Specifically, when the multimedia data frame is decoded using serial decoding, method two can also be used to obtain the global reference data block set. Here, the decoding processing conditions can refer to the parallel decoding processing conditions.

[0110] It should be noted that the specific method for obtaining the global reference data block set can be found in the method for obtaining the global reference data block set at the encoding end. Where there is overlap, it will not be repeated.

[0111] Optionally, having an adjacency relationship between the data block to be encoded and the encoded data block can mean that the encoding order of the data block to be encoded is adjacent to the encoding order of the encoded data block; not having an adjacency relationship between the data block to be encoded and the encoded data block can mean that the encoding order of the data block to be encoded is not adjacent to the encoding order of the encoded data block.

[0112] Optionally, the computer device may select one or at least a combination of the following two methods to obtain the local reference data block set: Method 1: The local data block set includes one or more third reference data blocks. The third reference data block in this local reference data block set is a decoded data block within the multimedia data frame that resides in the same decoding processing unit as the data block to be decoded. This decoding processing unit can also be called a Virtual Pipeline Data Unit (VPDU). The size information of the VPDU can be determined based on the decoding processing capability of the decoder. For example, the size of the VPDU is VPDU_W x VPDU_H, and the values ​​of VPDU_W and VPDU_H can be 4, 8, 16, 32, 64, 128, etc. In other words, the decoded data block within the multimedia data frame whose coordinates satisfy ref_x / VPDU_W = cur_x / VPDU_W and ref_y / VPDU_H = cur_y / VPDU_H is used as the third reference data block. For example, if the size of the VPDU is 64x64, then all decoded data blocks within the 64x64 VPDU to which the data block to be decoded belongs can be used as reference data blocks for the data block to be decoded. Alternatively, if the VPDU size is 64x64, the decoded data block in the multimedia data frame whose coordinates satisfy Floor(ref_x / 64) = Floor(cur_x / 64) and Floor(ref_y / 64) = Floor(cur_y / 64) is used as the third reference data block.

[0113] Method 2: The local data block set includes one or more fourth reference data blocks; the fourth reference data block in this local reference data block set is a decoded data block in the multimedia data frame that is adjacent to the data block to be decoded and belongs to the same decoding chip as the data block to be decoded. The adjacency relationship between the fourth reference data block and the data block to be decoded can include one or more combinations of the following three cases: a. The adjacency relationship between the fourth reference data block and the data block to be decoded means that the fourth reference data block and the data block to be decoded belong to the same maximum decoding block.

[0114] b. The adjacency relationship between the fourth reference data block and the data block to be decoded means that the fourth reference data block is located within the N largest decoding blocks adjacent to the data block to be decoded, and the size of the largest decoding block is less than the size threshold; N is determined based on the size information of the largest decoding block.

[0115] It should be noted that the size of the largest encoded block is the same as the size of the largest decoded block, the size of the data block to be encoded is the same as the size of the data block to be decoded, and the size of the encoded chip is the same as the size of the decoded chip.

[0116] Optionally, the distances between the first reference data block, the second reference data block, the third reference data block, the fourth reference data block, and the data block to be decoded are all less than a second distance threshold; the second distance threshold can be dynamically set according to the application scenario or user needs, such as the second distance threshold being 2. 14 Meanwhile, the first reference data block, the second reference data block, the third reference data block, and the fourth reference data block are located at integer pixel positions in the multimedia data frame.

[0117] Optionally, for any multimedia data frame, the computer device can select a reference data block from a global reference data block set and a local reference data block set for decoding the data block to be decoded. Alternatively, the computer device can adaptively select a set of reference data blocks based on the decoding mode of the data block to be decoded, and determine the reference data block for decoding the data block from the selected set. Specifically, the computer device can obtain the decoding mode of the data block to be decoded; if the decoding mode of the data block to be decoded is an intra-block copy mode or an intra-string copy mode, then a reference data block for decoding the data block to be decoded is selected from the global reference data block set and the local reference data block set. The decoding mode of the data block to be decoded can be determined based on the media type of the multimedia data frame, or it can be determined based on the decoding flag (or reference flag) of the multimedia data frame. In particular, the decoding mode of the data block to be decoded can refer to the prediction mode of the data block to be decoded.

[0118] Please see Figure 11 This is a schematic diagram of the structure of a multimedia data processing apparatus provided in an embodiment of this application. The aforementioned multimedia data processing apparatus can be a computer program (including program code) running on a computer device; for example, the multimedia data processing apparatus is an application software. This apparatus can be used to execute corresponding steps in the methods provided in the embodiments of this application. Figure 11 As shown, the multimedia data processing device may include: an acquisition module 111 and a selection module 112.

[0119] An acquisition module is used to acquire a global reference data block set and a local reference data block set associated with the data block to be encoded in a multimedia data frame; the global reference data block set includes encoded data blocks in the multimedia data frame that are not adjacent to the data block to be encoded, and the local reference data block set includes encoded data blocks in the multimedia data frame that are adjacent to the data block to be encoded. The selection module is used to select a reference data block from the global reference data block set and the local reference data block set for predictive coding of the data block to be encoded.

[0120] Optionally, the selection module selects reference data blocks from the global reference data block set and the local reference data block set for predictive coding of the data block to be encoded, including: Obtain the encoding mode of the data block to be encoded; If the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, then a reference data block for predictive coding of the data block to be encoded is selected from the global reference data block set and the local reference data block set.

[0121] Optionally, if the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, the selection module selects reference data blocks from the global reference data block set and the local reference data block set for predictive coding of the data block to be encoded, including: If the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, then the pixel difference between the second reference data block in the local reference data block set and the data block to be encoded is obtained; If there is a target data block in the local reference data block set whose pixel difference is less than the pixel difference threshold, then the target data block is used as a reference data block for predictive coding of the data block to be encoded. If there is no target data block in the local reference data block set whose pixel difference is less than the pixel difference threshold, then a reference data block for predictive coding of the data block to be encoded is selected from the global reference data block set.

[0122] Optionally, if the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, the selection module selects reference data blocks from the global reference data block set and the local reference data block set for predictive coding of the data block to be encoded, including: If the encoding mode of the data block to be encoded is intra-block copy mode or intra-string copy mode, then obtain the target encoded data block that has a spatial adjacency relationship with the data block to be encoded. If the reference data block of the target encoded data block belongs to the global reference data block set and the local reference data block set, then a search is conducted in the regions corresponding to the global reference data block set and the local reference data block set according to the movement parameters to obtain a reference data block for predictive coding of the data block to be encoded; the movement parameters include a movement window and a movement step size, the size of the movement window is the same as the size of the data block to be encoded, and the movement step size is smaller than the width of the movement window.

[0123] Optionally, the global reference data block set includes one or more first reference data blocks; when the encoding method of the multimedia data frame is serial encoding, the first reference data block in the global reference data block set is an encoded data block whose parameter association relationship in the multimedia data frame satisfies the write-back constraint condition and belongs to the same coding slice as the data block to be encoded. The parameter association is used to reflect the association between the first position information, the size information of the largest coding block, and the second position information. The first position information and the second position information refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame, respectively. The write-back constraint is used to reflect the time limit for writing the encoded data block to the global reference data block set.

[0124] Optionally, the acquisition module is configured to, when the encoding method of the multimedia data frame is serial encoding, acquire first position information, and the size information of the maximum coded block and second position information respectively; the first position information and the second position information refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame, respectively; acquire write-back constraint conditions; the write-back constraint conditions are used to reflect the time limit for writing the encoded data block to the global reference data block set; select encoded data blocks from the multimedia data frame whose parameter association relationship satisfies the write-back constraint conditions and belongs to the same coded slice as the data block to be encoded, as the first reference data block; the parameter association relationship is used to reflect the association relationship between the first position information, the size information of the maximum coded block and the second position information; and add the first reference data block to the global reference data block set.

[0125] Optionally, the acquisition module filters out encoded data blocks from the multimedia data frames whose parameter associations satisfy the write-back constraint conditions and belong to the same coding slice as the data block to be encoded, as the first reference data block, including: Based on the first location information and the second location information, an encoded data block belonging to the same coding slice as the data block to be encoded is selected from the multimedia data frame and used as a first candidate reference data block; Determine the number of interval data blocks between the encoded data blocks in the multimedia data frame and the data blocks to be encoded under the write-back constraint conditions; Based on the first position information, the size information of the largest coded block, and the second position information, the number of interval data blocks between the first candidate reference data block and the data block to be encoded is counted. The first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks is determined as the first reference data block.

[0126] Optionally, the acquisition module counts the number of interval data blocks between the first candidate reference data block and the data block to be encoded based on the first position information, the size information of the largest coded block, and the second position information, including: The row and column identifiers of the first candidate reference data block are determined based on the first location information and the size information of the largest coded block; The row and column identifiers of the data block to be encoded are determined based on the second location information and the size information of the largest encoded block; The number of data blocks per line in the multimedia data frame is determined based on the size information of the data blocks to be encoded. Based on the number of data blocks in each row, the row identifier and column identifier of the first candidate reference data block, and the row identifier and column identifier of the data block to be encoded, the number of interval data blocks between the first candidate reference data block and the data block to be encoded is counted.

[0127] Optionally, the global reference data block set includes one or more second reference data blocks; the second reference data blocks in the global reference data block set are encoded data blocks that belong to the same coding slice as the data block to be encoded, and whose parameter association relationship in the multimedia data frame satisfies the write-back constraint condition, and whose parameter association relationship satisfies the encoding processing condition; the parameter association relationship is used to reflect the association relationship between the position information of the encoded data block and the data block to be encoded in the multimedia data frame, and the size information of the largest coding block; the write-back constraint condition is used to reflect the time limit for writing the encoded data block to the global reference data block set.

[0128] Optionally, the acquisition module is used to acquire first position information, the size information of the largest coded block, and second position information, respectively; the first position information and the second position information refer to the position information of the encoded data block and the data block to be encoded in the multimedia data frame, respectively; acquire write-back constraints and encoding processing conditions; the write-back constraints are used to reflect the time limit for writing the encoded data block to the global reference data block set; select encoded data blocks from the multimedia data frame whose parameter association relationship satisfies the write-back constraints and the encoding processing conditions, and which belong to the same coded slice as the data block to be encoded, as the second reference data block; the parameter association relationship is used to reflect the association relationship between the first position information, the size information of the data block to be encoded, and the second position information; and add the second reference data block to the global reference data block set.

[0129] Optionally, the acquisition module filters out encoded data blocks from the multimedia data frames whose parameter associations satisfy the write-back constraint and the encoding processing conditions, and which belong to the same coded segment as the data block to be encoded, as the second reference data block, including: Based on the first location information and the second location information, an encoded data block belonging to the same coding slice as the data block to be encoded is selected from the multimedia data frame and used as a first candidate reference data block; Select the first candidate reference data block whose parameter association relationship meets the write-back restriction condition from the first candidate reference data block, and use it as the second candidate reference data block; Determine the first data block constraint distance between the encoded data block in the multimedia data frame under the write-back constraint and the data block to be encoded; Determine the second data block constraint distance between the encoded data block in the multimedia data frame under the encoding processing conditions and the data block to be encoded. Based on the first position information, the size information of the maximum coding block, the second position information, the first data block limit distance, and the second data block limit distance, a second candidate reference data block that meets the coding processing conditions is determined from the second candidate reference data blocks and used as the second reference data block.

[0130] Optionally, the acquisition module determines a second candidate reference data block that satisfies the encoding processing conditions from the second candidate reference data blocks based on the first position information, the size information of the largest coding block, the second position information, the first data block limit distance, and the second data block limit distance, and uses it as the second reference data block, including: The row and column identifiers of the second candidate reference data block are determined based on the first location information and the size information of the largest coded block; The row and column identifiers of the data block to be encoded are determined based on the second location information and the size information of the largest encoded block; Select a second candidate reference data block from the second candidate reference data block whose line identifier is less than or equal to the line identifier of the data block to be encoded, and use it as the third candidate reference data block; The row distance between the third candidate reference data block and the data block to be encoded is determined based on the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded. The column distance between the third candidate reference data block and the data block to be encoded is determined based on the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded. The third candidate reference data block whose column distance is less than the first distance threshold is determined as the second reference data block; the first distance threshold is determined based on the first data block limit distance, the second data block limit distance, and the row distance between the third candidate reference data block and the data block to be encoded.

[0131] Optionally, the local reference data block set includes one or more third reference data blocks, wherein the third reference data block is an encoded data block in the multimedia data frame that is located in the same encoding processing unit as the data block to be encoded.

[0132] Optionally, the local reference data block set includes one or more fourth reference data blocks, wherein the fourth reference data block is an encoded data block in the multimedia data frame that is adjacent to the data block to be encoded and belongs to the same coding slice as the data block to be encoded.

[0133] Optionally, the adjacency relationship between the fourth reference data block and the data block to be encoded means that the fourth reference data block and the data block to be encoded belong to the same maximum coding block.

[0134] Optionally, the adjacency relationship between the fourth reference data block and the data block to be encoded means that the fourth reference data block is located within N largest coded blocks adjacent to the data block to be encoded, and the size of the largest coded block is less than or equal to a size threshold; N is determined based on the size information of the largest coded block.

[0135] Optionally, the adjacency relationship between the fourth reference data block and the data block to be encoded means that: the fourth reference data block is located within N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is greater than or equal to a size threshold; the data block located in the upper left corner of the region where the second reference data block is moved one maximum coding block to the right in the multimedia data frame is an unreconstructed data block; the position information of the fourth reference data block when it moves one maximum coding block distance is different from the position information of the data block to be encoded; N is determined based on the size information of the maximum coding block.

[0136] Optionally, the distances between the first reference data block, the second reference data block, the third reference data block, the fourth reference data block and the data block to be encoded are all less than a second distance threshold; the first reference data block, the second reference data block, the third reference data block and the fourth reference data block are located at integer pixel positions in the multimedia data frame.

[0137] According to one embodiment of this application, Figure 7 The steps involved in the data processing method shown can be derived from... Figure 11 The data processing device shown is executed by various modules. For example, Figure 7 Step S101 shown can be performed by Figure 11 The acquisition module 111 in the middle is used to execute, Figure 7 Step S102 shown can be performed by Figure 11 The selection module 112 is used to execute the selection.

[0138] According to one embodiment of this application, Figure 11 The various modules in the multimedia data processing device shown can be individually or entirely combined into one or more units, or one or more of these units can be further divided into at least two functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division. In practical applications, the function of one module can also be implemented by at least two units, or the function of at least two modules can be implemented by one unit. In other embodiments of this application, the data processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by at least two units.

[0139] According to one embodiment of this application, a general-purpose computer device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 10 and Figure 11 The computer program (including program code) involved in each step of the corresponding method shown, to construct such... Figure 11 The multimedia data processing apparatus shown herein, and the data processing method for implementing the embodiments of this application, are described. The computer program described above can be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and run therein.

[0140] In this application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for the encoding terminal to adaptively select reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and to perform predictive encoding on the data blocks to be encoded in the multimedia data frame according to the reference data blocks; the selection range of reference data blocks is expanded, which can improve the encoding performance of multimedia data frames.

[0141] Please see Figure 12This is a schematic diagram of the structure of a multimedia data processing apparatus provided in an embodiment of this application. The aforementioned multimedia data processing apparatus can be a computer program (including program code) running on a computer device; for example, the multimedia data processing apparatus is an application software. This apparatus can be used to execute corresponding steps in the methods provided in the embodiments of this application. Figure 12 As shown, the multimedia data processing device may include: an acquisition module 121 and a selection module 122.

[0142] An acquisition module is used to acquire a global reference data block set and a local reference data block set associated with the data block to be decoded in a multimedia data frame; the global reference data block set includes decoded data blocks in the multimedia data frame that are not adjacent to the data block to be decoded, and the local reference data block set includes decoded data blocks in the multimedia data frame that are adjacent to the data block to be decoded; The selection module is used to select a reference data block from the global reference data block set and the local reference data block set for decoding the data block to be decoded.

[0143] Optionally, the selection module selects reference data blocks from the global reference data block set and the local reference data block set for decoding the data block to be decoded, including: Obtain the decoding mode of the data block to be decoded; If the decoding mode of the data block to be decoded is intra-block copy mode or intra-string copy mode, then a reference data block for decoding the data block to be decoded is selected from the global reference data block set and the local reference data block set.

[0144] Optionally, the global reference data block set includes one or more first reference data blocks; when the multimedia data frame is decoded in a serial decoding mode, the first reference data block in the global reference data block set is a decoded data block whose parameter association relationship in the multimedia data frame satisfies the write-back constraint condition and belongs to the same decoding chip as the data block to be decoded; The parameter association is used to reflect the association between the first position information, the size information of the largest decoded block, and the second position information. The first position information and the second position information refer to the position information of the decoded data block and the data block to be decoded in the multimedia data frame, respectively. The write-back constraint is used to reflect the time limit for writing the decoded data block to the global reference data block set.

[0145] Optionally, the global reference data block set includes one or more second reference data blocks; the second reference data blocks in the global reference data block set are decoded data blocks that belong to the same decoding chip as the data block to be decoded, and whose parameter association relationship in the multimedia data frame satisfies the write-back constraint condition, and whose parameter association relationship satisfies the decoding processing condition; the parameter association relationship is used to reflect the association relationship between the position information of the decoded data block and the data block to be decoded in the multimedia data frame, and the size information of the largest decoded block; the write-back constraint condition is used to reflect the time limit for writing the decoded data block to the global reference data block set.

[0146] Optionally, the local reference data block set includes one or more third reference data blocks; the third reference data block in the local reference data block set is a decoded data block in the multimedia data frame that is located in the same decoding processing unit as the data block to be decoded.

[0147] Optionally, the local reference data block set includes one or more fourth reference data blocks; the fourth reference data block of the local reference data block set is a decoded data block in the multimedia data frame that is adjacent to the data block to be decoded and belongs to the same decoding chip as the data block to be decoded.

[0148] According to one embodiment of this application, Figure 10 The steps involved in the data processing method shown can be derived from... Figure 12 The data processing device shown is executed by various modules. For example, Figure 10 Step S201 shown can be performed by Figure 12 The acquisition module 121 in the middle is used to execute, Figure 10 Step S202 shown can be performed by Figure 12 The selection module 122 is used to execute.

[0149] According to one embodiment of this application, Figure 12 The various modules in the multimedia data processing device shown can be individually or entirely combined into one or more units, or one or more of these units can be further divided into at least two functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division. In practical applications, the function of one module can also be implemented by at least two units, or the function of at least two modules can be implemented by one unit. In other embodiments of this application, the data processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by at least two units.

[0150] According to one embodiment of this application, a general-purpose computer device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 10 and Figure 11 The computer program (including program code) involved in each step of the corresponding method shown, to construct such... Figure 12 The multimedia data processing apparatus shown herein, and the data processing method for implementing the embodiments of this application, are described. The computer program described above can be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and run therein.

[0151] In this application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for adaptively selecting reference data blocks from the global reference data block set and the local reference data block set according to the application scenario or user needs, and performing predictive decoding of the data block to be decoded in the multimedia data frame based on the reference data blocks; it expands the selection range of reference data blocks and can improve the decoding performance of multimedia data frames.

[0152] Please see Figure 13 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 13 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi). I -F I (Interface). Memory 1005 can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, memory 1005 can also be at least one storage device located remotely from the aforementioned processor 1001. Figure 13 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0153] exist Figure 13In the computer device 1000 shown, the network interface 1004 provides network communication functions; while the user interface 1003 is mainly used to provide an interface for inputting media content.

[0154] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 7 and the preceding text Figure 10 The description of the multimedia data processing method in the corresponding embodiments can also be performed as described above. Figure 11 and Figure 12 The description of the multimedia data processing apparatus in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0155] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned multimedia data processing device. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 7 and Figure 10 The description of the multimedia data processing method in the corresponding embodiments is already provided, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0156] As an example, the above program instructions can be deployed and executed on a single computer device, or deployed and executed on at least two computer devices located in one location, or executed on at least two computer devices distributed in at least two locations and interconnected by a communication network. At least two computer devices distributed in at least two locations and interconnected by a communication network can form a blockchain network.

[0157] The aforementioned computer-readable storage medium can be an internal storage unit of the data processing apparatus or computer device provided in any of the foregoing embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0158] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the foregoing description. Figure 7 and Figure 10 The data processing method described in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiments of this application.

[0159] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different media content, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0160] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0161] The methods and related apparatuses provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0162] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A multimedia data processing method, characterized by, The method comprises: obtaining a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; selecting a reference data block for predicting encoding the to-be-encoded data block from the global reference data block set and the local reference data block set.

2. The method of claim 1, wherein, The global reference data block set comprises encoded data blocks in the multimedia data frame that have no adjacent relationship with the to-be-encoded data block, and the local reference data block set comprises encoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-encoded data block.

3. The method of claim 2, wherein, The selecting a reference data block for predicting encoding the to-be-encoded data block from the global reference data block set and the local reference data block set comprises: obtaining an encoding mode of the to-be-encoded data block; if the encoding mode of the to-be-encoded data block is an intra block copy mode or an intra string copy mode, selecting a reference data block for predicting encoding the to-be-encoded data block from the global reference data block set and the local reference data block set.

4. The method of claim 3, wherein, The selecting a reference data block for predicting encoding the to-be-encoded data block from the global reference data block set and the local reference data block set if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode comprises: if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, obtaining pixel difference between data blocks in the local reference data block set and the to-be-encoded data block; if there is a target data block in the local reference data block set with pixel difference less than a pixel difference threshold, taking the target data block as a reference data block for predicting encoding the to-be-encoded data block; if there is no target data block in the local reference data block set with pixel difference less than the pixel difference threshold, selecting a reference data block for predicting encoding the to-be-encoded data block from the global reference data block set.

5. The method of claim 3, wherein, The selecting a reference data block for predicting encoding the to-be-encoded data block from the global reference data block set and the local reference data block set if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode comprises: if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, obtaining a target encoded data block having a spatial adjacent relationship with the to-be-encoded data block; if a reference data block of the target encoded data block belongs to the global reference data block set and the local reference data block set, searching in regions corresponding to the global reference data block set and the local reference data block set according to a movement parameter to obtain a reference data block for predicting encoding the to-be-encoded data block; the movement parameter comprises a movement window and a movement step.

6. The method of claim 5, wherein, The size of the movement window is the same as the size of the to-be-encoded data block, and the movement step is smaller than the width of the movement window.

7. The method according to any one of claims 1 to 6, wherein The global reference data block set comprises one or more first reference data blocks; when the encoding mode of the multimedia data frame is a serial encoding mode, the first reference data blocks are the encoded data blocks in the multimedia data frame, which satisfy a write-back restriction condition and belong to the same encoding slice as the to-be-encoded data block; The parameter association relationship is used to reflect the association relationship between the first position information, the maximum encoding block size information and the second position information, and the first position information and the second position information respectively refer to the position information of the encoded data block and the to-be-encoded data block in the multimedia data frame; The write-back restriction condition is used to reflect the restriction time delay of writing the encoded data block into the global reference data block set.

8. The method according to any one of claims 1 to 6, wherein The global reference data block set comprises one or more second reference data blocks; the second reference data blocks are the encoded data blocks, which satisfy a write-back restriction condition and belong to the same encoding slice as the to-be-encoded data block, and satisfy an encoding processing condition in the parameter association relationship in the multimedia data frame; The parameter association relationship is used to reflect the association relationship between the position information of the encoded data block and the to-be-encoded data block in the multimedia data frame and the maximum encoding block size information; The write-back restriction condition is used to reflect the restriction time delay of writing the encoded data block into the global reference data block set.

9. The method according to any one of claims 1 to 6, wherein, The global reference data block set associated with the to-be-encoded data block in the multimedia data frame is obtained, comprising: When the encoding mode of the multimedia data frame is a serial encoding mode, the first position information, the maximum encoding block size information and the second position information are respectively obtained; the first position information and the second position information respectively refer to the position information of the encoded data block and the to-be-encoded data block in the multimedia data frame; The write-back restriction condition is obtained; the write-back restriction condition is used to reflect the restriction time delay of writing the encoded data block into the global reference data block set; The encoded data blocks, which satisfy the write-back restriction condition and belong to the same encoding slice as the to-be-encoded data block in the parameter association relationship in the multimedia data frame, are selected from the multimedia data frame as first reference data blocks; the parameter association relationship is used to reflect the association relationship between the first position information, the maximum encoding block size information and the second position information; The first reference data blocks are added to the global reference data block set.

10. The method of claim 9, wherein, The encoded data blocks, which satisfy the write-back restriction condition and belong to the same encoding slice as the to-be-encoded data block in the parameter association relationship in the multimedia data frame, are selected from the multimedia data frame as first reference data blocks, comprising: According to the first position information and the second position information, the encoded data blocks, which belong to the same encoding slice as the to-be-encoded data block, are selected from the multimedia data frame as first candidate reference data blocks; The interval data block restriction quantity between the encoded data blocks in the multimedia data frame and the to-be-encoded data block under the write-back restriction condition is determined; According to the first position information, the size information of the maximum coding block and the second position information, the number of interval data blocks between the first candidate reference data block and the data block to be coded is counted; The first candidate reference data block with the number of interval data blocks greater than the interval data block limit number is determined as the first reference data block.

11. The method of claim 10, wherein, The counting of the number of interval data blocks between the first candidate reference data block and the data block to be coded according to the first position information, the size information of the maximum coding block and the second position information comprises: The row identifier and the column identifier of the first candidate reference data block are determined according to the first position information and the size information of the maximum coding block; The row identifier and the column identifier of the data block to be coded are determined according to the second position information and the size information of the maximum coding block; The number of data blocks in each row in the multimedia data frame is determined based on the size information of the data block to be coded; The number of interval data blocks between the first candidate reference data block and the data block to be coded is counted according to the number of data blocks in each row, the row identifier and the column identifier of the first candidate reference data block, and the row identifier and the column identifier of the data block to be coded.

12. The method of any one of claims 1-6, wherein, The acquiring of the global reference data block set associated with the data block to be coded in the multimedia data frame comprises: The first position information, the size information of the maximum coding block and the second position information are respectively acquired; the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame; The write-back limit condition and the coding processing condition are acquired; the write-back limit condition is used to reflect the limit delay of writing the coded data block into the global reference data block set; The coded data block with the parameter association relationship satisfying the write-back limit condition and the coding processing condition and belonging to the same coding slice as the data block to be coded is selected from the multimedia data frame as the second reference data block; the parameter association relationship is used to reflect the association relationship between the first position information, the size information of the data block to be coded and the second position information; The second reference data block is added to the global reference data block set.

13. The method of claim 12, wherein, The selecting of the coded data block with the parameter association relationship satisfying the write-back limit condition and the coding processing condition and belonging to the same coding slice as the data block to be coded from the multimedia data frame as the second reference data block comprises: The coded data block belonging to the same coding slice as the data block to be coded is selected from the multimedia data frame as the first candidate reference data block according to the first position information and the second position information; The first candidate reference data block with the parameter association relationship satisfying the write-back limit condition is selected from the first candidate reference data block as the second candidate reference data block; The first data block limit distance between the coded data block in the multimedia data frame and the data block to be coded under the write-back limit condition is determined. determining a second data block limit distance between the coded data block in the multimedia data frame and the to-be-coded data block under the coding processing condition; determining a second candidate reference data block satisfying the coding processing condition from the second candidate reference data blocks as a second reference data block according to the first position information, the size information of the maximum coding block and the second position information, the first data block limit distance and the second data block limit distance.

14. The method of claim 13, wherein, The method for determining a second candidate reference data block satisfying the coding processing condition from the second candidate reference data blocks as a second reference data block according to the first position information, the size information of the maximum coding block and the second position information, the first data block limit distance and the second data block limit distance comprises: determining a row identifier and a column identifier of the second candidate reference data block according to the first position information and the size information of the maximum coding block; determining a row identifier and a column identifier of the to-be-coded data block according to the second position information and the size information of the maximum coding block; screening a second candidate reference data block with a row identifier less than or equal to the row identifier of the to-be-coded data block from the second candidate reference data blocks as a third candidate reference data block; determining a row distance between the third candidate reference data block and the to-be-coded data block according to the row identifier of the third candidate reference data block and the row identifier of the to-be-coded data block; determining a column distance between the third candidate reference data block and the to-be-coded data block according to the column identifier of the third candidate reference data block and the column identifier of the to-be-coded data block; determining a second reference data block with a column distance less than a first distance threshold from the third candidate reference data blocks, wherein the first distance threshold is determined according to the first data block limit distance, the second data block limit distance and the row distance between the third candidate reference data block and the to-be-coded data block.

15. A multimedia data processing method, characterized by, The method comprises: obtaining a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; selecting a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set.

16. The method of claim 15, wherein, The global reference data block set comprises decoded data blocks in the multimedia data frame having no adjacent relationship with the to-be-decoded data block, and the local reference data block set comprises decoded data blocks in the multimedia data frame having an adjacent relationship with the to-be-decoded data block.

17. A multimedia data processing apparatus, characterized by comprising: The method comprises: an obtaining module configured to obtain a global reference data block set and a local reference data block set associated with a to-be-coded data block in a multimedia data frame; a selecting module configured to select a reference data block for predicting coding the to-be-coded data block from the global reference data block set and the local reference data block set.

18. A multimedia data processing apparatus, characterized by comprising: The method comprises: an obtaining module configured to obtain a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; a selecting module configured to select a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set. The selecting module is configured to select, from the global reference data block set and the local reference data block set, a reference data block for decoding the to-be-decoded data block.

19. A computer device, comprising: The method comprises: a processor and a memory; The processor is connected with the memory; the memory is configured to store program code; and the processor is configured to invoke the program code to execute the method in any one of claims 1 to 16.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor, so that the computer device with the processor executes the method in any one of claims 1 to 16.

Images