Multimedia data processing method, apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose a multimedia data processing method, device and equipment, and a storage medium, wherein the method comprises: obtaining a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; the global reference data block comprises an encoded data block in the multimedia data frame which has no adjacent relationship with the to-be-encoded data block; the local reference data block set comprises an encoded data block in the multimedia data frame which has an adjacent relationship with the to-be-encoded data block; and selecting a reference data block for predicting and encoding the to-be-encoded data block from the global reference data block set and the local reference data block set. The encoding performance of the media data can be effectively improved by the present application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a multimedia data processing method and device, equipment and storage medium. BACKGROUND

[0002] In the process of multimedia data processing, the intra-frame prediction coding mode is usually used to predict and encode the multimedia data frame, which can improve the coding performance of the multimedia data frame, and therefore, the intra-frame prediction coding mode has been widely applied to the process of multimedia data processing. However, it is found in practice that the existing intra-frame prediction coding mode has limited reference data blocks for prediction coding, which leads to poor coding performance of the multimedia data frame. SUMMARY

[0003] The technical problem to be solved by the embodiments of the present application is to provide a multimedia data processing method, device, equipment and storage medium, which can improve the coding performance of the multimedia data frame.

[0004] The embodiments of the present application provide a multimedia data processing method, comprising:

[0005] obtaining a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; the global reference data block set comprises an already-encoded data block in the multimedia data frame which has no adjacent relationship with the to-be-encoded data block, and the local reference data block set comprises an already-encoded data block in the multimedia data frame which has an adjacent relationship with the to-be-encoded data block;

[0006] selecting a reference data block for predicting and encoding the to-be-encoded data block from the global reference data block set and the local reference data block set.

[0007] The embodiments of the present application provide a multimedia data processing method, comprising:

[0008] obtaining a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; the global reference data block set comprises an already-decoded data block in the multimedia data frame which has no adjacent relationship with the to-be-decoded data block, and the local reference data block set comprises an already-decoded data block in the multimedia data frame which has an adjacent relationship with the to-be-decoded data block;

[0009] selecting a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set.

[0010] The embodiments of the present application provide a multimedia data processing device, comprising:

[0011] The acquisition module is configured to acquire a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; the global reference data block set comprises encoded data blocks in the multimedia data frame that have no adjacent relationship with the to-be-encoded data block, and the local reference data block set comprises encoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-encoded data block.

[0012] The selection module is configured to select a reference data block for performing predictive encoding on the to-be-encoded data block from the global reference data block set and the local reference data block set.

[0013] The embodiment of the present application provides a multimedia data processing device, which comprises:

[0014] The acquisition module is configured to acquire a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; the global reference data block set comprises decoded data blocks in the multimedia data frame that have no adjacent relationship with the to-be-decoded data block, and the local reference data block set comprises decoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-decoded data block.

[0015] The selection module is configured to select a reference data block for performing decoding on the to-be-decoded data block from the global reference data block set and the local reference data block set.

[0016] The embodiment of the present application provides a computer device, which comprises a processor and a memory.

[0017] The memory is configured to store a computer program, and the processor is configured to invoke the computer program to execute the steps in the method.

[0018] The embodiment of the present application provides a computer readable storage medium, which stores a computer program; the computer program comprises program instructions; when the program instructions are executed by a processor, the steps in the method are executed.

[0019] The embodiment of the present application provides a computer program product, which comprises computer program / instructions; when the computer program / instructions are executed by a processor, the steps in the method are implemented.

[0020] In the present application, the computer device can acquire the global reference data block set and the local reference data block set, which is beneficial for the coding terminal to adaptively select the reference data block from the global reference data block set and the local reference data block set according to the application scenario or user demand, and to perform predictive coding on the to-be-coded data block in the multimedia data frame according to the reference data block; the selection range of the reference data block is expanded, and the coding performance of the multimedia data frame can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0022] Figure 1 is a flowchart of a video processing provided by the present application;

[0023] Figure 2 is a flowchart of a processing method for multimedia data provided by the present application;

[0024] Figure 3 is a schematic diagram of a coded data block provided by the present application;

[0025] Figure 4 is a scene diagram of inter-frame coding of a multimedia data frame provided by the present application;

[0026] Figure 5 is a scene diagram of intra-frame block copy of a multimedia data frame provided by the present application;

[0027] Figure 6 is a scene diagram of memory reuse of a reference data block of a multimedia data frame provided by the present application;

[0028] Figure 7 is a flowchart of a multimedia data processing method provided by the present application;

[0029] Figure 8 is a schematic diagram of the position relationship among a global reference data set, a local reference data block set and a to-be-coded data block provided by the present application;

[0030] Figure 9 is a schematic diagram of the position relationship among a global reference data set, a local reference data block set and a to-be-coded data block provided by the present application;

[0031] Figure 10 is a flowchart of a multimedia data processing method provided by the present application;

[0032] Figure 11 is a structural schematic diagram of a multimedia data processing device provided by the present application;

[0033] Figure 12 is a structural schematic diagram of a multimedia data processing device provided by the present application;

[0034] Figure 13 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0036] The embodiments of the present application relate to a processing technology of multimedia data. The multimedia data (or referred to as media data) refers to a composite data formed by media data such as text, graphics, image, sound, animation and moving image which are related to each other in content. The multimedia data mentioned in the embodiments of the present application mainly includes image data composed of images or video data composed of images and sound. In the embodiments of the present application, the data processing process of multimedia data is described in detail taking video data as an example. When the multimedia data is image data, refer to the embodiments of the present application. The processing process of multimedia data relates to media data collection, media data coding, media data file packaging, media data file transmission, media data decoding and final data presentation. When the multimedia data is video data, the complete processing process of video data can be as shown in FIG. 1, which can specifically include video collection, video coding, video file packaging, video transmission, video file unpacking, video decoding and final video presentation. Figure 1

[0037] ​Video capture is used to convert analog video to digital video and save it in the format of digital video file, that is, video capture can convert video signal to binary digital information, wherein the binary information converted from the video signal is a binary data stream, and the binary information can also be referred to as a code stream or a bit stream of the video signal. Video encoding is to convert a file in an original video format into another file in another video format by compression technology. In the embodiments of the present application, the generation of video media content includes the generation of real scenes by camera capture and the generation of screen content scenes by computer. From the perspective of the acquisition method of the video signal, the video signal can be divided into two types: camera shooting and computer generation. Due to the difference in statistical characteristics, the corresponding compression encoding methods may also be different. Modern mainstream video encoding technologies, such as international video encoding standards HEVC (High Efficiency Video Coding, international video encoding standard HEVC / H.265), VVC (versatile video coding, international video encoding standard VVC / H.266), and Chinese national video encoding standard AVS (Audio Video Coding Standard, Chinese national video encoding standard AVS), or AVS3 (third generation video encoding standard promoted by AVS standard group), adopt a hybrid encoding framework. For the input original video signal, a series of operations and processing are performed as follows, which can be specifically as shown in Figure 2

[0038] ​① Block partition structure: input multimedia data frame (e.g. a video frame in a video data) is divided into several non-overlapping processing units according to its size, each processing unit will undergo similar compression operation. In one embodiment, this processing unit is called CTU (Coding Tree Unit) or LCU (Largest Coding Unit). CTU can be further divided into one or more basic coding units, called CU (Coding Unit), for more detailed partitioning. Each CU is the most basic element in a coding loop. In another embodiment, this processing unit is also called tile (a rectangular area of multimedia data frame that can be independently decoded and encoded). Tile can be further divided into one or more SBs (Superblocks) for more detailed partitioning. SB can be further divided into one or more blocks (B) for more detailed partitioning. Each block is the most basic element in a coding loop. The following describes various encoding methods that can be used for each block. The relationship between SB and B can be as shown in Figure 3

[0039] ② Predictive coding: including intra-prediction and inter-prediction, etc. The original video signal is predicted by the selected reconstructed video signal to obtain the residual video signal. The encoding end needs to determine the most suitable one from the many possible predictive coding modes for the current encoding block (i.e. the to-be-encoded block), and inform the decoding end.

[0040] a. Intra-prediction: the predicted signal comes from the already encoded and reconstructed area within the same image.

[0041] b. Inter-prediction: the predicted signal comes from the already encoded image (called reference image) other than the current image.

[0042] ​③ Transform & Quantization: The residual video signal is transformed into transform domain by DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform, which is a subset of DFT) and other transform operations, and is converted into transform coefficients. The signal in the transform domain is further subjected to a lossy quantization operation, and certain information is lost, so that the quantized signal is conducive to compressed expression.

[0043] In some video coding standards, more than one transform method can be selected, so the encoding end also needs to select one of them for the current coding data block, and inform the decoding end. The degree of quantization precision is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a larger value range will be quantized to the same output, so it usually brings larger distortion and lower code rate; on the contrary, a smaller QP value means that coefficients with a smaller value range will be quantized to the same output, so it usually brings smaller distortion and higher code rate.

[0044] ④ Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and coded according to the frequency of each value, and finally output the binary (0 or 1) compressed code stream. At the same time, other information generated by encoding, such as selected mode, motion vector, etc., also needs to be entropy coded to reduce the code rate.

[0045] Statistical coding is a lossless coding method that can effectively reduce the code rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0046] ⑤ Loop Filtering: The encoded image (i.e., multimedia data frame) undergoes inverse quantization, inverse transform, and prediction compensation operations (the reverse of operations ② to ④ above) to obtain a reconstructed decoded image. Compared to the original image, the reconstructed image differs in some information due to the influence of quantization, resulting in distortion. Filtering the reconstructed image, such as deblocking, SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter), can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent encoded images to predict future signals, the above filtering operations are also called loop filtering, or filtering operations within the coding loop.

[0047] Figure 2 The document illustrates the basic process of a video encoder. Figure 2 The k-th CU (labeled as S) k Taking [x, y] as an example, where k is a positive integer greater than or equal to 1 and less than or equal to the number of CUs in the current input image, and S k [x, y] represents the pixel with coordinates [x, y] in the k-th CU, where x represents the x-coordinate of the pixel and y represents the mid-coordinate of the pixel. k The predicted signal is obtained after [x, y] undergoes a preferred processing method such as motion compensation or intra-frame prediction. S k [x, y] and Subtraction yields the residual signal U k [x, y], and then the residual signal U k The [x, y] array is transformed and quantized. The quantized output data goes to two different places: one is sent to an entropy encoder for entropy encoding, and the encoded bitstream is output to a buffer for storage, waiting to be transmitted; the other application is to perform inverse quantization and inverse transform to obtain the signal U'. k [x, y]. The signal U' k [x, y] and The summation yields a new prediction signal S* k [x, y], and S* k [x, y] are stored in the buffer of the current image. S* k [x, y] are obtained by intra-image prediction f(S*). k [x, y]), S* k [x, y] is filtered by a loop to obtain S' k[x, y], and S' k [x, y] to the decoded picture buffer for generating the reconstructed video. S' k [x, y] after motion-compensated prediction. S' r [x+m x , y+m y ], S' r [x+m x , y+m y ] represents the reference block, m x and m y represent the horizontal and vertical components of the motion vector, respectively.

[0048] According to the above encoding process, at the decoding end, for each data block, the decoder obtains the compressed code stream, first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient is subjected to inverse quantization and inverse transform to obtain a residual signal. On the other hand, according to the known encoding mode information, the prediction signal corresponding to the data block can be obtained, and after the two are added, the encoded data block is obtained. Finally, the encoded data block needs to be subjected to loop filtering operation to generate the final encoded data block.

[0049] The current mainstream video encoding standards, such as HEVC, VVC, AVS3, AV1, and AV2, all use a block-based hybrid encoding framework. They divide the original multimedia data frame into a series of encoding data blocks, combine prediction, transform, and entropy encoding video encoding methods to achieve video data compression. Among them, motion compensation is a commonly used prediction encoding method in video encoding. Motion compensation is based on the redundancy characteristics of video content in the time domain or spatial domain, and derives the prediction value of the current encoding data block from the already encoded area (i.e., the reference data block). This type of prediction encoding method includes inter prediction, intra block copy prediction, and intra string copy prediction. In specific prediction encoding implementations, these prediction encoding methods can be used alone or in combination. For encoding data blocks that use these prediction encoding methods, one or more two-dimensional displacement vectors usually need to be explicitly or implicitly encoded in the code stream to indicate the displacement of the current encoding data block (or the same block of the current block) relative to one or more reference data blocks.

[0050] It should be noted that in different prediction coding modes, the displacement vector can have different names, and the present application is uniformly described as follows: 1) the displacement vector in inter prediction is called motion vector (MV); 2) the displacement vector in intra block copy (i.e. intra prediction coding) is called block vector (BV); 3) the displacement vector in intra string copy is called string vector (SV). The following describes the technologies related to inter prediction and intra block copy prediction.

[0051] 1.1, Inter prediction. As shown in Figure 4 , inter prediction uses the correlation in the time domain of multimedia data frames to predict the pixels of the current image using the pixels of the adjacent coded image, so as to effectively remove the temporal redundancy of the multimedia data frames and effectively save the bits of the coded residual data. Wherein, P is the current frame, Pr is the reference frame, B is the current data block to be coded, and Br is the reference data block of B. B' has the same coordinate position as B in the image, and Br has coordinates (x r ,y r ), and B' has coordinates (x, y). The displacement between the current coded data block and its reference data block is called motion vector (MV), which can be represented by the following formula (1):

[0052] MV = (x r -x, y r -y) (1)

[0053] Considering that the adjacent blocks in the time domain or the spatial domain have strong correlation, MV prediction technology can be used to further reduce the bits required for coding MV. In H.265 / HEVC, inter prediction includes two MV prediction technologies, Merge and AMVP.

[0054] 2.1, Intra block copy prediction. Intra block copy (IBC) is a kind of intra coding tool adopted in HEVC screen content coding (SCC) extension, which significantly improves the coding efficiency of screen content. In AVS3, VVC and AV1, IBC technology is also adopted to improve the performance of screen content coding. IBC uses the spatial correlation of screen content video to predict the pixels of the data block to be coded using the pixels of the coded data block in the multimedia data frame, which can effectively save the bits required for coding pixels. Figure 5 is a schematic diagram of IBC technology, as Figure 5In the HEVC SCC extension, the IBC mode is implemented in a similar way as inter prediction. If the IBC mode is allowed for a multimedia data frame, the current decoded but unfiltered picture will be used as a special reference frame, and the encoding data block is derived from this reference frame in a similar way as inter prediction motion compensation. This implementation has the following characteristics:

[0055] In the HEVC SCC extension, the IBC mode is implemented in a similar way as inter prediction. If the IBC mode is allowed for a multimedia data frame, the current decoded but unfiltered picture will be used as a special reference frame, and the encoding data block is derived from this reference frame in a similar way as inter prediction motion compensation. This implementation has the following characteristics:

[0056] 1) BV is only in integer precision;

[0057] 2) A global reference data block set (i.e. global reference range) is used, i.e. all reconstructed parts of the current multimedia data frame can be used as reference data blocks for the encoding data block except for some regions which are forbidden to be used as reference data blocks due to parallel processing and chroma interpolation considerations;

[0058] 3) In the decoded picture buffer, the current decoded but unfiltered picture will occupy a picture size of buffer;

[0059] 4) The current decoded but unfiltered picture will be at the end of the reference picture list.

[0060] In hardware implementation, the computation core can read and write data from on-chip memory or off-chip memory. On-chip memory is usually static memory, which has very fast read and write speed, but it is relatively expensive, so only a small on-chip memory is usually used. Off-chip memory is usually much larger, but if it is to be accessed efficiently, it needs to follow some rules, and there is usually a writing-back delay when writing back the recently encoded data block to the off-chip picture memory. In a typical hardware implementation of motion compensation, the current decoded picture will be written to the off-chip memory after completing the loop filtering operation for future reference. When the motion compensation of the subsequent encoding data block needs to refer to the encoded data block located in the off-chip memory, the relevant data of the reference data block needs to be read from the off-chip memory. This process involves memory bandwidth consumption for writing and reading. Although the IBC scheme in HEVC-SCC has little impact on existing software design in software implementation, it has the following two problems in hardware implementation:

[0061] a. The reference picture of IBC is the unfiltered version, which is different from the final output decoded picture. Outputting this picture to off-chip will cause an increase in memory bandwidth.

[0062] b、Current decoded or one just reconstructed neighboring data block can be used as reference for the next encoding data block. This requires the decoded data block to be reconstructed, output to off-chip memory, read into on-chip memory, etc. within a limited clock cycle. This poses a big challenge for hardware implementation.

[0063] To address the IBC hardware implementation issue, some IBC related optimization schemes are adopted in the subsequent standards such as VVC, AVS3 and AV1. The IBC scheme in VVC, AVS3 standards uses a local reference data block set, i.e. using the neighboring encoded data blocks of the current encoding data block as the reference data block of the current encoding data block. In the specific hardware implementation, the reference data block in the local reference data block set stored in the on-chip memory can be used to solve the above problem. Considering the hardware implementation cost and encoding performance, IBC only uses one reference data block in the local reference data block set stored in the memory of the size of CTU, and adopts the reference memory reuse strategy as shown in FIG. 1. In addition to storing the 64x64 CU to be reconstructed, there are three 64x64 CUs that can be used to store the reconstructed data blocks. Therefore, IBC can only search for reference data blocks in the three 64x64 CUs and the reconstructed part of the current 64x64 CU. Figure 6

[0064] ​In summary, the IBC scheme in the AV1 standard uses a global reference data block set, i.e., the current encoding data block allows to reference the encoded data blocks in the current multimedia data frame, but this scheme has the following two defects: a, in order to meet the requirements of the write-back limitation condition, the encoded data blocks adjacent to the current encoding data block are not allowed to be used as reference data blocks; b, if the current multimedia data frame allows to use IBC, the loop filter will be disabled. However, screen content sequences often have strong spatial correlation, and adjacent data blocks have high reference value. Allowing adjacent encoded data blocks to be referenced will help improve the encoding performance. In some sequence content, disabling the loop filter will also cause serious multimedia data quality decline and affect the encoding performance. Although the IBC scheme based on the local reference data block set does not have the above defects, due to the limited number of reference data blocks available in the local reference data block set, only using the local reference data block set may cause the encoding performance of the multimedia data frame to decline. According to the characteristics of the sequence content, the encoding parameters such as the global reference data block set, the local reference data block set and the loop filter processing have different proportions of influence on the encoding performance. Therefore, based on the defects of the IBC scheme based on the global reference data block set and the IBC scheme based on the local reference data block set, in this application, according to the attribute information of the multimedia data frame, a suitable prediction encoding mode is selected, which will help to improve the encoding performance; that is, in this application, the global reference data block set and the local reference data block set are provided, which is helpful for the encoding terminal to adaptively select the reference data block from the global reference data block set and the local reference data block set according to the attribute information of the current encoding data block, expands the range of selectable reference data blocks, and solves the problem of limited number of reference data blocks. Further, according to the selected reference data block, the current encoding data block is predicted and encoded, which can improve the encoding performance of the multimedia data frame.

[0065] It should be noted that the present application can be executed by a terminal, can be executed by a server, and can also be executed by a terminal and a server together. The device for executing the method in the present application can be collectively referred to as a computer device. The server can be an independent physical server, or can be a server cluster or a distributed system composed of at least two physical servers, or can be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be an intelligent vehicle terminal, an intelligent camera, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a screen-equipped sound bar, a smart watch, a smart television, and the like, but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication, and the number of terminals and servers can be one or at least two, which is not limited in the present application.

[0066] It should be noted that the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, and assisted driving. For example, in the intelligent transportation scenario, the multimedia data frame is a video frame or an image frame obtained by a camera device shooting a road. The camera device can be a vehicle terminal or a camera device on the side of the road. By predicting and encoding the multimedia data frame, the encoding performance of the multimedia data frame can be improved, and the road condition can be observed in real time through the encoded multimedia data frame, thereby improving the driving safety of the vehicle.

[0067] Further, please refer to Figure 7 is a flowchart of a multimedia data processing method provided by an embodiment of the present application. As Figure 7 indicated, the method can at least include the following S101-S102:

[0068] S101, obtaining a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; the global reference data block set includes an encoded data block in the multimedia data frame that does not have an adjacent relationship with the to-be-encoded data block, and the local reference data block set includes an encoded data block in the multimedia data frame that has an adjacent relationship with the to-be-encoded data block.

[0069] In the present application, the computer device can acquire a multimedia data frame; the multimedia data frame can be obtained by the computer device, or the multimedia data frame can be obtained by the computer device from the network. The multimedia data frame refers to a video frame to be encoded in video data or an image to be encoded in image data. The video data can refer to live video data, film and television video data, daily life video data, road detection video data, etc. The image data can refer to person image data, building image data, and article image data, etc. Further, the computer device can acquire a global reference data block set associated with the multimedia data frame according to at least one of the encoding mode, the write-back delay, the correlation between the encoded data block and the to-be-encoded data block, and other information, and acquire the global reference data block set and the local reference data block set associated with the multimedia data frame according to at least one of the storage space size for storing the local reference data block set, the correlation between the encoded data block and the to-be-encoded data block, and other information.

[0070] It should be noted that the global reference data block set and the local reference data block set are respectively stored in the first storage space and the second storage space, and the first storage space and the second storage space can be located in the same computer device or different computer devices. The first storage space and the second storage space can specifically refer to a disk, or a storage medium array, etc., which is not limited in the present application. In particular, when the first storage space and the second storage space are located in the same computer device, the first storage space can refer to an off-chip memory of the computer device, and the second storage space can refer to an on-chip memory of the computer device; or the first storage space can refer to an on-chip memory of the computer device, and the second storage space can refer to an off-chip memory of the computer device; which is not limited in the present application.

[0071] S102, from the local reference data block set and the global reference data block set, selecting a reference data block for predicting and encoding the to-be-encoded data block.

[0072] In the present application, the computer device can acquire the global reference data block set and the local reference data block set, which is beneficial to adaptively selecting a reference data block from the global reference data block set and the local reference data block set according to the application scenario or user demand, and predicting and encoding the to-be-encoded data block in the multimedia data frame according to the reference data block; which expands the selection range of the reference data block and improves the encoding performance of the multimedia data frame.

[0073] For example, the computer device can acquire attribute information of the to-be-encoded data block, and select a reference data block for predicting and encoding the to-be-encoded data block from the local reference data block set and the global reference data block set according to the attribute information of the to-be-encoded data block. The attribute information of the to-be-encoded data block includes one or more of a media type of a multimedia data frame to which the to-be-encoded data block belongs, a reference flag of the multimedia data frame, and a definition of the multimedia data frame to which the to-be-encoded data block belongs. The media type includes intra-frame and inter-frame. The intra-frame type can refer to a multimedia data frame that represents a main meaning of the multimedia data, and is a full intra-frame, an Intra frame, an I frame, and a key frame. This type of frame only allows intra-frame encoding technology, including intra-frame block copy mode or intra-frame string copy mode, and does not need to rely on other frames for encoding. For example, the intra-frame type can refer to a multimedia data frame that represents a main meaning of the video data or the image data. The inter-frame type can refer to a multimedia data frame other than the intra-frame type in the video data or the image data, and can be referred to as an inter-frame encoding frame, a P frame, a B frame, and a non-key frame. This type of frame allows inter-frame encoding technology and intra-frame encoding technology, and generally needs to be encoded after the reference frame is encoded when the inter-frame encoding technology is used. The reference flag of the multimedia data frame is used to reflect the selection range of the reference data block of the to-be-encoded data block (i.e., which reference data block set is selected).

[0074] In the present application, the computer device can acquire the global reference data block set and the local reference data block set, which is beneficial to the encoding terminal to adaptively select a reference data block from the global reference data block set and the local reference data block set according to application scenarios or user requirements, and to predict and encode the to-be-encoded data block in the multimedia data frame according to the reference data block. The selection range of the reference data block is expanded, and the encoding performance of the multimedia data frame can be improved.

[0075] Optionally, the computer device can select one of the following two encoding modes according to the encoding mode of the multimedia data frame to acquire the global reference data block set.

[0076] In the first mode, the global reference data block set includes one or more first reference data blocks; when the encoding mode of the multimedia data frame is a serial encoding mode, the first reference data block in the global reference data block set is an encoded data block in the multimedia data frame, which satisfies a write-back restriction condition of a parameter association relationship and belongs to a same encoding slice as the to-be-encoded data block; the parameter association relationship is used to reflect an association relationship among first position information, maximum encoding block size information, and second position information; the first position information and the second position information respectively refer to position information of the encoded data block and the to-be-encoded data block in the multimedia data frame; and the write-back restriction condition is used to reflect a restriction time delay of writing the encoded data block into the global reference data block set. That is, the serial encoding mode refers to predicting and encoding only one data block in the multimedia data frame at the same time, and if the encoding mode of the multimedia data frame is the serial encoding mode, the encoding mode does not affect the determination process of the reference data block. However, writing the encoded data block into the global reference data block set causes a large write-back delay, that is, a long time is consumed for writing the encoded data block into the global reference data block set, and therefore, the encoded data block can be used as a reference data block only after waiting for a preset time length, which is determined according to the restriction time delay of writing the encoded data block into the global reference data block set. The restriction time delay can be one of an average time delay, a maximum time delay, and a minimum time delay of writing the encoded data block into the global reference data block set. Therefore, when the encoding mode of the multimedia data frame is the serial encoding mode, the first reference data block in the global reference data block set needs to satisfy the following two conditions: a, belongs to the same encoding slice as the to-be-encoded data block; and b, the parameter association relationship satisfies the write-back restriction condition.

[0077] Specifically, the computer device can perform the following steps s11-s14 to obtain the global reference data block set.

[0078] s11, when the encoding mode of the multimedia data frame is a serial encoding mode, first position information, maximum encoding block size information, and second position information are respectively obtained; the first position information and the second position information respectively refer to position information of an encoded data block and a to-be-encoded data block in the multimedia data frame.

[0079] s12, a write-back restriction condition is obtained; the write-back restriction condition is used to reflect a restriction time delay of writing the encoded data block into the global reference data block set.

[0080] s13, filtering out, from the multimedia data frame, an encoded data block that satisfies the write-back restriction condition and belongs to the same coding tile as the to-be-encoded data block, as a first reference data block; the parameter association relationship is used to reflect an association relationship between the first position information, the size information of the maximum coding block, and the second position information.

[0081] s14, adding the first reference data block to the global reference data block set.

[0082] In steps s11-s14, when the encoding mode of the multimedia data frame is a serial encoding mode, the computer device can obtain the first position information, the size information of the maximum coding block, and the second position information, respectively. The first position information and the second position information respectively refer to position information of the encoded data block and the to-be-encoded data block in the multimedia data frame, more specifically, the first position information and the second position information can refer to coordinates of pixels of the encoded data block and pixels of the to-be-encoded data block in the multimedia data frame, and the size information of the maximum coding block can refer to a width and a height of the maximum coding block. Further, the computer device can obtain the write-back restriction condition, filter out, from the multimedia data frame, an encoded data block that satisfies the write-back restriction condition and belongs to the same coding tile as the to-be-encoded data block, as a first reference data block, and add the first reference data block to the global reference data block set. By filtering the first reference data block according to the write-back restriction condition, the first position information, the second position information, and the size information of the maximum coding block, it can be ensured that the first reference data block has strong spatial correlation with the to-be-encoded data block, and the availability of the first reference data block can be ensured.

[0083] Optionally, the above step s13 can include steps s21-s24 as follows:

[0084] s21, filtering out, from the multimedia data frame, an encoded data block that belongs to the same coding tile as the to-be-encoded data block, as a first candidate reference data block, according to the first position information and the second position information.

[0085] s22, determining a limit number of interval data blocks between the encoded data block in the multimedia data frame and the to-be-encoded data block under the write-back restriction condition.

[0086] s23, according to the first position information, the size information of the maximum coding block, and the second position information, counting a number of interval data blocks between the first candidate reference data block and the to-be-encoded data block.

[0087] s24, determining, as the first reference data block, the first candidate reference data block whose number of interval data blocks is greater than the limit number of interval data blocks.

[0088] In steps s21-s24, assuming that the first position information and the second position information can be coordinates of pixels of the encoded data block and pixels of the data block to be encoded in the multimedia data frame, the position information of the encoded data block is (ref_x, ref_y), and the position information of the data block to be encoded is (cur_x, cur_y), the computer device can filter out the encoded data block belonging to the same encoding slice as the data block to be encoded from the multimedia data frame as the first candidate reference data block. Further, the interval data block limit number between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back limit condition is determined, which can refer to the minimum number of data blocks that should be spaced between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back limit condition. Then, the interval data block number between the first candidate reference data block and the data block to be encoded is counted according to the first position information, the size information of the maximum encoding block, and the second position information, which refers to the actual number of data blocks spaced between the first candidate reference data block and the data block to be encoded. The first candidate reference data block with an interval data block number greater than the interval data block limit number is determined as the first reference data block. The global reference data block set is obtained through the position information of the encoded data block, the position information of the data block to be encoded, and the write-back delay limit condition, which can ensure that the first reference data block has strong spatial correlation with the data block to be encoded and can ensure the availability of the first reference data block. At the same time, no additional hardware cost is required, which can save costs.

[0089] For example, the computer device can determine the first candidate reference data block satisfying the following formula (2) as the first reference data block:

[0090] D < cur_num - src_num (2)

[0091] In formula (2), cur_num-src_num represents the interval data block number between the first candidate reference data block and the data block to be encoded, cur_num and src_num respectively refer to the number of the first candidate reference data block and the number of the data block to be encoded, and D refers to the interval data block limit number under the write-back limit condition. cur_num and src_num can be calculated through the following formula (3) and formula (4).

[0092] Optionally, the above step s23 can include the following steps s31-s34:

[0093] s31, determining the row identifier and the column identifier of the first candidate reference data block according to the first position information and the size information of the maximum encoding block.

[0094] s32, determining the row identifier and the column identifier of the data block to be encoded according to the second position information and the size information of the maximum coding block.

[0095] s33, determining the number of data blocks per row in the multimedia data frame based on the size information of the data block to be encoded.

[0096] s34, counting the number of interval data blocks between the first candidate reference data block and the data block to be encoded according to the number of data blocks per row, the row identifier and the column identifier of the first candidate reference data block, and the row identifier and the column identifier of the data block to be encoded.

[0097] In steps s31-s34, the row identifier and the column identifier of the first candidate reference data block are ref_y_num and ref_x_num respectively, and the row identifier and the column identifier of the data block to be encoded are cur_x_num and cur_y_num respectively. The computer device can obtain the row identifier and the column identifier of the first candidate reference data block and the row identifier and the column identifier of the data block to be encoded by using the following formula (3) respectively:

[0098]

[0099] In formula (3), Floor() represents rounding down, and W and H can be determined in any of the following four ways: 1. W and H are determined according to the size information of the maximum coding block; 2. W and H are determined according to the size information of VPDU (VPDU, Virtual Pipeline Data Units, basic unit of on-chip memory); 3. H can be determined according to the size information of the maximum coding block, for example, the maximum coding block is 128x128, H is 128, the maximum coding block is 64x64, H is 64, and W can be a default value, such as 64. 4. W and H are both preset values, which can be 4, 64, 128, etc. One maximum coding block can include one or more data blocks, and the size of the maximum coding block and the size of the data block are determined according to the blocking manner of the multimedia data frame.

[0100] Then, the computer device can determine the number of data blocks in each row in the multimedia data frame based on the size information of the data block to be encoded, determine the number of the first candidate reference data block according to the number of data blocks in each row, the row identifier and the column identifier of the first candidate reference data block, and determine the number of the data block to be encoded according to the number of data blocks in each row and the row identifier and the column identifier of the data block to be encoded, and determine the number of interval data blocks between the first candidate reference data block and the data block to be encoded as the difference value between the number of the first candidate reference data block and the number of the data block to be encoded. For example, the computer device can determine the number of data blocks in each row in the multimedia data frame, the number of the first candidate reference data block, and the number of the data block to be encoded by using the following formula (4):

[0101]

[0102] In formula (4), stride refers to the number of data blocks in each row in the multimedia data frame, cur_num and src_num respectively refer to the number of the first candidate reference data block and the number of the data block to be encoded, Ceil() represents rounding up, tile_w represents the width of the coding tile, and the coding tile includes one or more maximum coding blocks, so the width of the coding tile can be determined according to the size information of the maximum coding block. For example, when the size of the maximum coding block is 128x128, the coding tile can be 1024x720.

[0103] For example, the first position information and the second position information are respectively the coordinates of the coded data block and the data block to be encoded in the multimedia data frame; when the coding mode of the multimedia data frame is a serial coding mode, the size of the maximum coding block is 128x128, the width and height of the coding tile are respectively 1024 and 720, the values of W and H are both equal to 64, and the number of interval data blocks between the coded data block and the data block to be encoded is limited to 4 under the write-back restriction condition, the first reference data block in the global reference data block set is the coded data block that belongs to the same coding tile as the data block to be encoded and satisfies the following formula (5).

[0104] (Floor(ref_y / 128) * 16 + Floor(ref_x / 64)) <(Floor(cur _y / 128) *16 + Floor(cur_x / 64) -4 (5)

[0105] Wherein, (ref_x, ref_y), (cur_x, cur_y) respectively represent the coordinates of the coded data block, the data block to be coded in the multimedia data frame, Floor() represents down rounding.

[0106] For another example, the first position information and the second position information are respectively the coordinates of the coded data block, the data block to be coded in the multimedia data frame; when the encoding mode of the multimedia data frame is a serial encoding mode, the size of the maximum coding block is 64x64, the width and height of the encoding slice are respectively 1024 and 720, the values of W and H are both equal to 64, and the interval data block limit quantity between the coded data block and the data block to be coded under the write-back restriction condition is 4, the first reference data block in the global reference data block set is the coded data block belonging to the same encoding slice as the data block to be coded, and the coordinates satisfy the following formula (6).

[0107] (Floor(ref_y / 64) * 16 + Floor(ref_x / 64) < (Floor(cur_y / 64) * 16 + Floor(cur_x / 64) - 4 (6)

[0108] Wherein, (ref_x, ref_y), (cur_x, cur_y) respectively represent the coordinates of the coded data block, the data block to be coded in the multimedia data frame, Floor() represents down rounding.

[0109] It should be noted that when the encoding mode of the multimedia data frame is a serial encoding mode, the computer device can also obtain the global reference data block set by the following manner: according to the first position information, the second position information and the size information of the maximum coding block, determining the interval distance between the coded data block and the data block to be coded, and determining the interval limit distance between the coded data block and the data block to be coded under the write-back delay condition, the interval limit distance refers to the minimum interval distance between the coded data block and the data block to be coded under the write-back delay condition, the coded data block belonging to the same encoding slice as the data block to be coded in the multimedia data frame and having an interval distance greater than the interval limit distance is taken as the first reference data block, and the first reference data block is added to the global reference data block set.

[0110] Optionally, the adjacent relationship between the data block to be coded and the coded data block can mean that the coding order of the data block to be coded is adjacent to the coding order of the coded data block; and the non-adjacent relationship between the data block to be coded and the coded data block can mean that the coding order of the data block to be coded is not adjacent to the coding order of the coded data block.

[0111] In the second mode, the global reference data block set includes one or more second reference data blocks. When the encoding mode of the multimedia data frame is a parallel encoding mode or a serial encoding mode, the second reference data block of the global reference data block set is an encoded data block that belongs to the same encoding tile as the to-be-encoded data block, and that satisfies a write-back limitation condition and an encoding processing condition. The parameter association relationship reflects the association relationship among the first position information, the size information of the to-be-encoded data block, and the second position information. The write-back limitation condition reflects the limitation time delay of writing the encoded data block into the global reference data block set. That is, the parallel encoding mode refers to predicting and encoding multiple data blocks in a multimedia data frame at the same time. If the encoding mode of the multimedia data frame is the parallel encoding mode, the encoding mode will affect the determination process of the reference data block. Meanwhile, writing the encoded data block into the global reference data block set will cause a large write-back delay. Therefore, when the encoding mode of the multimedia data frame is the parallel encoding mode, the second reference data block in the global reference data block set needs to satisfy the following three conditions: a, belonging to the same encoding tile as the to-be-encoded data block; b, the parameter association relationship satisfying the write-back limitation condition; and c, the parameter association relationship satisfying the encoding processing condition. In particular, when the encoding mode of the multimedia data frame is the serial encoding mode, the second mode can also be used to obtain the global reference data block set. Here, the encoding processing condition can be a parallel encoding processing condition.

[0112] Specifically, the computer device can perform the following steps s41-s44 to obtain the global reference data block set:

[0113] s41, respectively obtaining the first position information, the size information of the maximum encoding block, and the second position information. The first position information and the second position information respectively refer to the position information of the encoded data block and the to-be-encoded data block in the multimedia data frame.

[0114] s42, obtaining the write-back limitation condition and the encoding processing condition. The write-back limitation condition reflects the limitation time delay of writing the encoded data block into the global reference data block set.

[0115] s43, selecting, from the multimedia data frame, an encoded data block that satisfies the write-back limitation condition and the encoding processing condition and belongs to the same encoding tile as the to-be-encoded data block as the second reference data block. The parameter association relationship reflects the association relationship among the first position information, the size information of the to-be-encoded data block, and the second position information.

[0116] s44, adding the second reference data block to the global reference data block set.

[0117] In steps s41-s44, the first position information and the second position information can be coordinates of pixels of the encoded data block and pixels of the data block to be encoded in the multimedia data frame, and the size information of the maximum coding block can be width and height of the maximum coding block. The write-back restriction condition is used to reflect a restriction delay of writing the encoded data block into the global reference data block set, and the coding processing condition is used to reflect a restriction distance between the reference data block and the data block to be encoded when encoding the multimedia data frame in parallel; or, the coding processing condition is used to reflect a number of interval data blocks between the reference data block and the data block to be encoded when encoding the multimedia data frame in parallel. By screening the encoded data block that satisfies the write-back restriction condition and the coding processing condition and belongs to the same coding slice as the data block to be encoded from the multimedia data frame, as the second reference data block, and adding the second reference data block to the global reference data block set, it can be ensured that the second reference data block has strong spatial correlation with the data block to be encoded, and the availability of the second reference data block can be ensured.

[0118] Optionally, the above step s43 can include steps s51-s55 as follows:

[0119] s51, according to the first position information and the second position information, screening the encoded data block that belongs to the same coding slice as the data block to be encoded from the multimedia data frame, as the second candidate reference data block.

[0120] s52, screening the first candidate reference data block that satisfies the write-back restriction condition from the first candidate reference data block, as the second candidate reference data block.

[0121] s53, determining a first data block restriction distance between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back restriction condition.

[0122] s54, determining a second data block restriction distance between the encoded data block in the multimedia data frame and the data block to be encoded under the coding processing condition.

[0123] s55, according to the first position information, the size information of the maximum coding block, the second position information, the first data block restriction distance, and the second data block restriction distance, determining the second candidate reference data block that satisfies the coding processing condition from the second candidate reference data block, as the second reference data block.

[0124] In steps s51-s55, it is assumed that the first position information and the second position information can refer to the coordinates of the pixels of the encoded data block and the pixels of the data block to be encoded in the multimedia data frame, the position information of the encoded data block is (ref_x, ref_y), and the position information of the data block to be encoded is (cur_x, cur_y). The computer device can filter out the encoded data block belonging to the same encoding slice as the data block to be encoded from the multimedia data frame as the first candidate reference data block. Further, the computer device can filter out the first candidate reference data block satisfying the above formula (4) from the first candidate reference data block as the second candidate reference data block. Further, the first data block limiting distance between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back limiting condition can be determined, which can refer to the minimum distance that should be maintained between the encoded data block in the multimedia data frame and the data block to be encoded under the write-back delay limiting condition; the second data block limiting distance between the encoded data block in the multimedia data frame and the data block to be encoded under the encoding processing condition can be determined, which can refer to the minimum distance that should be maintained between the encoded data block in the multimedia data frame and the data block to be encoded under the encoding processing condition. Then, the second candidate reference data block satisfying the encoding processing condition can be determined from the second candidate reference data block as the second reference data block according to the first position information, the size information of the maximum encoding block, and the second position information, the first data block limiting distance, and the second data block limiting distance.

[0125] Optionally, the above step s55 can include steps s61-s66 as follows:

[0126] s61, determining the row identifier and the column identifier of the second candidate reference data block according to the first position information and the size information of the maximum encoding block.

[0127] s62, determining the row identifier and the column identifier of the data block to be encoded according to the second position information and the size information of the maximum encoding block.

[0128] s63, filtering out the second candidate reference data block with the row identifier less than or equal to the row identifier of the data block to be encoded from the second candidate reference data block as the third candidate reference data block.

[0129] s64, determining the row distance between the third candidate reference data block and the data block to be encoded according to the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded.

[0130] s65, determining a column distance between the third candidate reference data block and the data block to be encoded according to the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded.

[0131] s66, determining the third candidate reference data block in which the column distance is less than a first distance threshold, as the second reference data block; the first distance threshold is determined according to the first data block limit distance, the second data block limit distance, and a row distance between the third candidate reference data block and the data block to be encoded.

[0132] In steps s61-s66, the computer device can respectively obtain the row identifier and the column identifier of the second candidate reference data block, the row identifier and the column identifier of the data block to be encoded by using the above formula (5), and then can screen the second candidate reference data block in which the row identifier is less than or equal to the row identifier of the data block to be encoded from the second candidate reference data block as the third candidate reference data block. Further, the difference between the row identifier of the third candidate reference data block and the row identifier of the data block to be encoded can be determined as the row distance between the third candidate reference data block and the data block to be encoded, and the difference between the column identifier of the third candidate reference data block and the column identifier of the data block to be encoded can be determined as the column distance between the third candidate reference data block and the data block to be encoded; and the third candidate reference data block in which the column distance is less than the first distance threshold can be determined as the second reference data block. For example, the computer device can determine the second candidate reference data block in which the coordinates satisfy the following formula (7) as the second reference data block:

[0133]

[0134] In the formula (7), ref_y_num and ref_x_num respectively represent the row identifier and the column identifier of the second candidate reference data block, cur_y_num and cur_x_num respectively represent the row identifier and the column identifier of the data block to be encoded, P represents the first data block limiting distance between the data block already encoded in the multimedia data frame under the write-back limiting condition and the data block to be encoded, and E represents the second data block limiting distance between the data block already encoded in the multimedia data frame under the encoding processing condition and the data block to be encoded. In particular, the first data block limiting distance can be determined according to the interval limiting data block quantity between the data block already encoded in the multimedia data frame under the write-back limiting condition and the data block to be encoded, for example, P can be the same in value as D; the second data block limiting distance is determined according to the interval limiting data block quantity between the data block already encoded in the multimedia data frame under the encoding processing condition and the data block to be encoded. ref_x_num-cur_x_num represents the column distance between the third candidate reference data block and the data block to be encoded, and cur_y_num-ref_y_num represents the row distance between the third candidate reference data block and the data block to be encoded.

[0135] For example, the first position information and the second position information are respectively the coordinates of the data block already encoded and the data block to be encoded in the multimedia data frame; when the encoding mode of the multimedia data frame is a parallel encoding mode or a serial encoding mode, the size of the maximum encoding block is 128x128, the width and height of the encoding slice are respectively 1024 and 720, the values of W and H are both equal to 64, the interval data block limiting distance between the data block already encoded and the data block to be encoded under the write-back limiting condition is 4, and the interval data block limiting distance between the data block already encoded and the data block to be encoded under the encoding processing condition is 2, the second reference data block in the global reference data block set is the data block already encoded which belongs to the same encoding slice as the data block to be encoded and satisfies the following formula (8):

[0136]

[0137] In the formula (8), (ref_x, ref_y) and (cur_x, cur_y) respectively represent the coordinates of the data block already encoded and the data block to be encoded in the multimedia data frame, and Floor() represents the down rounding.

[0138] For another example, the first position information and the second position information are respectively coordinates of the coded data block and the data block to be coded in the multimedia data frame; when the coding mode of the multimedia data frame is a parallel coding mode or a serial coding mode, the size of the maximum coding block is 64x64, the width and height of a coding slice are respectively 1024 and 720, the values of W and H are both equal to 64, and the interval data block limit distance between the coded data block and the data block to be coded under the write-back restriction condition is 4, and the interval data block limit distance between the coded data block and the data block to be coded under the coding processing condition is 1, the second reference data block in the global reference data block set is a coded data block belonging to the same coding slice as the data block to be coded and satisfying the following formula (9):

[0139]

[0140] Wherein, (ref_x, ref_y) and (cur_x, cur_y) respectively represent the coordinates of the coded data block and the data block to be coded in the multimedia data frame, and Floor() represents down rounding.

[0141] It should be noted that when the coding mode of the multimedia data frame is a parallel coding mode or a serial coding mode, the computer device can also obtain the global reference data block set by the following method: according to the first position information, the second position information and the size information of the maximum coding block, determining the number of interval data blocks between the coded data block and the data block to be coded, and determining the target interval limit data block number between the coded data block and the data block to be coded under the write-back delay condition and the coding processing condition, the target interval limit data block number refers to the minimum number of interval data blocks between the coded data block and the data block to be coded under the write-back delay condition and the coding processing condition, the coded data block belonging to the same coding slice as the data block to be coded and having a number of interval data blocks greater than the target interval limit data block number in the multimedia data frame is taken as the second reference data block, and the second reference data block is added to the global reference data block set.

[0142] Optionally, the computer device can select one or at least two combination ways from the following two ways to obtain the local reference data block set:

[0143] The first manner is that the local reference data block set includes one or more third reference data blocks, and the third reference data block of the local reference data block set is an encoded data block in the multimedia data frame which is located in the same encoding processing unit as the to-be-encoded data block. The encoding processing unit can also be referred to as a basic unit (VPDU, Virtual Pipeline Data Units) of on-chip memory. The size information of the VPDU can be determined according to the encoding processing capability of the encoder, for example, the size of the VPDU is VPDU_WxVPDU_H, and the values of VPDU_W and VPDU_H can be 4, 8, 16, 32, 64, 128, etc. That is, the encoded data block in the multimedia data frame which satisfies ref_x / VPDU_W=cur_x / VPDU_W and ref_y / VPDU_H=cur_y / VPDU_H is taken as the second reference data block. For another example, if the size of the VPDU is 64x64, the encoded data block in the VPDU 64x64 to which the to-be-encoded data block belongs can be taken as the reference data block of the to-be-encoded data block. Or, if the size of the VPDU is 64x64, the encoded data block in the multimedia data frame which satisfies Floor(ref_x / 64)=Floor(cur_x / 64) and Floor(ref_y / 64)=Floor(cur_y / 64) is taken as the second reference data block.

[0144] The second manner is that the local reference data block set includes one or more fourth reference data blocks, and the fourth reference data block of the local reference data block set is an encoded data block in the multimedia data frame which has a neighboring relationship with the to-be-encoded data block and belongs to the same encoding slice as the to-be-encoded data block. The neighboring relationship between the fourth reference data block and the to-be-encoded data block can include one or more combined cases of the following three cases:

[0145] a. The fourth reference data block has a neighboring relationship with the to-be-encoded data block, which means that the fourth reference data block and the to-be-encoded data block belong to the same maximum coding block.

[0146] b、the fourth reference data block has a neighboring relationship with the data block to be encoded, which means that the fourth reference data block is located in N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is less than a size threshold; N is determined according to the size information of the maximum coding block. For example, the encoding direction of the multimedia data frame is from left to right, and the size threshold can be dynamically set according to the application scenario or user demand, for example, the size threshold can be 128x128, and the fourth reference data block has a neighboring relationship with the data block to be encoded, which means that the fourth reference data block is located in N maximum coding blocks adjacent to the left side of the data block to be encoded, and the size of the maximum coding block is less than 128x128. N can be determined according to the following formula (10):

[0147]

[0148] wherein log2 sb size is an intermediate variable, and sb size is the size of the maximum coding block.

[0149] c、the fourth reference data block has a neighboring relationship with the data block to be encoded, which means that the fourth reference data block is located in N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is greater than or equal to a size threshold; the data block located at the top left corner in the area of the multimedia data frame where the fourth reference data block moves one maximum coding block to the right is an unreconstructed data block; the position information of the fourth reference data block when it moves one maximum coding block (the moving direction is the same as the encoding direction) is different from the position information of the data block to be encoded; N is determined according to the size information of the maximum coding block. That is, when the size of the maximum coding block is greater than or equal to the size threshold, for example, the size of the maximum coding block is 128x128, the fourth reference data block needs to satisfy the following three conditions: 1, the fourth reference data block is an already encoded data block belonging to the same encoding slice as the data block to be encoded; 2, the coordinate of the fourth reference data block when it moves 128 pixels to the right is (ref x+128, ref y), and the data block with the coordinate (ref x+128, ref y) in the multimedia data frame is an unreconstructed data block; 3, the position information of the fourth reference data block when it moves one maximum coding block to the right is different from the position information of the data block to be encoded, which can be expressed by formula (11):

[0150]

[0151] wherein ≠ in formula (11) means not equal.

[0152] Optionally, distances between the first reference data block, the second reference data block, the third reference data block, the fourth reference data block and the data block to be encoded are all less than a second distance threshold; the second distance threshold can be dynamically set according to application scenarios or user requirements, for example, the second distance threshold can be 2 14 Meanwhile, the first reference data block, the second reference data block, the third reference data block and the fourth reference data block are located at integral pixel positions of the multimedia data frame.

[0153] For example, when the encoding mode of the multimedia data frame is a parallel encoding mode, the size of the maximum encoding block is 128*128, the interval between the encoded data block and the data block to be encoded is limited to 4 data blocks under the write-back delay restriction condition, the interval between the encoded data block and the data block to be encoded is limited to 2 data blocks under the encoding processing condition, the acquisition mode of the local reference data block set is the above-described mode one, and the size of the VPDU is 64x64, the relationship between the global reference data block set, the local reference data block set and the data block to be encoded is as shown in Figure 8 . Figure 8 In the above-described mode one, the size of one minimum square (i.e., the square drawn by the dashed line) is 64x64, the size of the square drawn by the solid line is 128x128, and the size of the data block is 32x32. Therefore, Figure 8 In the above-described mode one, the encoded data block located in the 19th square in the first row of the multimedia data frame belongs to the local reference data block set, and the local reference data block set is used for encoding the data block to be encoded in the 19th square in the first row. The encoded data blocks in the first row and the second row of the multimedia data frame, which are 8 data blocks away from the data block to be encoded, belong to the global reference data block set, and the global reference data block set is used for encoding the data block to be encoded in the 19th square in the first row. It can be known from Figure 8 that the number of reference data blocks in the global reference data block set is greater than the number of reference data blocks in the local reference data block set, the reference data blocks in the global reference data block set do not have an adjacent relationship with the data block to be encoded, and the data blocks in the local reference data block set have an adjacent relationship with the data block to be encoded.

[0154] For example, when the encoding mode of the multimedia data frame is a parallel encoding mode, the size of the maximum encoding block is 128*128, the interval between the encoded data block and the data block to be encoded is limited to 4 data blocks under the write-back delay restriction condition, the interval between the encoded data block and the data block to be encoded is limited to 2 data blocks under the encoding processing condition, the acquisition mode of the local reference data block set is the above-described mode two, and the size of the VPDU is 64x64, the relationship between the global reference data block set, the local reference data block set and the data block to be encoded is as shown in Figure 9 . Figure 9One of the smallest squares (i.e. the small square drawn by the dashed line) has a size of 64x64, the small square drawn by the solid line has a size of 128x128, and the data block has a size of 32x32. Therefore, Figure 9 In the multimedia data frame, the coded data block in the 19th square of the first row and the coded data block in the largest coding block on the left of the to-be-coded data block belong to the local reference data block set, and the local reference data block set is used for coding the to-be-coded data block in the 19th square of the first row. The coded data blocks in the first row and the second row of the multimedia data frame, which are 8 data blocks away from the to-be-coded data block, belong to the global reference data block set, and the global reference data block set is used for coding the to-be-coded data block in the 19th square of the first row. From Figure 9 It can be seen that the number of reference data blocks in the global reference data block set is greater than the number of reference data blocks in the local reference data block set, the reference data blocks in the global reference data block set do not have an adjacent relationship with the to-be-coded data block, and the data blocks in the local reference data block set have an adjacent relationship with the to-be-coded data block.

[0155] Optionally, for any multimedia data frame, the computer device can select a reference data block for coding the to-be-coded data block from the global reference data block set and the local reference data block set. Alternatively, the computer device can adaptively select a reference data block set according to the coding mode of the to-be-coded data block, and determine the reference data block for coding the to-be-coded data block from the selected reference data block set. Specifically, the computer device can obtain the coding mode of the to-be-coded data block; if the coding mode of the to-be-coded data block is the intra block copy mode or the intra string copy mode, the computer device selects a reference data block for predicting coding the to-be-coded data block from the global reference data block set and the local reference data block set. The coding mode of the to-be-coded data block can be determined according to the media type of the multimedia data frame, or the coding mode of the to-be-coded data block can be determined according to the coding flag (or reference flag) of the multimedia data frame. In particular, the coding mode of the to-be-coded data block can be the prediction mode of the to-be-coded data block.

[0156] Optionally, if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, the selecting, from the global reference data block set and the local reference data block set, the reference data block used for the predictive encoding of the to-be-encoded data block comprises: if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, the second reference data block in the local reference data block set has a stronger correlation with the to-be-encoded data block due to the adjacent relationship between the local reference data block and the to-be-encoded data block, and therefore, the computer device can preferentially search the local reference data block set, and if a reference data block meeting the condition is searched in the local reference data block set, there is no need to search the global reference data block set, and if a reference data block meeting the condition is not searched in the local reference data block set, the global reference data block set is searched again, which is beneficial to improving the efficiency of searching the reference data block and avoiding unnecessary waste of resources. For example, first, the computer device can obtain the pixel difference between the second reference data block in the local reference data block set and the to-be-encoded data block, where the pixel difference can be the sum of the mean square error or the absolute difference between the pixels of the second reference data block and the pixels of the to-be-encoded data block; if there is a target data block with a pixel difference less than a pixel difference threshold in the local reference data block set, it indicates that the target data block has a stronger correlation with the to-be-encoded data block, and the target data block is selected as the reference data block used for the predictive encoding of the to-be-encoded data block; if there is no target data block with a pixel difference less than the pixel difference threshold in the local reference data block set, it indicates that the second reference data block in the local reference data block set has a weaker correlation with the to-be-encoded data block, and the reference data block used for the predictive encoding of the to-be-encoded data block is selected from the global reference data block set.

[0157] Optionally, if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, the computer device selects the reference data block for the prediction encoding of the to-be-encoded data block from the global reference data block set and the local reference data block set, including: if the encoding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, the computer device can determine the reference data block according to the target encoded data block having the spatial adjacent relationship with the to-be-encoded data block. Here, the target encoded data block having the spatial adjacent relationship can refer to all the encoded data blocks having the position adjacent relationship with the to-be-encoded data block, or the target encoded data block having the spatial adjacent relationship can refer to the encoded data block having the position adjacent relationship with the to-be-encoded data block and having the strong correlation with the pixels of the to-be-encoded data block. Specifically, the computer device can obtain the target encoded data block having the spatial adjacent relationship with the to-be-encoded data block; if the reference data block of the target encoded data block belongs to the global reference data block set and the local reference data block set, the computer device performs the fine search in the regions corresponding to the global reference data block set and the local reference data block set according to the movement parameter, to obtain the reference data block for the prediction encoding of the to-be-encoded data block; at this time, the reference data block of the to-be-encoded data block belongs to the global reference data block set or belongs to the local reference data block set, or the first region of the reference data block of the to-be-encoded data block belongs to the global reference data block set and the second region of the reference data block of the to-be-encoded data block belongs to the local reference data block set, the data block where the first region is located has the position adjacent relationship with the data block where the second region is located. By searching the reference data block according to the movement parameter, the optimal reference data block can be searched, and the encoding performance can be improved. If the reference data block of the target encoded data block all belongs to the global reference data block set, the computer device skips the local reference data block set and performs the fine search in the region corresponding to the global reference data block set according to the movement parameter, to obtain the reference data block for the prediction encoding of the to-be-encoded data block; if the reference data block of the target encoded data block all belongs to the local reference data block set, the computer device skips the global reference data block set and performs the fine search in the region corresponding to the local reference data block set according to the movement parameter, to obtain the reference data block for the prediction encoding of the to-be-encoded data block.

[0158] The movement parameter includes a movement window and a movement step, the size of the movement window is the same as the size of the to-be-encoded data block, and the movement step is smaller than the width of the movement window.

[0159] Further, please refer to Figure 10 is a flowchart of a multimedia data processing method provided by the embodiment of the present application. As shown in Figure 10As shown, the method can at least include the following S201-S202:

[0160] S201, obtaining a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; the global reference data block set includes decoded data blocks in the multimedia data frame that have no adjacent relationship with the to-be-decoded data block, and the local reference data block set includes decoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-decoded data block.

[0161] In this application, the computer device can obtain a multimedia data frame; the multimedia data frame can be obtained by the computer device, or the multimedia data frame can be downloaded by the computer device from the network. The multimedia data frame refers to a to-be-decoded video frame in video data or a to-be-decoded image in image data. Video data can refer to live video data, movie video data, daily life video data, road detection video data, etc. Image data can refer to person image data, building image data, and object image data, etc. Further, the computer device can obtain a global reference data block set associated with the multimedia data frame according to at least one of the decoding mode, the write-back delay, the correlation between the decoded data block and the to-be-decoded data block, etc. And obtain a global reference data block set and a local reference data block set associated with the multimedia data frame according to at least one of the storage space size for storing the local reference data block set, the correlation between the decoded data block and the to-be-decoded data block, etc.

[0162] It should be noted that the global reference data block set and the local reference data block set are respectively stored in the first storage space and the second storage space, and the first storage space and the second storage space can be located in the same computer device or in different computer devices. The first storage space and the second storage space can be a disk or a storage medium array, etc. The application does not limit this. In particular, when the first storage space and the second storage space are located in the same computer device, the first storage space can be an off-chip memory of the computer device, and the second storage space can be an on-chip memory of the computer device; or the first storage space can be an on-chip memory of the computer device, and the second storage space can be an off-chip memory of the computer device; the application does not limit this.

[0163] S202, selecting a reference data block for decoding the to-be-decoded data block from the local reference data block set and the global reference data block set.

[0164] In the present application, the computer device can select reference data blocks from the global reference data block set and the local reference data block set according to application scenarios or user requirements, and decode the to-be-decoded data blocks in the multimedia data frame according to the reference data blocks; the selection range of the reference data blocks is expanded, and the decoding performance of the multimedia data frame can be improved.

[0165] For example, the computer device can obtain attribute information of the to-be-decoded data blocks, and select reference data blocks for decoding the to-be-decoded data blocks from the global reference data block set and the local reference data block set according to the attribute information of the to-be-decoded data blocks. The attribute information of the to-be-decoded data blocks includes one or more of the media type of the multimedia data frame to which the to-be-decoded data blocks belong, the reference flag of the multimedia data frame, the definition of the multimedia data frame to which the to-be-decoded data blocks belong, and the like. The media type includes intra-frame and inter-frame. The intra-frame type can refer to a representative multimedia data frame, and the intra-frame type is an all-intra decoding frame, an Intra frame, an I frame, and a key frame. This type of frame only allows the use of intra-frame decoding technology and does not need to rely on other frames for decoding. For example, the intra-frame type can refer to a multimedia data frame that can represent the main meaning in video data or image data. The inter-frame type can refer to a multimedia data frame other than the intra-frame type in video data or image data, and the inter-frame type can be referred to as an inter-frame decoding frame, a P frame, a B frame, and a non-key frame. This type of frame allows the use of inter-frame decoding technology and intra-frame decoding technology, and generally requires decoding after the reference frame decoding is completed when the inter-frame decoding technology is used. The reference flag of the multimedia data frame is used to reflect the selection range of the reference data blocks of the to-be-decoded data blocks (i.e., which reference data block set is selected).

[0166] In the present application, the computer device can obtain the global reference data block set and the local reference data block set, which is conducive to selecting reference data blocks from the global reference data block set and the local reference data block set according to application scenarios or user requirements, and decoding the to-be-decoded data blocks in the multimedia data frame according to the reference data blocks; the selection range of the reference data blocks is expanded, and the decoding performance of the multimedia data frame can be improved.

[0167] Optionally, the computer device can select one of the following two ways to obtain the global reference data block set according to the decoding mode of the multimedia data frame:

[0168] In the first mode, the global reference data block set includes one or more first reference data blocks; when the decoding mode of the multimedia data frame is a serial decoding mode, the first reference data blocks in the global reference data block set are decoded data blocks in the multimedia data frame that satisfy a write-back restriction condition of a parameter association relationship and belong to the same decoding tile as the to-be-decoded data block; the parameter association relationship reflects an association relationship between first position information, maximum decoding block size information, and second position information; the first position information and the second position information are position information of the decoded data block and the to-be-decoded data block in the multimedia data frame, respectively; and the write-back restriction condition reflects a restriction delay of writing the decoded data block into the global reference data block set. In other words, the serial decoding mode refers to decoding only one data block in the multimedia data frame at the same time, and if the decoding mode of the multimedia data frame is the serial decoding mode, the decoding mode does not affect the determination process of the reference data block. However, writing the decoded data block into the global reference data block set causes a large write-back delay, that is, a long time is consumed for writing the decoded data block into the global reference data block set, and therefore, the decoded data block can be used as a reference data block only after waiting for a preset time period, which is determined according to the restriction delay of writing the decoded data block into the global reference data block set. The restriction delay can be one of an average delay, a maximum delay, and a minimum delay of writing the decoded data block into the global reference data block set. Therefore, when the decoding mode of the multimedia data frame is the serial decoding mode, the first reference data blocks in the global reference data block set need to satisfy the following two conditions: a, belong to the same decoding tile as the to-be-decoded data block; and b, satisfy the write-back restriction condition of the parameter association relationship.

[0169] Option two: the global reference data block set includes one or more second reference data blocks; when the decoding mode of the multimedia data frame is a parallel decoding mode or a serial decoding mode, the second reference data block of the global reference data block set is a decoded data block belonging to the same decoding tile as the to-be-decoded data block, and the parameter association relationship in the multimedia data frame satisfies a write-back limitation condition and a decoding processing condition; the parameter association relationship reflects the association relationship between the position information of the decoded data block and the to-be-decoded data block in the multimedia data frame and the size information of the maximum decoding block; the write-back limitation condition reflects the limitation delay of writing the decoded data block into the global reference data block set. That is, the parallel decoding mode refers to predicting and decoding multiple data blocks in a multimedia data frame at the same time, and if the decoding mode of the multimedia data frame is the parallel decoding mode, the decoding mode will affect the determination process of the reference data block. At the same time, writing the decoded data block into the global reference data block set will bring a large write-back delay, so when the decoding mode of the multimedia data frame is the parallel decoding mode, the second reference data block in the global reference data block set needs to satisfy the following three conditions: a, belonging to the same decoding tile as the to-be-decoded data block; b, the parameter association relationship satisfies the write-back limitation condition; c, the parameter association relationship satisfies the decoding processing condition. In particular, when the decoding mode of the multimedia data frame is the serial decoding mode, option two can also be used to obtain the global reference data block set. Here, the decoding processing condition can be a parallel decoding processing condition.

[0170] It should be noted that the specific obtaining method of the global reference data block set can refer to the obtaining method of the global reference data block set at the encoding end, and the repeated parts will not be described herein.

[0171] Optionally, the adjacent relationship between the to-be-encoded data block and the encoded data block can mean that the encoding order of the to-be-encoded data block is adjacent to the encoding order of the encoded data block; and the non-adjacent relationship between the to-be-encoded data block and the encoded data block can mean that the encoding order of the to-be-encoded data block is not adjacent to the encoding order of the encoded data block.

[0172] Optionally, the computer device can select one of the following two options or at least two combinations to obtain the local reference data block set:

[0173] The first manner is that the local reference data block set includes one or more third reference data blocks; the third reference data block of the local reference data block set is a decoded data block in the multimedia data frame which is located in the same decoding processing unit as the to-be-decoded data block. The decoding processing unit can also be referred to as a basic unit (VPDU, Virtual Pipeline Data Units) of on-chip memory. The size information of the VPDU can be determined according to the decoding processing capability of the decoder, for example, the size of the VPDU is VPDU_WxVPDU_H, and the values of VPDU_W and VPDU_H can be 4, 8, 16, 32, 64, 128, etc. That is, the decoded data block in the multimedia data frame which satisfies ref_x / VPDU_W=cur_x / VPDU_W and ref_y / VPDU_H=cur_y / VPDU_H is taken as the third reference data block. For another example, if the size of the VPDU is 64x64, the decoded data block in the VPDU 64x64 to which the to-be-decoded data block belongs can be taken as the reference data block of the to-be-decoded data block. Or, if the size of the VPDU is 64x64, the decoded data block in the multimedia data frame which satisfies Floor(ref_x / 64)=Floor(cur_x / 64) and Floor(ref_y / 64)=Floor(cur_y / 64) is taken as the third reference data block.

[0174] The second manner is that the local reference data block set includes one or more fourth reference data blocks; the fourth reference data block of the local reference data block set is a decoded data block in the multimedia data frame which has a neighboring relationship with the to-be-decoded data block and belongs to the same decoding tile as the to-be-decoded data block. The neighboring relationship between the fourth reference data block and the to-be-decoded data block can include one or more combined cases of the following three cases:

[0175] a. The fourth reference data block has a neighboring relationship with the to-be-decoded data block, which means that the fourth reference data block and the to-be-decoded data block belong to the same maximum decoding block.

[0176] b. The fourth reference data block has a neighboring relationship with the to-be-decoded data block, which means that the fourth reference data block is located in N maximum decoding blocks adjacent to the to-be-decoded data block, and the size of the maximum decoding block is smaller than a size threshold; N is determined according to the size information of the maximum decoding block.

[0177] It should be noted that the size of the maximum coding block is the same as the size of the maximum decoding block, the size of the to-be-encoded data block is the same as the size of the to-be-decoded data block, and the size of the coding tile is the same as the size of the decoding tile.

[0178] Optionally, distances between the first reference data block, the second reference data block, the third reference data block, the fourth reference data block and the to-be-decoded data block are all less than a second distance threshold; the second distance threshold can be dynamically set according to application scenarios or user requirements, for example, the second distance threshold can be 2 14 Meanwhile, the first reference data block, the second reference data block, the third reference data block and the fourth reference data block are located at integral pixel positions of the multimedia data frame.

[0179] Optionally, for any multimedia data frame, the computer device can select a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set. Alternatively, the computer device can adaptively select a reference data block set according to a decoding mode of the to-be-decoded data block, and determine a reference data block for decoding the to-be-decoded data block from the selected reference data block set. Specifically, the computer device can obtain the decoding mode of the to-be-decoded data block; if the decoding mode of the to-be-decoded data block is an intra block copy mode or an intra string copy mode, a reference data block for decoding the to-be-decoded data block is selected from the global reference data block set and the local reference data block set. The decoding mode of the to-be-decoded data block can be determined according to the media type of the multimedia data frame, or the decoding mode of the to-be-decoded data block can be determined according to a decoding flag (or a reference flag) of the multimedia data frame. In particular, the decoding mode of the to-be-decoded data block can be a prediction mode of the to-be-decoded data block.

[0180] Please refer to Figure 11 is a structural schematic diagram of a multimedia data processing apparatus provided by an embodiment of the present application. The multimedia data processing apparatus can be a computer program (including program code) running in a computer device, for example, the multimedia data processing apparatus is an application software; the apparatus can be used to execute corresponding steps in the method provided by the embodiments of the present application. As shown in Figure 11 The multimedia data processing apparatus can include an obtaining module 111 and a selecting module 112.

[0181] The obtaining module is configured to obtain a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; the global reference data block set includes already-encoded data blocks in the multimedia data frame that do not have an adjacent relationship with the to-be-encoded data block, and the local reference data block set includes already-encoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-encoded data block.

[0182] The selecting module is configured to select a reference data block from the global reference data block set and the local reference data block set for predictive encoding of the data block to be encoded.

[0183] Optionally, the selecting module is configured to select a reference data block from the global reference data block set and the local reference data block set for predictive encoding of the data block to be encoded, including:

[0184] obtaining an encoding mode of the data block to be encoded;

[0185] If the encoding mode of the data block to be encoded is the intra block copy mode or the intra string copy mode, the selecting module is configured to select a reference data block from the global reference data block set and the local reference data block set for predictive encoding of the data block to be encoded.

[0186] Optionally, the selecting module is configured to select a reference data block from the global reference data block set and the local reference data block set for predictive encoding of the data block to be encoded, including:

[0187] If the encoding mode of the data block to be encoded is the intra block copy mode or the intra string copy mode, the selecting module is configured to obtain a pixel difference between a second reference data block in the local reference data block set and the data block to be encoded.

[0188] If there is a target data block in the local reference data block set with a pixel difference less than a pixel difference threshold, the selecting module is configured to select the target data block as the reference data block for predictive encoding of the data block to be encoded.

[0189] If there is no target data block in the local reference data block set with a pixel difference less than the pixel difference threshold, the selecting module is configured to select a reference data block from the global reference data block set for predictive encoding of the data block to be encoded.

[0190] Optionally, the selecting module is configured to select a reference data block from the global reference data block set and the local reference data block set for predictive encoding of the data block to be encoded, including:

[0191] If the encoding mode of the data block to be encoded is the intra block copy mode or the intra string copy mode, the selecting module is configured to obtain a target encoded data block having a spatial adjacent relationship with the data block to be encoded.

[0192] If the reference data block of the target coded data block belongs to the global reference data block set and the local reference data block set, search in the areas corresponding to the global reference data block set and the local reference data block set according to a moving parameter to obtain a reference data block for predicting coding of the data block to be coded; the moving parameter includes a moving window and a moving step, the size of the moving window is the same as the size of the data block to be coded, and the moving step is smaller than the width of the moving window.

[0193] Optionally, the global reference data block set includes one or more first reference data blocks; when the coding mode of the multimedia data frame is a serial coding mode, the first reference data block in the global reference data block set is a coded data block in the multimedia data frame, which satisfies a write-back restriction condition of a parameter association relationship and belongs to the same coding slice as the data block to be coded.

[0194] The parameter association relationship is used to reflect the association relationship between the first position information and the second position information and the size information of the maximum coding block; the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame.

[0195] The write-back restriction condition is used to reflect a restriction time delay of writing the coded data block into the global reference data block set.

[0196] Optionally, the obtaining module is configured to: when the coding mode of the multimedia data frame is a serial coding mode, obtain the first position information, the second position information, and the size information of the maximum coding block respectively; the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame; obtain the write-back restriction condition; the write-back restriction condition is used to reflect a restriction time delay of writing the coded data block into the global reference data block set; filter, from the multimedia data frame, the coded data block that satisfies the write-back restriction condition of the parameter association relationship and belongs to the same coding slice as the data block to be coded, as the first reference data block; the parameter association relationship is used to reflect the association relationship between the first position information and the second position information and the size information of the maximum coding block; and add the first reference data block to the global reference data block set.

[0197] Optionally, the obtaining module filters, from the multimedia data frame, the coded data block that satisfies the write-back restriction condition of the parameter association relationship and belongs to the same coding slice as the data block to be coded, as the first reference data block, including:

[0198] screening, according to the first position information and the second position information, an encoded data block belonging to the same coding slice as the to-be-encoded data block from the multimedia data frame as a first candidate reference data block;

[0199] determining a limit number of interval data blocks between the encoded data block in the multimedia data frame and the to-be-encoded data block under the write-back limit condition;

[0200] counting, according to the first position information, the size information of the maximum coding block and the second position information, a number of interval data blocks between the first candidate reference data block and the to-be-encoded data block;

[0201] determining the first candidate reference data block with the number of interval data blocks greater than the limit number of interval data blocks as a first reference data block.

[0202] Optionally, the obtaining module counts, according to the first position information, the size information of the maximum coding block and the second position information, a number of interval data blocks between the first candidate reference data block and the to-be-encoded data block, including:

[0203] determining a row identifier and a column identifier of the first candidate reference data block according to the first position information and the size information of the maximum coding block;

[0204] determining a row identifier and a column identifier of the to-be-encoded data block according to the second position information and the size information of the maximum coding block;

[0205] determining a number of data blocks in each row in the multimedia data frame based on the size information of the to-be-encoded data block;

[0206] counting, according to the number of data blocks in each row, the row identifier and the column identifier of the first candidate reference data block, and the row identifier and the column identifier of the to-be-encoded data block, the number of interval data blocks between the first candidate reference data block and the to-be-encoded data block.

[0207] Optionally, the global reference data block set includes one or more second reference data blocks; the second reference data block of the global reference data block set is an encoded data block belonging to the same coding slice as the to-be-encoded data block, and satisfying a write-back limit condition and a coding processing condition of a parameter association relationship in the multimedia data frame; the parameter association relationship is used to reflect an association relationship between position information of the encoded data block and the to-be-encoded data block in the multimedia data frame and size information of a maximum coding block; the write-back limit condition is used to reflect a limit time delay of writing the encoded data block into the global reference data block set.

[0208] Optionally, the obtaining module is configured to obtain the first position information and the size information of the maximum coded block and the second position information respectively; the first position information and the second position information respectively refer to position information of the coded data block and the data block to be coded in the multimedia data frame; obtain a write-back limitation condition and a coding processing condition; the write-back limitation condition is used to reflect a limitation delay of writing the coded data block into the global reference data block set; filter out, from the multimedia data frame, a coded data block that satisfies the write-back limitation condition and the coding processing condition in a parameter association relationship and belongs to the same coding slice as the data block to be coded, as a second reference data block; the parameter association relationship is used to reflect an association relationship between the first position information, the size information of the data block to be coded and the second position information; and add the second reference data block to the global reference data block set.

[0209] Optionally, the obtaining module filters out, from the multimedia data frame, a coded data block that satisfies the write-back limitation condition and the coding processing condition in a parameter association relationship and belongs to the same coding slice as the data block to be coded, as a second reference data block, including:

[0210] According to the first position information and the second position information, filter out, from the multimedia data frame, a coded data block that belongs to the same coding slice as the data block to be coded, as a first candidate reference data block;

[0211] Filter out, from the first candidate reference data block, a first candidate reference data block that satisfies the write-back limitation condition in a parameter association relationship, as a second candidate reference data block;

[0212] Determine a first data block limitation distance between the coded data block in the multimedia data frame and the data block to be coded under the write-back limitation condition;

[0213] Determine a second data block limitation distance between the coded data block in the multimedia data frame and the data block to be coded under the coding processing condition;

[0214] According to the first position information, the size information of the maximum coded block, the second position information, the first data block limitation distance and the second data block limitation distance, determine a second candidate reference data block that satisfies the coding processing condition from the second candidate reference data block, as a second reference data block.

[0215] Optionally, the obtaining module determines a second candidate reference data block satisfying the encoding processing condition from the second candidate reference data block as a second reference data block according to the first position information, the size information of the maximum coding block and the second position information, the first data block limit distance and the second data block limit distance, comprising:

[0216] determining the row identifier and the column identifier of the second candidate reference data block according to the first position information and the size information of the maximum coding block;

[0217] determining the row identifier and the column identifier of the to-be-encoded data block according to the second position information and the size information of the maximum coding block;

[0218] screening a second candidate reference data block with a row identifier less than or equal to the row identifier of the to-be-encoded data block from the second candidate reference data block as a third candidate reference data block;

[0219] determining the row distance between the third candidate reference data block and the to-be-encoded data block according to the row identifier of the third candidate reference data block and the row identifier of the to-be-encoded data block;

[0220] determining the column distance between the third candidate reference data block and the to-be-encoded data block according to the column identifier of the third candidate reference data block and the column identifier of the to-be-encoded data block;

[0221] determining a third candidate reference data block with a column distance less than a first distance threshold in the third candidate reference data block as a second reference data block; the first distance threshold is determined according to the first data block limit distance, the second data block limit distance, and the row distance between the third candidate reference data block and the to-be-encoded data block.

[0222] Optionally, the local reference data block set comprises one or more third reference data blocks, and the third reference data block is an encoded data block in the multimedia data frame and located in the same encoding processing unit as the to-be-encoded data block.

[0223] Optionally, the local reference data block set comprises one or more fourth reference data blocks, and the fourth reference data block is an encoded data block in the multimedia data frame and having an adjacent relationship with the to-be-encoded data block and belonging to the same encoding slice as the to-be-encoded data block.

[0224] Optionally, the fourth reference data block and the to-be-encoded data block having an adjacent relationship means that the fourth reference data block and the to-be-encoded data block belong to the same maximum coding block.

[0225] Optionally, the fourth reference data block has the adjacent relationship with the data block to be encoded, which means that the fourth reference data block is located in N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is less than or equal to a size threshold; N is determined according to the size information of the maximum coding block.

[0226] Optionally, the fourth reference data block has the adjacent relationship with the data block to be encoded, which means that the fourth reference data block is located in N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is greater than or equal to a size threshold; the data block located at the top-left corner in the area in the multimedia data frame where the second reference data block moves one maximum coding block to the right is an unreconstructed data block; the position information of the fourth reference data block when moving a distance of one maximum coding block is different from the position information of the data block to be encoded; N is determined according to the size information of the maximum coding block.

[0227] Optionally, the distances between the first reference data block, the second reference data block, the third reference data block, the fourth reference data block and the data block to be encoded are all less than a second distance threshold; the first reference data block, the second reference data block, the third reference data block and the fourth reference data block are located at integral pixel positions of the multimedia data frame.

[0228] According to an embodiment of the present application, Figure 7 The steps involved in the data processing method shown in the figure can be executed by Figure 11 The modules in the data processing device shown in the figure can be executed by. For example, Figure 7 The step S101 shown in the figure can be executed by Figure 11 the acquisition module 111 in the figure, Figure 7 The step S102 shown in the figure can be executed by Figure 11 the selection module 112 in the figure.

[0229] According to an embodiment of the present application, Figure 11 The modules in the multimedia data processing device shown in the figure can be combined into one or several units respectively or all, or some of the units can be further split into at least two sub-units with smaller functions, which can realize the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions, and in actual application, the functions of one module can also be realized by at least two units, or the functions of at least two modules can be realized by one unit. In other embodiments of the present application, the data processing device can also include other units, and in actual application, these functions can also be realized by other units and can be realized by at least two units.

[0230] According to an embodiment of the present application, the multimedia data processing apparatus as shown in Figure 10 and Figure 11 the computer program (including program codes) involved in each step of the corresponding method as shown in Figure 11 may be constructed, and the data processing method of the embodiments of the present application can be implemented. The above computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above computing device through the computer readable recording medium, and run therein.

[0231] In the present application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial for the encoding terminal to adaptively select a reference data block from the global reference data block set and the local reference data block set according to application scenarios or user requirements, and to predictively encode the to-be-encoded data block in the multimedia data frame according to the reference data block; the selection range of the reference data block is expanded, and the encoding performance of the multimedia data frame can be improved.

[0232] Please refer to Figure 12 is a structural schematic diagram of a multimedia data processing apparatus provided by an embodiment of the present application. The above multimedia data processing apparatus can be a computer program (including program codes) running in a computer device, for example, the multimedia data processing apparatus is an application software; the apparatus can be used to execute the corresponding steps in the method provided by the embodiments of the present application. As shown in Figure 12 , the multimedia data processing apparatus can include an obtaining module 121 and a selecting module 122.

[0233] The obtaining module is configured to obtain a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; the global reference data block set includes decoded data blocks in the multimedia data frame that do not have an adjacent relationship with the to-be-decoded data block, and the local reference data block set includes decoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-decoded data block.

[0234] The selecting module is configured to select a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set.

[0235] Optionally, the selecting module selects a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set, including:

[0236] obtaining a decoding mode of the to-be-decoded data block;

[0237] if the decoding mode of the to-be-decoded data block is the intra block copy mode or the intra string copy mode, selecting a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set.

[0238] Optionally, the global reference data block set comprises one or more first reference data blocks; when the decoding mode of the multimedia data frame is the string decoding mode, the first reference data block in the global reference data block set is a decoded data block in the multimedia data frame which satisfies a write-back restriction condition and belongs to the same decoding slice as the to-be-decoded data block.

[0239] The parameter association relationship is used to reflect an association relationship between first position information, maximum decoding block size information and second position information, and the first position information and the second position information respectively refer to position information of the decoded data block and the to-be-decoded data block in the multimedia data frame.

[0240] The write-back restriction condition is used to reflect a restriction delay of writing the decoded data block into the global reference data block set.

[0241] Optionally, the global reference data block set comprises one or more second reference data blocks; the second reference data block in the global reference data block set is a decoded data block which satisfies a write-back restriction condition and a decoding processing condition, and belongs to the same decoding slice as the to-be-decoded data block; the parameter association relationship is used to reflect an association relationship between position information of the decoded data block and the to-be-decoded data block in the multimedia data frame, and maximum decoding block size information; and the write-back restriction condition is used to reflect a restriction delay of writing the decoded data block into the global reference data block set.

[0242] Optionally, the local reference data block set comprises one or more third reference data blocks; the third reference data block in the local reference data block set is a decoded data block which is located in the same decoding processing unit as the to-be-decoded data block in the multimedia data frame.

[0243] Optionally, the local reference data block set comprises one or more fourth reference data blocks; the fourth reference data block in the local reference data block set is a decoded data block which has an adjacent relationship with the to-be-decoded data block and belongs to the same decoding slice as the to-be-decoded data block in the multimedia data frame.

[0244] According to an embodiment of the present application,Figure 10 The steps involved in the data processing method shown can be performed by Figure 12 The modules in the data processing apparatus shown can perform the steps shown in Figure 10 The step S201 shown in Figure 12 The obtaining module 121 in Figure 10 The step S202 shown in Figure 12 The selecting module 122 in

[0245] According to one embodiment of the present application, Figure 12 The modules in the multimedia data processing apparatus shown can be combined into one or several units respectively or all, or some of the units can be further split into at least two sub-units with smaller functions, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions, and in actual applications, the functions of one module can also be implemented by at least two units, or the functions of at least two modules can be implemented by one unit. In other embodiments of the present application, the data processing apparatus can also include other units, and in actual applications, these functions can also be assisted by other units, and can be implemented by at least two units.

[0246] According to one embodiment of the present application, the multimedia data processing apparatus shown in Figure 10 and Figure 11 The computer program (including program code) that can execute the steps involved in the corresponding method shown in Figure 12 The multimedia data processing apparatus shown in

[0247] In the present application, the computer device can obtain the global reference data block set and the local reference data block set, which is beneficial to adaptively selecting a reference data block from the global reference data block set and the local reference data block set according to the application scenario or user demand, and predicting and decoding the to-be-decoded data block in the multimedia data frame according to the reference data block; the selection range of the reference data block is expanded, and the decoding performance of the multimedia data frame can be improved.

[0248] Please refer to Figure 13 is a structural schematic diagram of a computer device provided by an embodiment of the present application. As shown in Figure 13As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a media content interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The media content interface 1003 may include a display screen and a keyboard; optionally, the media content interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi). I -F I (Interface). Memory 1005 can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, memory 1005 can also be at least one storage device located remotely from the aforementioned processor 1001. Figure 13 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a media content interface module, and a device control application.

[0249] exist Figure 13 In the computer device 1000 shown, the network interface 1004 provides network communication functions; while the media content interface 1003 is mainly used to provide an interface for inputting media content.

[0250] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 7 and the preceding text Figure 10 The description of the multimedia data processing method in the corresponding embodiments can also be performed as described above. Figure 11 and Figure 12 The description of the multimedia data processing apparatus in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0251] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned multimedia data processing device. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 7 and Figure 10 The description of the multimedia data processing method in the corresponding embodiments is already provided, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0252] As an example, the above program instructions can be deployed on a computer device to execute, or be deployed on at least two computer devices located in one place to execute, or be deployed on at least two computer devices distributed in at least two places and interconnected through a communication network to execute, and the at least two computer devices distributed in at least two places and interconnected through a communication network can constitute a blockchain network.

[0253] The computer readable storage medium described above can be an internal storage unit of the data processing apparatus provided in any of the preceding embodiments or the computer device described above, for example, a hard disk or a memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the computer device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0254] The embodiments of the present application also provide a computer program product, comprising computer programs / instructions, which, when executed by a processor, implement the preceding Figure 7 and Figure 10 The description of the above data processing method in the corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will also not be repeated. For technical details of the embodiments of the computer program product involved in the present application that are not disclosed, please refer to the description of the method embodiments of the present application.

[0255] The terms "first", "second", etc. in the specification and claims and drawings of the embodiments of the present application are used to distinguish different media contents, and are not used to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units is not limited to the listed steps or modules, but can optionally include steps or modules that are not listed, or can optionally include other steps or units inherent to the process, method, device, product or equipment.

[0256] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0257] The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. The computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the functions specified in the flow Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the functions specified in the flow Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the functions specified in the flow Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the functions specified in the flow Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the functions specified in the flow

[0258] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made by the claims of the present application still fall within the scope of the present application.

Claims

1. A multimedia data processing method, characterized by, The method comprises: obtaining a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; the global reference data block set comprises encoded data blocks in the multimedia data frame that have no adjacent relationship with the to-be-encoded data block, and the local reference data block set comprises encoded data blocks in the multimedia data frame that have an adjacent relationship with the to-be-encoded data block; the global reference data block set comprises one or more first reference data blocks; when the encoding mode of the multimedia data frame is a serial encoding mode, the first reference data block is an encoded data block in the multimedia data frame that meets a write-back restriction condition and belongs to the same encoding slice as the to-be-encoded data block; the parameter association relationship is used to reflect an association relationship between first position information and maximum coding block size information and second position information, and the first position information and the second position information respectively refer to position information of the encoded data block and the to-be-encoded data block in the multimedia data frame; the write-back restriction condition is used to reflect a restriction delay for writing the encoded data block into the global reference data block set; from the global reference data block set and the local reference data block set, a reference data block used for predictive encoding of the to-be-encoded data block is selected.

2. The method of claim 1, wherein, The method comprises: obtaining an encoding mode of the to-be-encoded data block; if the encoding mode of the to-be-encoded data block is an intra block copy mode or an intra string copy mode, a reference data block used for predictive encoding of the to-be-encoded data block is selected from the global reference data block set and the local reference data block set.

3. The method of claim 2, wherein, The method comprises: if the encoding mode of the to-be-encoded data block is an intra block copy mode or an intra string copy mode, pixel differences between data blocks in the local reference data block set and the to-be-encoded data block are obtained; if there is a target data block in the local reference data block set whose pixel difference is less than a pixel difference threshold, the target data block is taken as a reference data block used for predictive encoding of the to-be-encoded data block; if there is no target data block in the local reference data block set whose pixel difference is less than the pixel difference threshold, a reference data block used for predictive encoding of the to-be-encoded data block is selected from the global reference data block set.

4. The method of claim 2, wherein, The method comprises: if the encoding mode of the to-be-encoded data block is an intra block copy mode or an intra string copy mode, a reference data block used for predictive encoding of the to-be-encoded data block is selected from the global reference data block set and the local reference data block set. If the coding mode of the to-be-encoded data block is the intra block copy mode or the intra string copy mode, a target coded data block having a spatial neighboring relationship with the to-be-encoded data block is obtained; If the reference data block of the target coded data block belongs to the global reference data block set and the local reference data block set, a reference data block used for prediction coding of the to-be-encoded data block is searched in the areas corresponding to the global reference data block set and the local reference data block set according to a moving parameter; the moving parameter includes a moving window and a moving step, the size of the moving window is the same as the size of the to-be-encoded data block, and the moving step is smaller than the width of the moving window.

5. The method according to any one of claims 1 to 4, wherein The global reference data block set further includes one or more second reference data blocks; the second reference data block is a coded data block belonging to the same coding slice as the to-be-encoded data block, and the parameter association relationship in the multimedia data frame satisfies a write-back restriction condition and a coding processing condition; The parameter association relationship is used to reflect the association relationship between the position information of the coded data block and the to-be-encoded data block in the multimedia data frame and the size information of the maximum coding block; The write-back restriction condition is used to reflect the restriction time delay of writing the coded data block into the global reference data block set.

6. The method according to any one of claims 1 to 4, wherein The global reference data block set associated with the to-be-encoded data block in the multimedia data frame is obtained, including: When the coding mode of the multimedia data frame is a serial coding mode, first position information, maximum coding block size information and second position information are respectively obtained; the first position information and the second position information are respectively the position information of the coded data block and the to-be-encoded data block in the multimedia data frame; A write-back restriction condition is obtained; the write-back restriction condition is used to reflect the restriction time delay of writing the coded data block into the global reference data block set; A coded data block satisfying the write-back restriction condition and belonging to the same coding slice as the to-be-encoded data block is selected from the multimedia data frame as a first reference data block; the parameter association relationship is used to reflect the association relationship between the first position information, the maximum coding block size information and the second position information; The first reference data block is added to the global reference data block set.

7. The method of claim 6, wherein, The coded data block satisfying the write-back restriction condition and belonging to the same coding slice as the to-be-encoded data block is selected from the multimedia data frame as a first reference data block, including: According to the first position information and the second position information, a coded data block belonging to the same coding slice as the to-be-encoded data block is selected from the multimedia data frame as a first candidate reference data block; The interval data block restriction quantity between the coded data block in the multimedia data frame and the to-be-encoded data block under the write-back restriction condition is determined; According to the first position information, the size information of the maximum coding block and the second position information, the number of interval data blocks between the first candidate reference data block and the data block to be coded is counted; The first candidate reference data block with the number of interval data blocks greater than the interval data block limit number is determined as the first reference data block.

8. The method of claim 7, wherein, The counting of the number of interval data blocks between the first candidate reference data block and the data block to be coded according to the first position information, the size information of the maximum coding block and the second position information comprises: The row identifier and the column identifier of the first candidate reference data block are determined according to the first position information and the size information of the maximum coding block; The row identifier and the column identifier of the data block to be coded are determined according to the second position information and the size information of the maximum coding block; The number of data blocks in each row in the multimedia data frame is determined based on the size information of the data block to be coded; The number of interval data blocks between the first candidate reference data block and the data block to be coded is counted according to the number of data blocks in each row, the row identifier and the column identifier of the first candidate reference data block, and the row identifier and the column identifier of the data block to be coded.

9. The method of any one of claims 1-4, wherein, The method further comprises: The first position information, the size information of the maximum coding block and the second position information are respectively acquired; the first position information and the second position information respectively refer to the position information of the coded data block and the data block to be coded in the multimedia data frame; The write-back limit condition and the coding processing condition are acquired; the write-back limit condition is used to reflect the limit delay of writing the coded data block into the global reference data block set; The coded data block with the parameter association relationship satisfying the write-back limit condition and the coding processing condition and belonging to the same coding slice as the data block to be coded is selected from the multimedia data frame as a second reference data block; the parameter association relationship is used to reflect the association relationship between the first position information, the size information of the data block to be coded and the second position information; The second reference data block is added to the global reference data block set.

10. The method of claim 9, wherein, The coded data block with the parameter association relationship satisfying the write-back limit condition and the coding processing condition and belonging to the same coding slice as the data block to be coded is selected from the multimedia data frame as a second reference data block, which comprises: The coded data block belonging to the same coding slice is selected from the multimedia data frame as a first candidate reference data block according to the first position information and the second position information; The first candidate reference data block with the parameter association relationship satisfying the write-back limit condition is selected from the first candidate reference data block as a second candidate reference data block; The first data block limit distance between the coded data block in the multimedia data frame and the data block to be coded under the write-back limit condition is determined; The second data block limit distance between the coded data block in the multimedia data frame and the data block to be coded under the coding processing condition is determined; According to the first position information, the size information of the maximum coding block, the second position information, the first data block limit distance and the second data block limit distance, a second candidate reference data block satisfying the coding processing condition is determined from the second candidate reference data blocks as a second reference data block.

11. The method of claim 10, wherein, The determining of the second candidate reference data block satisfying the coding processing condition from the second candidate reference data blocks as a second reference data block according to the first position information, the size information of the maximum coding block, the second position information, the first data block limit distance and the second data block limit distance comprises: determining the row identifier and the column identifier of the second candidate reference data block according to the first position information and the size information of the maximum coding block; determining the row identifier and the column identifier of the data block to be coded according to the second position information and the size information of the maximum coding block; screening a second candidate reference data block with a row identifier less than or equal to the row identifier of the data block to be coded from the second candidate reference data blocks as a third candidate reference data block; determining the row distance between the third candidate reference data block and the data block to be coded according to the row identifier of the third candidate reference data block and the row identifier of the data block to be coded; determining the column distance between the third candidate reference data block and the data block to be coded according to the column identifier of the third candidate reference data block and the column identifier of the data block to be coded; determining a second reference data block with a column distance less than a first distance threshold from the third candidate reference data blocks; the first distance threshold is determined according to the first data block limit distance, the second data block limit distance, the row distance between the third candidate reference data block and the data block to be coded.

12. The method of any one of claims 1-4, wherein, The local reference data block set comprises one or more third reference data blocks; the third reference data block is a coded data block in the same coding processing unit as the data block to be coded in the multimedia data frame.

13. The method of any one of claims 1-4, wherein, The local reference data block set comprises one or more fourth reference data blocks; the fourth reference data block is a coded data block in the multimedia data frame having an adjacent relationship with the data block to be coded and belonging to the same coding slice as the data block to be coded.

14. The method of claim 13, wherein, The fourth reference data block has an adjacent relationship with the data block to be coded, which means that the fourth reference data block and the data block to be coded belong to the same maximum coding block.

15. The method of claim 13, wherein, The fourth reference data block has an adjacent relationship with the data block to be coded, which means that the fourth reference data block is located within N maximum coding blocks adjacent to the data block to be coded, and the size of the maximum coding block is less than or equal to a size threshold; N is determined according to the size information of the maximum coding block.

16. The method of claim 13, wherein, The fourth reference data block has an adjacent relationship with the data block to be encoded, which means that the fourth reference data block is located in N maximum coding blocks adjacent to the data block to be encoded, and the size of the maximum coding block is greater than or equal to a size threshold; the data block located at the top left corner in the area where the fourth reference data block in the multimedia data frame moves one maximum coding block to the right is an unreconstructed data block; The position information of the fourth reference data block when it moves one maximum coding block is different from the position information of the data block to be encoded; N is determined according to the size information of the maximum coding block.

17. A multimedia data processing method, characterized by, Comprise: Obtain a global reference data block set and a local reference data block set associated with a data block to be decoded in a multimedia data frame; The global reference data block set comprises decoded data blocks in the multimedia data frame that do not have an adjacent relationship with the data block to be decoded, and the local reference data block set comprises decoded data blocks in the multimedia data frame that have an adjacent relationship with the data block to be decoded; The global reference data block set comprises one or more first reference data blocks; When the decoding mode of the multimedia data frame is a serial decoding mode, the first reference data block is a decoded data block in the multimedia data frame that satisfies a write-back restriction condition and belongs to the same decoding slice as the data block to be decoded; The parameter association relationship is used to reflect the association relationship between the first position information, the size information of the maximum decoding block and the second position information, and the first position information and the second position information are respectively the position information of the decoded data block and the data block to be decoded in the multimedia data frame; The write-back restriction condition is used to reflect the restriction delay of writing the decoded data block into the global reference data block set; From the global reference data block set and the local reference data block set, select a reference data block for decoding the data block to be decoded.

18. The method of claim 17, wherein, The global reference data block set comprises one or more first reference data blocks; The second reference data block is a decoded data block that belongs to the same decoding slice as the data block to be decoded, and the parameter association relationship in the multimedia data frame satisfies the write-back restriction condition and the decoding processing condition; The parameter association relationship is used to reflect the association relationship between the position information of the decoded data block and the data block to be decoded in the multimedia data frame, and the size information of the maximum decoding block; 19. The method of claim 17 or 18, wherein, ​ ​ ​ The write-back restriction condition is used to reflect a restriction time delay of writing the decoded data block into the global reference data block set.

20. The method of claim 17 or 18, wherein, The local reference data block set comprises one or more third reference data blocks. The third reference data block is a decoded data block in the multimedia data frame which is in the same decoding processing unit as the to-be-decoded data block.

21. The method of claim 17 or 18, wherein, The local reference data block set comprises one or more fourth reference data blocks. The fourth reference data block is a decoded data block in the multimedia data frame which has an adjacent relationship with the to-be-decoded data block and belongs to the same decoding slice as the to-be-decoded data block.

22. A multimedia data processing apparatus, characterized by comprising: Comprise: An acquisition module is used to acquire a global reference data block set and a local reference data block set associated with a to-be-encoded data block in a multimedia data frame; The global reference data block set comprises encoded data blocks in the multimedia data frame which do not have an adjacent relationship with the to-be-encoded data block, and the local reference data block set comprises encoded data blocks in the multimedia data frame which have an adjacent relationship with the to-be-encoded data block; The global reference data block set comprises one or more first reference data blocks; When the encoding mode of the multimedia data frame is a serial encoding mode, the first reference data block is an encoded data block in the multimedia data frame which satisfies a write-back restriction condition in a parameter association relationship and belongs to the same encoding slice as the to-be-encoded data block; The parameter association relationship is used to reflect an association relationship between first position information and maximum coding block size information and second position information, and the first position information and the second position information respectively refer to position information of the encoded data block and the to-be-encoded data block in the multimedia data frame; The write-back restriction condition is used to reflect a restriction time delay of writing the encoded data block into the global reference data block set; A selection module is used to select a reference data block used for predictive encoding of the to-be-encoded data block from the global reference data block set and the local reference data block set.

23. A multimedia data processing apparatus, characterized by comprising: Comprise: An acquisition module is used to acquire a global reference data block set and a local reference data block set associated with a to-be-decoded data block in a multimedia data frame; The global reference data block set comprises decoded data blocks in the multimedia data frame which do not have an adjacent relationship with the to-be-decoded data block, and the local reference data block set comprises decoded data blocks in the multimedia data frame which have an adjacent relationship with the to-be-decoded data block; The global reference data block set comprises one or more first reference data blocks; When the decoding mode of the multimedia data frame is a serial decoding mode, the first reference data block is a decoded data block in the multimedia data frame which satisfies a write-back restriction condition in a parameter association relationship and belongs to the same decoding slice as the to-be-decoded data block; The parameter correlation relationship is used for reflecting an association relationship between first position information, maximum decoding block size information and second position information, wherein the first position information and the second position information respectively refer to position information of a decoded data block and the to-be-decoded data block in the multimedia data frame. The write-back restriction condition is used for reflecting a restriction delay for writing the decoded data block into the global reference data block set. The selection module is used for selecting a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set.

24. A computer device, comprising: Comprise: A processor and a memory; The processor is connected with the memory; the memory is used for storing program code, and the processor is used for calling the program code to execute the method in any one of claims 1 to 21. The parameter correlation relationship is used for reflecting an association relationship between first position information, maximum decoding block size information and second position information, wherein the first position information and the second position information respectively refer to position information of a decoded data block and the to-be-decoded data block in the multimedia data frame. The write-back restriction condition is used for reflecting a restriction delay for writing the decoded data block into the global reference data block set. The selection module is used for selecting a reference data block for decoding the to-be-decoded data block from the global reference data block set and the local reference data block set. Comprise: A processor and a memory; The processor is connected with the memory; the memory is used for storing program code, and the processor is used for calling the program code to execute the method in any one of claims 1 to 21.

Images