Data processing method and device and related equipment

Abstract

The embodiment of the invention provides a data processing method and device and related equipment, and the method comprises the steps: obtaining a to-be-processed data block which is data to be subjected to image horizontal amplification processing; a target data set in the to-be-processed data block is read, the target data set is data units located in the same row in the to-be-processed data block, the to-be-processed data block is divided into a plurality of data units with preset sizes, and the data units are minimum calculation units for graphic processing; performing super-resolution filtering on the target data set to obtain an initial processing result corresponding to the target data set; and executing loop recovery filtering corresponding to the initial processing result to obtain a target processing result. According to the embodiment of the invention, loop recovery filtering can be started without waiting for super-resolution filtering of the whole to-be-processed data block, so that the data processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of video encoding and decoding technology, specifically to a data processing method, apparatus, and related equipment. Background Technology

[0002] Video encoding refers to compressing video data using specific algorithms to efficiently transmit and store video content within limited bandwidth and storage space. Video decoding refers to decoding the compressed video file according to the corresponding algorithm to recover the original video data. In the video decoding process, to recover the original video data, it needs to be processed sequentially by a deblocking filter, a constrained direction enhancement filter, a super-resolution filter, and a loop recovery filter.

[0003] However, the data processing efficiency in video decoding still needs to be improved. Summary of the Invention

[0004] In view of this, embodiments of this application provide a data processing method, apparatus, and related equipment to improve the decoding efficiency of video decoding.

[0005] To achieve the above objectives, the embodiments of this application provide the following technical solutions.

[0006] In a first aspect, embodiments of this application provide a data processing method, including: Obtain the data block to be processed, which is the data for which horizontal image magnification processing is to be performed; Read the target data group in the data block to be processed, wherein the target data group is the data unit located in the same row in the data block to be processed, the data block to be processed is divided into multiple data units with a preset size, and the data unit is the smallest computing unit for graphics processing; Super-resolution filtering is performed on the target data set to obtain the initial processing result corresponding to the target data set; Perform loop recovery filtering corresponding to the initial processing result to obtain the target processing result.

[0007] Optionally, the step of obtaining the data block to be processed includes: Obtain the initial horizontal starting coordinates, initial width, and magnification of the data block to be processed.

[0008] Optionally, after the step of obtaining the data block to be processed and before the step of reading the target data group in the data block to be processed, the method further includes: Generate the read address for the target data group; Determine the filtering parameters for each data unit in the corresponding target data set.

[0009] Optionally, the step of generating the read address of the target data group includes: Based on the initial horizontal starting coordinates, initial width, and magnification factor of the data block to be processed, calculate the target horizontal starting coordinates and target width of the data block after magnification. Based on the initial horizontal starting coordinates, initial width, target horizontal starting coordinates, and target width, determine the pixel coordinates in the data block to be processed corresponding to the pixel coordinates in the target data block after the data block to be processed is magnified; Based on the pixel coordinates in the target data block corresponding to each pixel coordinate in the target data block, and the row information of the target data group, a read address for reading the target data group is generated.

[0010] Optionally, the step of determining the filtering parameters for each data unit in the corresponding target data group includes: Based on the lookup table, the filtering parameters for each data unit in the corresponding target data group are determined.

[0011] Optionally, the step of reading the target data group in the data block to be processed includes: Based on the read address, the target data group in the data block to be processed is read and the target data group is stored in a preset buffer.

[0012] Optionally, the preset buffer includes a first buffer and a second buffer, wherein the first buffer is used to store the current target data group and the second buffer is used to store the next target data group.

[0013] Optionally, the step of performing super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group includes: Based on the filtering parameters, super-resolution filtering is performed on the target data group to obtain the initial processing result corresponding to the target data group.

[0014] Secondly, embodiments of this application provide a data processing apparatus, including: A super-resolution filtering circuit is used to acquire a data block to be processed, wherein the data block to be processed is data for which horizontal image magnification processing is to be performed; read a target data group in the data block to be processed, wherein the target data group is a data unit located in the same row in the data block to be processed, the data block to be processed is divided into multiple data units with a preset size, and the data unit is the smallest computing unit for image processing; and perform super-resolution filtering on the target data group to obtain an initial processing result corresponding to the target data group. The loop recovery filter circuit is used to perform loop recovery filtering corresponding to the initial processing result to obtain the target processing result.

[0015] Optionally, the super-resolution filtering circuit includes: The magnification control module is used to acquire the data block to be processed, which is the data of the image to be horizontally magnified, and to generate the read address of the target data group and determine the filtering parameters of each data unit in the corresponding target data group. The preprocessing module is used to read the target data group in the data block to be processed based on the read address, and store the target data group in a preset buffer; An amplification module is used to perform super-resolution filtering on the target data group based on the filtering parameters to obtain the initial processing result corresponding to the target data group.

[0016] Optionally, the preset buffer includes a first buffer and a second buffer, wherein the first buffer is used to store the current target data group and the second buffer is used to store the next target data group.

[0017] Thirdly, embodiments of this application provide an electronic device, including at least one memory and at least one processor, wherein the memory stores one or more computer-executable instructions, and the processor invokes the one or more computer-executable instructions to execute the data processing method as described in the first aspect above.

[0018] Fourthly, embodiments of this application provide a storage medium that stores one or more computer-executable instructions, which, when executed, implement the data processing method described in the first aspect above.

[0019] Fifthly, embodiments of this application provide a computer program product including one or more computer-executable instructions, which, when executed, implement the data processing method described in the first aspect above.

[0020] This application provides a data processing method, apparatus, and related equipment. The method includes: acquiring a data block to be processed, wherein the data block to be processed is data for which horizontal image magnification processing is to be performed; reading a target data group in the data block to be processed, wherein the target data group is a data unit located in the same row in the data block to be processed, the data block to be processed is divided into multiple data units of a preset size, and the data unit is the smallest computing unit for image processing; performing super-resolution filtering on the target data group to obtain an initial processing result corresponding to the target data group; and performing loop recovery filtering corresponding to the initial processing result to obtain a target processing result.

[0021] As can be seen, the data processing method provided in this application reads the target data group in the data block to be processed, and then performs super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group; then, it performs loop recovery filtering corresponding to the initial processing result to obtain the target processing result. This allows the super-resolution filtering and loop recovery filtering to be executed in a target data group-level pipeline manner, so that the loop recovery filtering can start without waiting for the entire data block to be processed to complete the super-resolution filtering, thereby improving the data processing efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of an optional encoding / decoding architecture; Figure 2 This is an optional flowchart illustrating the data processing method provided in the embodiments of this application; Figure 3 This is another optional flowchart illustrating the data processing method provided in the embodiments of this application; Figure 4 This is an optional flowchart of step S110 provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the magnified process of the data block to be processed provided in the embodiments of this application; Figure 6 This is a schematic diagram of an optional structure of the data processing apparatus provided in the embodiments of this application; Figure 7 This is a schematic diagram of an optional structure of the super-resolution filtering circuit provided in the embodiments of this application; Figure 8 This is a schematic diagram of an optional structure of the pixel convolution module provided in an embodiment of this application; Figure 9 This is an optional block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0025] As described in the background section, video encoding refers to compressing video data using specific algorithms to efficiently transmit and store video content within limited bandwidth and storage space. Video decoding refers to decoding the compressed video file according to the corresponding algorithm to recover the original video data. In the video decoding process, to recover the original video data, it needs to be processed sequentially by a deblocking filter, a constrained direction enhancement filter, a super-resolution filter, and a loop recovery filter.

[0026] For ease of understanding, please refer to Figure 1 An exemplary schematic diagram of an optional structure of an encoding / decoding architecture is shown, such as... Figure 1 As shown, the encoding / decoding architecture may include block partitioning, encoding controller, transform, quantization, inverse transform, inverse quantization, post-processing, reference frame buffer, intra-frame prediction, inter-frame prediction, and entropy coding.

[0027] In video encoding, to effectively remove spatial and temporal redundancy in the video stream, intra-frame and inter-frame coding can be performed on the image information. Specifically, the images in the video stream are first divided into blocks. Then, for each block, intra-frame or inter-frame prediction is performed depending on the specific circumstances. Next, the original image and the predicted image are compared and calculated to obtain the residual signal. Finally, the residual signal is transformed, quantized, and entropy-coded to obtain the compressed image bitstream.

[0028] During video decoding, the decoding loop within the encoding / decoding architecture generates a reconstructed image. This reconstructed image can be stored as a reference frame in a reference frame buffer, providing crucial information for subsequent image encoding. In the encoding / decoding architecture, inverse transform and inverse quantization are essential components of the decoding loop. During encoding, the original image data undergoes block partitioning, transforming, and quantization operations, resulting in information loss and residuals. Therefore, during decoding, inverse quantization and inverse transform are used for reverse processing to recover the residuals corresponding to each block, thus obtaining the initial reconstructed image. Next, the initial reconstructed image can be post-processed to obtain the final reconstructed image. Specifically, the initial reconstructed image can be sequentially processed by a deblocking filter (as shown in "Deblock"), a constrained direction enhancement filter (as shown in "CDEF"), a super-resolution filter (as shown in "Super Resolution"), and a loop restoration filter (as shown in "LoopRestoration") to obtain the final reconstructed image.

[0029] Among them, the deblocking filter is used to eliminate the block effect in the image, making the transition of block boundaries in the image more natural; the constrained direction enhancement filter is used to eliminate the ringing effect in the image, improving the image's clarity and quality; the super-resolution filter is used to perform horizontal magnification processing on the image, restoring the image to its original size and compensating for the information loss caused by scaling during the encoding process; and the loop recovery filter is used to restore the high-frequency component information lost during the scaling process, further enhancing the image's details and realism.

[0030] The inventors analyzed that the aforementioned encoding / decoding architecture employs a block-based processing approach. Specifically, to improve the encoding / decoding efficiency of a single frame, it can be divided into multiple tiles, each a separate part of the image that can be encoded / decoded independently. Furthermore, each tile can be further divided into superblocks, which are data blocks obtained after dividing the images in the video stream into blocks in the aforementioned encoding / decoding architecture. Therefore, in the post-processing of video decoding, the deblocking filter, constrained orientation enhancement filter, super-resolution filter, and loop recovery filter all process the image based on superblocks. This block-based processing approach requires waiting for the entire superblock to complete the current stage of processing (e.g., super-resolution filtering) before starting the next stage (e.g., loop recovery filtering), which severely impacts the data processing efficiency during video decoding.

[0031] In view of this, embodiments of this application provide a data processing method, apparatus, and related equipment. The method includes: acquiring a data block to be processed, the data block being data for which horizontal image magnification processing is to be performed; reading a target data group from the data block to be processed, wherein the target data group is a data unit located in the same row within the data block to be processed, the data block to be processed is divided into multiple data units of a preset size, and the data unit is the smallest computational unit for image processing; performing super-resolution filtering on the target data group to obtain an initial processing result corresponding to the target data group; and performing loop recovery filtering corresponding to the initial processing result to obtain a target processing result.

[0032] As can be seen, the data processing method provided in this application reads the target data group in the data block to be processed, and then performs super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group; then, it performs loop recovery filtering corresponding to the initial processing result to obtain the target processing result. This allows the super-resolution filtering and loop recovery filtering to be executed in a target data group-level pipeline manner, so that the loop recovery filtering can start without waiting for the entire data block to be processed to complete the super-resolution filtering, thereby improving the data processing efficiency.

[0033] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0034] refer to Figure 2 , Figure 2 This is a schematic diagram of an optional flow of the data processing method provided in the embodiments of this application. For example... Figure 2 As shown, the method may include the following steps: Step S100: Obtain the data block to be processed, which is the data for which horizontal magnification processing of the image is to be performed.

[0035] In an optional implementation, the data block to be processed can be the result of constraint direction enhancement filtering, which may include... and Two sizes are available. For compatibility. and This application adopts a two-size approach for processing data blocks, and aims to minimize the use of cache resources. The data processing method is introduced by taking super-resolution processing of a data block of a certain size as an example.

[0036] In one alternative implementation, if the size of the data block to be processed is Then you can first break it down into multiple parts. Sub-data blocks. For example, The data block is divided into two equal parts both horizontally and vertically, resulting in 4. The sub-data block. Then, use as The processing flow for the data blocks of varying sizes involves processing each of the four sub-data blocks separately. Finally, in each... After the sub-data blocks are processed, they are merged back according to the rules used during the splitting process. The data block is sufficient. This is understandable. The splitting and merging of data blocks is to maintain the consistency of the processing logic and facilitate super-resolution filtering and loop recovery filtering. It is not a step in super-resolution filtering or loop recovery filtering.

[0037] In an optional implementation, the step of obtaining the data block to be processed may include: obtaining the initial horizontal starting coordinates, initial width, and magnification factor of the data block to be processed, so as to provide a data basis for subsequently generating the read address of the target data group to be read.

[0038] Further reference Figure 3 The exemplary illustration shows another optional flow diagram of the data processing method provided in the embodiments of this application, such as... Figure 3 As shown, after the step of obtaining the data block to be processed and before the step of reading the target data group in the data block to be processed, the following may also be included: Step S110: Generate the read address for the target data group.

[0039] Based on the read address, the target data group in the data block to be processed can be read, thereby providing a data basis for subsequent super-resolution filtering of the target data group.

[0040] refer to Figure 4 The exemplary schematic diagram of the optional process of step S110 provided in the embodiment of this application is shown as follows: Figure 4 As shown, the step of generating the read address for the target data group may include: Step S111: Based on the initial horizontal starting coordinates, initial width, and magnification factor of the data block to be processed, calculate the target horizontal starting coordinates and target width of the data block after magnification.

[0041] refer to Figure 5 An exemplary schematic diagram of the scaling process for the data block to be processed is shown, such as... Figure 5As shown, src_picture is the initial image, src_block is the data block to be processed, dst_picture is the target image after magnification of the initial image, and dst_block is the target data block after magnification of the data block to be processed. Figure 5 The exemplary diagram illustrates the structure of the magnification process for the data block to be processed. In a specific implementation, the process of calculating the target horizontal starting coordinates and target width of the magnified data block can, for example, be: First, based on the initial horizontal starting coordinates src_block_x0 and the initial width src_block_width of the data block to be processed, calculate the initial horizontal ending coordinates src_block_x1 of the data block to be processed. The calculation formula can be, for example, as follows: src_block_x1=src_block_x0+src_block_width; Then, using the known magnification factor (e.g., superres_denom / SCALE_NUMERATOR), calculate the target horizontal starting coordinate dst_block_x0 and target horizontal ending coordinate dst_block_x1 of the magnified data block to be processed. The calculation formula can be, for example: dst_block_x0=src_block_x0*(superres_denom / SCALE_NUMERATOR); dst_block_x1=src_block_x1*(superres_denom / SCALE_NUMERATOR); Finally, using the calculated target horizontal starting coordinates dst_block_x0 and target horizontal ending coordinates dst_block_x1, the enlarged target width dst_block_width of the data block to be processed is calculated. The calculation formula could be, for example: dst_block_width=dst_block_x1 dst_block_x0; In this context, `src_block_x0`, `src_block_width`, and `superres_denom / SCALE_NUMERATOR` are known variables. `superres_denom / SCALE_NUMERATOR` represents the magnification factor. In super-resolution upscaling, this factor determines the ratio by which the data block to be processed is magnified to the target size. For example, if `superres_denom=3` and `SCALE_NUMERATOR=2`, then `superres_denom / SCALE_NUMERATOR=1.5`, meaning the data block will be magnified by a factor of 1.5.

[0042] refer to Figure 4 Continue executing step S112, based on the initial horizontal starting coordinates, initial width, target horizontal starting coordinates and target width, determine the pixel coordinates in the data block to be processed corresponding to each pixel coordinate in the target data block after the data block to be processed is magnified.

[0043] In an optional implementation, the initial horizontal error value `initialSubpelX` and the pixel coordinate step value `stepX`, which includes the step error, can be obtained from a register. `initialSubpelX` indicates the initial sub-pixel level positional deviation information in the horizontal direction to improve image magnification quality; `stepX` indicates the change in pixel coordinates and error information during each horizontal movement, controlling the pixel coordinate movement step size.

[0044] Then, by combining the initial horizontal starting coordinates, initial width, target horizontal starting coordinates, and target width, the pixel coordinates in the target data block after magnification are determined, corresponding to the pixel coordinates in the data block to be processed. In an optional implementation, the pixel coordinates in the data block to be processed can be calculated by traversing the pixel coordinates of the target data block, based on `initialSubpelX` and `stepX`, and the known initial horizontal starting coordinates, initial width, target horizontal starting coordinates, and target width. For example, for each pixel coordinate in the target data block, the corresponding pixel coordinate in the data block to be processed can be obtained using `src_pixel_x = (dst_pixel_x - initialSubpelX) / stepX` (or a similar calculation). Here, `src_pixel_x` is the horizontal coordinate of the pixel coordinates in the data block to be processed, and `dst_pixel_x` is the horizontal coordinate of the pixel coordinates in the target data block.

[0045] It should be noted that since super-resolution filtering only performs horizontal magnification on the data block to be processed, it is only necessary to determine the horizontal coordinates of the pixel coordinates in the data block to be processed that correspond to the horizontal coordinates of each pixel coordinate in the target data block.

[0046] refer to Figure 4 Continue executing step S113, generating a read address for reading the target data group based on the pixel coordinates in the data block to be processed corresponding to each pixel coordinate in the target data block, and the row information of the target data group.

[0047] Based on the pixel coordinates in the corresponding data block to be processed calculated in step S112, and the row information of the target data group, the specific position of the target data group in the data block to be processed can be determined, and a read address for reading the target data group can be generated. This read address is used to read the pixel data of the target data group for subsequent operations such as convolution calculations in image magnification processing.

[0048] refer to Figure 3 Continue with step S120 to determine the filtering parameters for each data unit in the corresponding target data group.

[0049] In an optional implementation, the step of determining the filtering parameters of each data unit in the corresponding target data group may include: determining the filtering parameters of each data unit in the corresponding target data group based on a lookup table. In a specific implementation, the filtering parameters of each data unit in the corresponding target data group can be read from the lookup table. The lookup table stores pre-calculated filtering parameters, which are used to perform convolution calculations on the read target data group to achieve image magnification and optimization processing.

[0050] refer to Figure 2 Continue with step S200 to read the target data group in the data block to be processed.

[0051] The target data group is a data unit located in the same row in the data block to be processed. The data block to be processed is divided into multiple data units with a preset size. The data unit is the smallest computing unit for graphics processing.

[0052] In a specific implementation, the step of reading the target data group from the data block to be processed may include: reading the target data group from the data block to be processed based on the read address, and storing the target data group in a preset buffer. The preset buffer may include a first buffer and a second buffer, wherein the first buffer is used to store the current target data group, and the second buffer is used to store the next target data group.

[0053] Since image processing is typically continuous, the next target data group needs to be processed immediately after the current target data group is processed. Therefore, this application pre-stores the next target data group in a second buffer. Once the current target data group in the first buffer is processed, the next target data group in the second buffer can be quickly switched to the first buffer, achieving seamless data transfer. For example, after processing the current target data group, the next target data group is already stored in the second buffer, allowing for direct processing of the next target data group. This avoids processing interruptions caused by waiting for the next target data group to be read, thus improving data processing efficiency.

[0054] refer to Figure 2 Then, continue with step S300 to perform super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group.

[0055] In an optional implementation, the step of performing super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group may include: performing super-resolution filtering on the target data group based on the filtering parameters to obtain the initial processing result corresponding to the target data group.

[0056] Super-resolution filtering improves image resolution and quality by utilizing information from surrounding pixels. The filtering parameters determine how pixels in the target data set are weighted and combined. Before processing the target data set, filtering parameters for each data unit within the target data set can be determined, and then super-resolution filtering (e.g., convolution calculation) is performed on the target data set based on these filtering parameters. The convolution calculation involves pointwise multiplication and accumulation of the filtering parameters (typically represented as filter kernels) with the pixels in the target data set.

[0057] Specifically, for each pixel in the target data set, take the pixels within a certain range of its neighbors (this range can be determined based on the size of the filter), then multiply the values ​​of these pixels by the coefficients at the corresponding positions in the filter kernel, and finally add all the product results together. The result is the new value of the pixel after super-resolution filtering.

[0058] For example, suppose the filter is a The kernel, the pixel adjacent to a certain pixel in the target data group. Given pixel regions p10, p11, p12, p13, p14, p15, p16, and p17, and filter kernel coefficients w10, w11, w12, w13, w14, w15, w16, and w17, the new value new_p of this pixel after convolution is: new_p=p10*w10+p11*w11+p12*w12+p13*w13+p14*w14+p15*w15+p16*w16+p17*w17.

[0059] The initial processing result is pixel data after super-resolution filtering, which has improved resolution and image quality compared to the original target data group, thus providing a data foundation for further image processing.

[0060] refer to Figure 2 Then, continue to execute step S400, perform loop recovery filtering corresponding to the initial processing result, and obtain the target processing result.

[0061] The loop recovery filter is primarily used to recover information lost during image compression. It further optimizes the initial processing result obtained through super-resolution filtering, leading to a target processing result. Compared to the initial processing result, the target processing result exhibits improved image quality, being clearer, more accurate, and better meeting the requirements of high-resolution images.

[0062] As can be seen, the data processing method provided in this application reads the target data group in the data block to be processed, and then performs super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group; then, it performs loop recovery filtering corresponding to the initial processing result to obtain the target processing result. This allows the super-resolution filtering and loop recovery filtering to be executed in a target data group-level pipeline manner, so that the loop recovery filtering can start without waiting for the entire data block to be processed to complete the super-resolution filtering, thereby improving the data processing efficiency.

[0063] In an optional implementation, data processing can also be performed in a target data group-level pipeline manner during the constraint direction enhancement filtering stage. This allows constraint direction enhancement filtering, super-resolution filtering, and loop recovery filtering to all be executed in a target data group-level pipeline manner, thereby improving data processing efficiency.

[0064] The data processing apparatus provided in the embodiments of this application will be described below. The content of the data processing apparatus described below can be referred to in correspondence with the content of the method described above.

[0065] In an optional implementation, refer to Figure 6 The exemplary illustration shows an optional structure diagram of the data processing apparatus provided in an embodiment of this application, such as... Figure 6 As shown, the data processing device may include: The super-resolution filtering circuit 11 is used to acquire a data block to be processed, which is data for which horizontal image magnification processing is to be performed; read a target data group in the data block to be processed, wherein the target data group is a data unit located in the same row in the data block to be processed, the data block to be processed is divided into multiple data units with a preset size, and the data unit is the smallest computing unit for image processing; and perform super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group. The loop recovery filter circuit 12 is used to perform loop recovery filtering corresponding to the initial processing result to obtain the target processing result.

[0066] Further reference Figure 7 An exemplary schematic diagram of an optional structure for a super-resolution filtering circuit is shown. Figure 7 As shown, the super-resolution filtering circuit may include three modules: an amplification control module, a preprocessing module, and an amplification module. These three modules work together to complete the horizontal amplification of the image.

[0067] The magnification control module is used to acquire the data block to be processed, which is the data for which horizontal magnification processing of the image is to be performed, and to generate the read address of the target data group and determine the filtering parameters of each data unit in the corresponding target data group. Figure 7 As shown, the scaling control module includes a buffer reading module (buffer_rd), a pixel position calculation module (pixel_loc_calc), a finite state machine (fsm2), a pipeline control module (pipe_ctrl), and an upsampling control generation module (upscale_ctrl_gen).

[0068] The buffer read module `buffer_rd` is used to read data from the buffer. In the circuit, the buffer is used to store temporary data. The `buffer_rd` module can read the data block `data_in` to be processed from the buffer based on the `data_in_vld` signal. The `data_in_vld` signal indicates whether the data in `data_in` is valid. When the `data_in_vld` signal is valid (e.g., represented by 1), it means that the data in `data_in` is valid, and the amplification control module can read and process this data; when the `data_in_vld` signal is invalid (e.g., represented by 0), it means that the data in `data_in` is invalid, and the amplification control module cannot read and process this data.

[0069] The pixel_loc_calc module is used to calculate the position information of pixels. In image processing, determining the coordinates of each pixel within an image is crucial, especially in super-resolution processing. Precise position information helps in accurately processing and reconstructing images. For example, during image upscaling, the corresponding positions of pixels in the new image need to be calculated based on their positions in the original image to achieve pixel interpolation and resampling.

[0070] The finite state machine fsm2 is responsible for coordinating and controlling the entire amplification control module's workflow. Based on the input control signal ctrl_in, the ctrl_in_vld signal indicating whether ctrl_in is valid, and the internal state of the amplification control module, it determines the working order and timing of each sub-module within the amplification control module, ensuring the correctness and orderliness of the amplification control process.

[0071] The pipeline control module `pipe_ctrl` is used to control the pipeline operations within the magnification module. In image processing, a pipelined architecture is often used to improve processing speed, dividing the processing into multiple stages, each with a different module performing a specific task. `pipe_ctrl` is responsible for coordinating the data flow between these stages, ensuring smooth data transmission and processing within the pipeline.

[0072] The upsampling control generation module `upscale_ctrl_gen` is used to generate control signals for upsampling (image magnification). In super-resolution filtering, upsampling is a key step in achieving image magnification. The upsampling control generation module can generate corresponding control signals based on information such as the image magnification factor and algorithm requirements, guiding the magnification module on how to upsample the input image data.

[0073] The preprocessing module is used to read the target data group from the data block to be processed based on the read address, and store the target data group into a preset buffer (e.g., ...). Figure 7 As shown in the Ping-Pong Buffer.

[0074] In an optional implementation, the preset buffer may include a first buffer and a second buffer, wherein the first buffer is used to store the current target data group, and the second buffer is used to store the next target data group. Specifically, as shown below... Figure 7 As shown, when the first buffer tileline_buf0 is read by the amplification module, the second buffer tileline_buf1 can be written to, thus enabling continuous and close read and write operations and improving data processing efficiency.

[0075] Furthermore, such as Figure 7As shown, the preprocessing module also includes a buffer writing module (buffer_wr) and two finite state machines (fsm0 and fsm1). The finite state machines fsm0 and fsm1 are used to control the data flow and processing order. The buffer writing module (buffer_wr) is used to store the input data.

[0076] The amplification module is used to perform super-resolution filtering on the target data group based on the filtering parameters to obtain the initial processing result corresponding to the target data group. Specifically, as shown... Figure 7 As shown, the magnification module can read the target data group `tileline_data` to be processed from the preprocessing module based on the `tileline_vld` signal. Then, using the pixel convolution modules `pixel_conv0`, `pixel_conv1`, ..., `pixel_conv7` in the magnification module, super-resolution filtering (e.g., convolution calculation) is performed on the target data group based on the filtering parameters. After the magnification operation is completed, the magnification module can set the `data_vld_out` signal to an active state and output the initial processing result `data_out` corresponding to the target data group.

[0077] The tileline_vld signal is used to indicate whether the data in tileline_data is valid. When the tileline_vld signal is valid (e.g., represented by 1), it means that the data in tileline_data is valid and the amplification module can read and process this data; when the tileline_vld signal is invalid (e.g., represented by 0), it means that the data in tileline_data is invalid and the amplification module cannot read and process this data.

[0078] The data_vld_out signal is used to indicate to subsequent circuits (e.g., loop recovery filter circuits) whether the initial processing result data_out corresponding to the target data group is valid. When the data_vld_out signal is valid (e.g., represented by 1), it indicates that the data in data_out is valid data after being correctly processed by the amplification module, and subsequent circuits can receive and use it; when the data_vld_out signal is invalid (e.g., represented by 0), it indicates that the data in data_out is invalid, and subsequent circuits cannot receive and use it.

[0079] It should be noted that the target data group read by the amplification module from the preprocessing module is 8*64 pixels in size. The eight rows of data in this target data group can be input into eight corresponding filters, namely the pixel convolution modules pixel_conv0, pixel_conv1, ..., pixel_conv7. Then, each of the eight filters performs calculations on the input data, with each filter outputting one pixel per cycle, resulting in eight pixels output per cycle for all eight filters.

[0080] As the sliding window moves, the filter continuously processes new pixel segments until the entire row of data has been processed. The size of the sliding window is 8 pixels in one row of the target data group.

[0081] Further reference Figure 8 An exemplary schematic diagram of an optional structure for a pixel convolution module is shown, such as... Figure 8 As shown, the pixel convolution module includes eight multipliers (mult0, mult1, ..., mult7), an adder (add), a round calculation module (round), and a clipping module (clip). The eight multipliers (mult0, mult1, ..., mult7) are used to process eight pixels in one row of the target data set.

[0082] The multiplier is used to multiply the eight pixels in one row of the target data group by the corresponding filter parameters.

[0083] The adder is used to sum the multiplication results calculated by the eight multipliers to obtain a convolution result, that is, to sum the product of each pixel and the filter parameters.

[0084] The round module is used to reduce the calculation result output by the adder to a reasonable range. Because the pixel values ​​were magnified in the preceding multiplication and accumulation stages to avoid precision loss, the result needs to be reduced to the original data scale.

[0085] The clipping module is used to limit the range of data. In image processing, pixel values ​​after the rounding calculation may still exceed the normal range of pixel values ​​(for example, for an 8-bit image, pixel values ​​are typically between 0 and 255). The clipping module limits the input pixel values ​​to this reasonable range, avoiding invalid pixel values. For example, if the convolution calculation result is 300, the clipping module will crop it to 255; if the result is -50, it will crop it to 0.

[0086] Optionally, the step of acquiring the data block to be processed may include: acquiring the initial horizontal starting coordinates, initial width, and magnification factor of the data block to be processed.

[0087] Optionally, generating the read address of the target data group may include: Based on the initial horizontal starting coordinates, initial width, and magnification factor of the data block to be processed, calculate the target horizontal starting coordinates and target width of the data block after magnification. Based on the initial horizontal starting coordinates, initial width, target horizontal starting coordinates, and target width, determine the pixel coordinates in the data block to be processed corresponding to the pixel coordinates in the target data block after the data block to be processed is magnified; Based on the pixel coordinates in the target data block corresponding to each pixel coordinate in the target data block, and the row information of the target data group, a read address for reading the target data group is generated.

[0088] Optionally, determining the filtering parameters for each data unit in the corresponding target data group may include: Based on the lookup table, the filtering parameters for each data unit in the corresponding target data group are determined.

[0089] This application also provides an electronic device that may include at least one memory and at least one processor. The memory stores one or more computer-executable instructions, and the processor invokes the one or more computer-executable instructions to execute the data processing method as described above.

[0090] As an optional implementation, refer to Figure 9 , Figure 9 This is an optional block diagram of the electronic device provided in the embodiments of this application. For example... Figure 9 As shown, the electronic device may include: at least one processor 21, at least one communication interface 22, at least one instruction memory 23 and at least one communication bus 24.

[0091] In this embodiment, the number of processor 21, communication interface 22, instruction memory 23 and communication bus 24 is at least one, and processor 21, communication interface 22 and instruction memory 23 communicate with each other through communication bus 24.

[0092] Optionally, the processor 21 may be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural-network Processing Unit), FPGA (Field Programmable Gate Array), TPU (Tensor Processing Unit), AI chip, ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0093] Optionally, the communication interface 22 can be an interface for a communication module used for network communication.

[0094] Instruction memory 23 may include high-speed RAM and may also include non-volatile memory, such as at least one disk storage device. Instruction memory 23 stores one or more computer-executable instructions, which processor 21 invokes to perform the data processing method described above.

[0095] This application embodiment also provides a storage medium that stores one or more computer-executable instructions, which, when executed, implement the data processing method described above.

[0096] This application also provides a computer program product that may include one or more computer-executable instructions, which, when executed, implement the data processing method described above.

[0097] The foregoing describes multiple embodiment schemes provided by the embodiments of this application. The optional methods described in each embodiment scheme can be combined and cross-referenced with each other without conflict, thereby extending to a variety of possible embodiment schemes. These can all be considered as the embodiment schemes disclosed and published by the embodiments of this application.

[0098] While the embodiments disclosed above are described in this application, this application is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. A data processing method, characterized in that, include: Obtain the data block to be processed, which is the data for which horizontal image magnification processing is to be performed; Read the target data group in the data block to be processed, wherein the target data group is the data unit located in the same row in the data block to be processed, the data block to be processed is divided into multiple data units with a preset size, and the data unit is the smallest computing unit for graphics processing; Super-resolution filtering is performed on the target data set to obtain the initial processing result corresponding to the target data set; Perform loop recovery filtering corresponding to the initial processing result to obtain the target processing result.

2. The data processing method according to claim 1, characterized in that, The step of obtaining the data block to be processed includes: Obtain the initial horizontal starting coordinates, initial width, and magnification of the data block to be processed.

3. The data processing method according to claim 2, characterized in that, After the step of acquiring the data block to be processed and before the step of reading the target data group in the data block to be processed, the method further includes: Generate the read address for the target data group; Determine the filtering parameters for each data unit in the corresponding target data set.

4. The data processing method according to claim 3, characterized in that, The step of generating the read address of the target data group includes: Based on the initial horizontal starting coordinates, initial width, and magnification factor of the data block to be processed, calculate the target horizontal starting coordinates and target width of the data block after magnification. Based on the initial horizontal starting coordinates, initial width, target horizontal starting coordinates, and target width, determine the pixel coordinates in the data block to be processed corresponding to the pixel coordinates in the target data block after the data block to be processed is magnified; Based on the pixel coordinates in the target data block corresponding to each pixel coordinate in the target data block, and the row information of the target data group, a read address for reading the target data group is generated.

5. The data processing method according to claim 3, characterized in that, The step of determining the filtering parameters for each data unit in the corresponding target data group includes: Based on the lookup table, the filtering parameters for each data unit in the corresponding target data group are determined.

6. The data processing method according to claim 3, characterized in that, The step of reading the target data group in the data block to be processed includes: Based on the read address, the target data group in the data block to be processed is read and the target data group is stored in a preset buffer.

7. The data processing method according to claim 6, characterized in that, The preset buffer includes a first buffer and a second buffer. The first buffer is used to store the current target data group, and the second buffer is used to store the next target data group.

8. The data processing method according to claim 3, characterized in that, The step of performing super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group includes: Based on the filtering parameters, super-resolution filtering is performed on the target data group to obtain the initial processing result corresponding to the target data group.

9. A data processing apparatus, characterized in that, include: A super-resolution filtering circuit is used to acquire a data block to be processed, wherein the data block to be processed is the data for which horizontal image magnification processing is to be performed; Read the target data group in the data block to be processed, wherein the target data group is the data unit located in the same row in the data block to be processed, the data block to be processed is divided into multiple data units with a preset size, and the data unit is the smallest computing unit for graphics processing; and perform super-resolution filtering on the target data group to obtain the initial processing result corresponding to the target data group. The loop recovery filter circuit is used to perform loop recovery filtering corresponding to the initial processing result to obtain the target processing result.

10. The data processing apparatus according to claim 9, characterized in that, The super-resolution filtering circuit includes: The magnification control module is used to acquire the data block to be processed, which is the data of the image to be horizontally magnified, and to generate the read address of the target data group and determine the filtering parameters of each data unit in the corresponding target data group. The preprocessing module is used to read the target data group in the data block to be processed based on the read address, and store the target data group in a preset buffer; An amplification module is used to perform super-resolution filtering on the target data group based on the filtering parameters to obtain the initial processing result corresponding to the target data group.

11. The data processing apparatus according to claim 10, characterized in that, The preset buffer includes a first buffer and a second buffer. The first buffer is used to store the current target data group, and the second buffer is used to store the next target data group.

12. An electronic device, characterized in that, It includes at least one memory and at least one processor, the memory storing one or more computer-executable instructions, the processor invoking the one or more computer-executable instructions to perform the data processing method as described in any one of claims 1 to 8.

13. A storage medium, characterized in that, The storage medium stores one or more computer-executable instructions, which, when executed, implement the data processing method as described in any one of claims 1 to 8.

14. A computer program product, characterized in that, It includes one or more computer-executable instructions, which, when executed, implement the data processing method as described in any one of claims 1 to 8.

Images