Image data processing method and device, computer device and storage medium

Abstract

The application relates to an image data processing method, device and equipment and a storage medium. The method comprises the following steps: receiving image data to be processed, the image data to be processed comprising a pixel set extracted from an image to be processed according to a selected data input mode and input at a current time point, the pixel set being composed of a plurality of continuous pixels in a first pixel arrangement direction, performing parallel interpolation processing on the pixel set according to the first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set, when there is a previous interpolation result of the current interpolation result, performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction perpendicular to the first pixel arrangement direction to obtain an interpolation result corresponding to adjacent pixel sets, and splicing the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain processed image data. The method can achieve the effects of low hardware resource occupation and high image processing real-time performance.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] Currently, image scaling is widely used in the field of image or video display. Commonly used image scaling algorithms include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and higher-order interpolation algorithms.

[0003] Because LED (light emitting diode) displays are spliced ​​screens, their resolution may not exactly match the image resolution. Therefore, image scaling is often required when displaying images or videos on LED displays, and scaling is also used in special scenarios where multiple video sources need to be displayed. For example, when inputting a 4K or 8K resolution video source at 60 frames per second, the scaling cores on commonly used commercial hardware operate at a frequency of approximately 150MHz to 200MHz, processing one display pixel per clock cycle. An 8K@30fps screen display would require 5 to 7 scaling cores. When applied to an LED control system, the 8K video source is usually split by the sending card to output via 8 or 16 network ports, requiring 40 to 112 scaling cores. Obviously, this number of cores consumes unreasonable hardware resources. The existing multi-core scaling function in LED control systems requires a complete image cache, followed by the division of the cached image into multiple image sub-blocks. These sub-blocks are then simultaneously input into multiple scaling cores for parallel processing. Because image caching is required for parallel processing, the real-time performance of image scaling decreases, and caching high-resolution images consumes a significant amount of resources.

[0004] Therefore, there is a need to provide a solution that achieves higher real-time performance in image processing using fewer hardware resources. Summary of the Invention

[0005] Therefore, it is necessary to provide an image data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that consumes less hardware resources and has high real-time image processing performance to address the above-mentioned technical problems.

[0006] Firstly, this application provides an image data processing method. The method includes:

[0007] Receive image data to be processed, which includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0008] Parallel interpolation is performed on the pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0009] When there is a previous interpolation result for the current interpolation result, parallel interpolation processing is performed on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0010] The interpolation results corresponding to each set of adjacent pixels of the image to be processed are collected, and then combined and processed to obtain the processed image data.

[0011] In one embodiment, parallel interpolation of the pixel set includes:

[0012] Obtain the position information of each pixel in the pixel set;

[0013] Based on the position information of each pixel, multiple arrays of first-type interpolation coefficients are obtained from the preset first-type interpolation coefficient table.

[0014] Parallel interpolation processing is performed on the pixel set based on multiple arrays of first-type interpolation coefficients;

[0015] The first type of interpolation coefficient table contains the mapping relationship between the position information of the pixel to be interpolated and the first type of interpolation coefficient array. The number of interpolation coefficients in the first type of interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

[0016] In one embodiment, performing parallel interpolation processing on a pixel set based on a plurality of first-type interpolation coefficient arrays includes:

[0017] Starting with each pixel in the pixel set, adjacent pixels are taken to obtain adjacent pixel groups. Each adjacent pixel group is then input into the built-in first-type interpolation processing kernel. Based on the position information of the first pixel in the adjacent pixel group, the first-type interpolation coefficient array is input into the corresponding first-type interpolation processing kernel. This allows the first-type interpolation processing kernel to perform interpolation processing on each of the input adjacent pixel groups based on the input first-type interpolation coefficient array, thus obtaining the interpolation array corresponding to each adjacent pixel group.

[0018] In one embodiment, parallel interpolation processing of the previous interpolation result and the current interpolation result includes:

[0019] Obtain the starting row and column positions of the pixel set, as well as the number of rows and columns involved in the interpolation process;

[0020] Based on the starting row and column positions and the number of rows and columns involved in the interpolation, multiple arrays of second-type interpolation coefficients are obtained from the preset second-type interpolation coefficient table.

[0021] Based on multiple second-type interpolation coefficient arrays, perform parallel interpolation processing on the previous interpolation result and the current interpolation result;

[0022] The second type of interpolation coefficient table contains the mapping relationship between the position information of the pixel to be interpolated and the second type of interpolation coefficient array. The number of interpolation coefficients in the second type of interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

[0023] In one embodiment, the previous interpolation result and the current interpolation result include multiple interpolation arrays;

[0024] Parallel interpolation processing of the previous and current interpolation results based on multiple arrays of second-type interpolation coefficients includes:

[0025] The first interpolation array from the previous interpolation result and the second interpolation array from the current interpolation result are extracted sequentially. The first interpolation array and the second interpolation array are combined to obtain multiple target interpolation arrays.

[0026] The target interpolation array and the second type of interpolation coefficient array are respectively input into the built-in second type of interpolation processing cores, so that each second type of interpolation processing core performs interpolation processing on the input target interpolation array according to the input second type of interpolation coefficient array.

[0027] In one embodiment, prior to performing parallel interpolation on the pixel set, the method further includes:

[0028] Select the valid pixels to be interpolated from the pixel set to obtain the valid pixel set;

[0029] Parallel interpolation of the pixel set includes:

[0030] Parallel interpolation is performed on the effective pixel set.

[0031] In one embodiment, prior to performing parallel interpolation on the pixel set, the method further includes:

[0032] Filter out the valid pixels in the pixel set that need interpolation and the invalid pixels that do not need interpolation;

[0033] Parallel interpolation of the pixel set includes:

[0034] The built-in first-type interpolation processing kernel that corresponds to invalid pixels is disabled, so that the remaining first-type interpolation processing kernels can perform parallel interpolation processing on valid pixels.

[0035] In one embodiment, the data input mode includes a point-to-line input mode, a point-to-column input mode, a multi-line point-to-point input mode, or a multi-column point-to-point input mode.

[0036] If the data input mode is point-by-point input mode, then the pixel set consists of multiple consecutive pixels located in the horizontal pixel arrangement direction;

[0037] If the data input mode is point-by-point input mode, then the pixel set consists of multiple consecutive pixels located in the vertical pixel arrangement direction;

[0038] If the data input mode is multi-line point-by-point input mode, then the pixel set consists of multiple pixels with the same column number and consecutive row number;

[0039] If the data input mode is multi-column point-by-point input mode, then the pixel set consists of multiple pixels with the same row number and consecutive column numbers.

[0040] In one embodiment, the data input mode includes a point-to-line input mode, a point-to-column input mode, a multi-line point-to-point input mode, or a multi-column point-to-point input mode, and the image data to be processed includes the image scaling ratio, as well as the row position information and / or column position information of each pixel in the pixel set.

[0041] The valid pixels to be interpolated in the pixel set include:

[0042] If the data input mode is point-by-point and line-by-line input mode, the effective pixels to be interpolated in the pixel set are selected according to the image scaling ratio and the row position information of each pixel in the pixel set.

[0043] If the data input mode is point-by-point and column-by-column input mode, the effective pixels to be interpolated in the pixel set are selected according to the image scaling ratio and the column position information of each pixel in the pixel set.

[0044] If the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, then the effective pixels to be interpolated in the pixel set are selected based on the image scaling ratio, row position information, and column position information.

[0045] Secondly, this application also provides an image data processing apparatus. The apparatus includes:

[0046] The data receiving module is used to receive image data to be processed. The image data to be processed includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0047] The first parallel interpolation processing module is used to perform parallel interpolation processing on the pixel set according to the first pixel arrangement direction, obtain the current interpolation result corresponding to the pixel set, and cache it. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0048] The second parallel interpolation processing module is used to perform parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction when there is a previous interpolation result in the cached data. This process yields the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0049] The data merging module is used to collect the interpolation results corresponding to each set of adjacent pixels of the image to be processed, and to combine and process the interpolation results corresponding to each set of adjacent pixels of the image to be processed to obtain the processed image data.

[0050] Thirdly, this application also provides an image data processing device. The device includes a data filtering module, a first type of interpolation module, a second type of interpolation module, a data cache management module, and a data merging module.

[0051] The data filtering module receives the pixel set extracted from the image to be processed according to the selected data input mode at the current time point, and inputs the pixel set to the first type of interpolation module. The first type of interpolation module performs parallel interpolation processing on the received pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set, and sends the current interpolation result to the data cache management module for caching. When the data cache management module detects that there is a previous interpolation result in the cached data, it sends the current interpolation result and the previous interpolation result to the second type of interpolation module. The second type of interpolation module performs parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set, and outputs the interpolation result corresponding to the adjacent pixel set to the data merging module. The data merging module collects the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and performs splicing and merging processing on the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain the processed image data, and outputs the processed image data.

[0052] The pixel set consists of multiple consecutive pixels located in the first pixel arrangement direction. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0053] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0054] Receive image data to be processed, which includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0055] Parallel interpolation is performed on the pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0056] When there is a previous interpolation result for the current interpolation result, parallel interpolation processing is performed on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0057] The interpolation results corresponding to each set of adjacent pixels of the image to be processed are collected, and then combined and processed to obtain the processed image data.

[0058] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0059] Receive image data to be processed, which includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0060] Parallel interpolation is performed on the pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0061] When there is a previous interpolation result for the current interpolation result, parallel interpolation processing is performed on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0062] The interpolation results corresponding to each set of adjacent pixels of the image to be processed are collected, and then combined and processed to obtain the processed image data.

[0063] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0064] Receive image data to be processed, which includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0065] Parallel interpolation is performed on the pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0066] When there is a previous interpolation result for the current interpolation result, parallel interpolation processing is performed on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0067] The interpolation results corresponding to each set of adjacent pixels of the image to be processed are collected, and then combined and processed to obtain the processed image data.

[0068] The aforementioned image data processing methods, apparatuses, devices, computer equipment, storage media, and computer program products, on the one hand, take continuous pixels extracted from the image to be processed as input and perform parallel interpolation processing on the input pixels. This eliminates the need for image caching and segmentation, achieving a "pipeline"-style real-time image processing approach where processing occurs immediately upon input, thus resulting in higher real-time performance. On the other hand, since the input pixel set has a temporal order and is not input simultaneously, parallel interpolation processing of pixel sets input at different times along the first pixel arrangement direction can share a single interpolation processing kernel. Therefore, the number of interpolation processing kernels used can be reduced, saving hardware resources. In summary, this solution achieves the effect of low hardware resource consumption and high real-time image processing performance. Attached Figure Description

[0069] Figure 1 This is an application environment diagram of an image data processing method in one embodiment;

[0070] Figure 2This is a flowchart illustrating an image data processing method in one embodiment;

[0071] Figure 3 This is a flowchart illustrating an image data processing method in another embodiment;

[0072] Figure 4 This is a detailed flowchart of the image data processing method in another embodiment;

[0073] Figure 5 This is a structural block diagram of an image data processing device in one embodiment;

[0074] Figure 6 This is a structural block diagram of an image data processing apparatus in another embodiment;

[0075] Figure 7 This is a structural block diagram of an image data processing device in one embodiment;

[0076] Figure 8 This is a structural block diagram of an image data processing device in one embodiment;

[0077] Figure 9 This is a schematic diagram of the signal flow of an image data processing device in one embodiment;

[0078] Figure 10 This is an internal structure diagram of the data filtering module in one embodiment;

[0079] Figure 11 This is an internal structure diagram of the scaling factor calculation module in one embodiment;

[0080] Figure 12 This is an internal structure diagram of the lateral interpolation module in one embodiment;

[0081] Figure 13 This is an internal structure diagram of the pixel interpolation submodule in one embodiment;

[0082] Figure 14 This is an internal structure diagram of the data cache management module in one embodiment;

[0083] Figure 15 This is an internal structure diagram of the cache submodule in one embodiment;

[0084] Figure 16 This is an internal structure diagram of the vertical interpolation module in one embodiment. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0086] The image data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the host computer 102 and the image processing terminal 104 communicate with each other. Taking image scaling as an example, the operator uploads the image to be processed to the host computer 102, selects the corresponding data input mode, inputs the corresponding image processing data such as the size information before and after image scaling, and sends an image processing command to the host computer 102. The host computer 102 receives the image processing data and extracts a certain number of consecutive pixels from the image to be processed according to the selected data input mode to obtain the image data to be processed. Then, the extracted pixel set and the image processing data are sequentially input to the image processing terminal 104. The image processing terminal 104 receives the image data to be processed, which includes the pixel set extracted from the image to be processed according to the selected data input mode at the current time point. The pixel set consists of multiple consecutive pixels located in the first pixel arrangement direction. The image is composed of pixels. Then, following the first pixel arrangement direction, parallel interpolation is performed on the pixel set to obtain the current interpolation result corresponding to the pixel set. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set. When a previous interpolation result exists, parallel interpolation is performed on the previous and current interpolation results following the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction. The interpolation results corresponding to each group of adjacent pixel sets in the image to be processed are collected and combined and processed to obtain the processed image data. The processed image data is then output to an external data receiving end, such as a receiving card. The host computer 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The image processing unit 104 includes, but is not limited to, a transmitting card for the LED control system.

[0087] In one embodiment, such as Figure 2 As shown, an image data processing method is provided, which is applied to... Figure 1 Taking the image processing end in the image as an example, the following steps are included:

[0088] Step 100: Receive image data to be processed. The image data to be processed includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0089] In this embodiment, image scaling is used as an example for explanation. The data input mode refers to the pixel input mode of the image, including row-by-row and pixel-by-pixel input (simply referred to as point-by-row input mode). Point-by-row input mode means taking the pixels of the first row as the input pixels at the current time point and inputting the pixels of the first row sequentially. The image data to be processed includes image size information, scaling ratio, and the pixel set extracted from the image to be processed according to the selected data input mode at the current time point. The pixel set consists of multiple consecutive pixels located in the first pixel arrangement direction; specifically, it can consist of multiple consecutive pixels in memory located in the first pixel arrangement direction. If the data input mode is point-by-row input mode, then the pixel set consists of multiple consecutive pixels located in the horizontal pixel arrangement direction. It is understood that the data input mode can also be point-by-column input, or other input modes, depending on the actual situation, and is not limited here. In practice, the pixels of the image to be processed are continuously input in a "data stream" manner according to the selected data input mode. That is, the input pixel set is processed in the order of data input, so as to achieve a "pipeline" operation of processing as soon as input.

[0090] Step 200: Perform parallel interpolation on the pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0091] In this embodiment, bilinear interpolation is used as an example. Upon receiving the pixel set to be processed at the current time, parallel interpolation processing is performed according to the inherent first pixel arrangement direction of each pixel in the pixel set. If the pixel set consists of consecutive pixels in the horizontal pixel arrangement direction, parallel horizontal interpolation processing is performed on the pixel set; if the pixel set consists of consecutive pixels in the vertical pixel arrangement direction, parallel vertical interpolation processing is performed on the pixel set. Specifically, parallel interpolation processing can be performed simultaneously using multiple built-in interpolation processing kernels to obtain the current interpolation result corresponding to the pixel set. The current interpolation result includes multiple interpolation arrays, and the number of elements in each interpolation array is linearly related to the number of elements in the pixel set. The specific linear relationship is determined by the scaling ratio. If the number of pixels in the pixel set is 4, and the scaling ratio is 1:5 (i.e., the ratio of the number of pixels at the pixel input end to the number of pixels at the pixel display end is 1:5), then the current interpolation result corresponding to the pixel set includes 20 pixels. In specific implementation, the current interpolation result corresponding to the pixel set is cached simultaneously. Then, at the next time point, the input pixel set is received. Then, according to the first pixel arrangement direction, the pixel set is subjected to parallel interpolation processing. Since the input of the pixel set has a time sequence, the same interpolation processing kernel can be used to interpolate the pixel set after each pixel set is received. That is, pixel sets input at different time points can share the same interpolation processing kernel for interpolation processing. Specifically, taking the data input mode as point-by-point and line-by-line input as an example, and the first pixel arrangement direction as horizontal, after parallel horizontal interpolation processing is performed on the pixel set and the horizontal interpolation result is obtained, the number of horizontal interpolation results and the interpolation result array can be cached. Each horizontal interpolation processing core has an interpolation result cache area, and each interpolation result cache area has at least the space to cache the number of pixels of the terminal display width for two lines. Although this processing method only needs to cache one line, in order to avoid the second line of data of the current interpolation being the first line of the next interpolation, the current line of interpolation needs to cache two lines of interpolation results. When caching, the interpolation results of odd-numbered lines and even-numbered lines are written to fixed positions. This processing ensures that when inputting pixels, other lines except the first line can always obtain the interpolation result of the previous line.

[0092] Step 300: When there is a previous interpolation result for the current interpolation result, perform parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0093] Following the above embodiments, for each pixel set corresponding to the current interpolation result, the current interpolation result is cached. Therefore, when the cached data contains the previous interpolation result of the current interpolation result, it indicates that parallel interpolation processing in the second pixel arrangement direction can be performed at this time. For example, if starting from time point t0, a pixel set consisting of the pixels of the first row of the image to be processed is input, and parallel horizontal interpolation processing is performed on the pixel set input at t0 to obtain the horizontal interpolation result corresponding to t0, and cached, since the cached data only contains the horizontal interpolation result of one row of pixels, parallel vertical interpolation processing cannot be performed. Therefore, the pixel set consisting of the pixels of the second row of the image to be processed input at time point t1 is received, and horizontal interpolation processing is performed on the received pixel set. When the interpolation result corresponding to the pixel set input at time point t1 is obtained... At this point, the cached data contains two consecutive sets of interpolation results corresponding to time points t0 and t1, i.e., interpolation results for the current row of pixels and the previous row of pixels. Therefore, vertical interpolation processing can be performed. This involves parallel interpolation of the two consecutive sets of interpolation results corresponding to time points t0 and t1 to obtain the interpolation result corresponding to the adjacent pixel set, i.e., time point t2. Once the interpolation result corresponding to time point t2 (the third row of pixels) is obtained, vertical interpolation processing can be performed on the two consecutive sets of interpolation results corresponding to t1 and t2 to obtain the interpolation result corresponding to the adjacent pixel set. In this way, the interpolation result corresponding to each adjacent pixel set of the image to be processed can be obtained.

[0094] Step 400: Collect the interpolation results corresponding to each set of adjacent pixels of the image to be processed, and combine and process the interpolation results corresponding to each set of adjacent pixels of the image to be processed to obtain the processed image data.

[0095] After obtaining the interpolation results corresponding to each set of adjacent pixels of the image to be processed, the interpolation results corresponding to each set of adjacent pixels of the image to be processed can be collected, and then the interpolation results corresponding to each set of adjacent pixels of the image to be processed can be stitched together and processed according to the pixel position to obtain the complete scaled image information so as to display the image data normally.

[0096] The aforementioned image data processing methods, apparatuses, devices, computer equipment, storage media, and computer program products, on the one hand, take continuous pixels extracted from the image to be processed as input and perform parallel interpolation processing on the input pixels. This eliminates the need for image caching and segmentation, achieving a "pipeline"-style real-time image processing approach where processing occurs immediately upon input, thus resulting in higher real-time performance. On the other hand, since the input pixel set has a temporal order and is not input simultaneously, parallel interpolation processing of pixel sets input at different times along the first pixel arrangement direction can share a single interpolation processing kernel. Therefore, the number of interpolation processing kernels used can be reduced, saving hardware resources. In summary, this solution achieves the effect of low hardware resource consumption and high real-time image processing performance.

[0097] In one embodiment, the data input mode includes a point-to-line input mode, a point-to-column input mode, a multi-line point-to-point input mode, or a multi-column point-to-point input mode.

[0098] If the data input mode is point-by-point input mode, then the pixel set consists of multiple consecutive pixels located in the horizontal pixel arrangement direction;

[0099] If the data input mode is point-by-point input mode, then the pixel set consists of multiple consecutive pixels located in the vertical pixel arrangement direction;

[0100] If the data input mode is multi-line point-by-point input mode, then the pixel set consists of multiple pixels with the same column number and consecutive row number;

[0101] If the data input mode is multi-column point-by-point input mode, then the pixel set consists of multiple pixels with the same row number and consecutive column numbers.

[0102] In this embodiment, the data input modes include point-to-line input mode, point-to-column input mode, multi-row point-to-point input mode, or multi-column point-to-point input mode. In practical applications, any one of the above data input modes can be selected for pixel input. Specifically, if the data input mode is point-to-line input mode, the pixel set consists of multiple consecutive pixels located in the horizontal pixel arrangement direction. In simpler terms, it consists of consecutive pixels located in the same row. Starting from the first row, all pixels in the first row are input sequentially, or multiple pixels in the first row are input at once. If the data input mode is point-to-column input mode, the pixel set consists of multiple consecutive pixels located in the vertical pixel arrangement direction. In simpler terms, it consists of consecutive pixels located in the same column. Starting from the first column, all pixels in the first column are input sequentially, or multiple pixels in the first column are input at once. If the data input mode is a multi-row point-by-point input mode, the pixel set consists of multiple pixels with the same column number and consecutive row numbers. That is, inputting a portion of pixels from multiple consecutive rows can be done by inputting the first pixel of the first, second, third, and fourth rows at a time, then inputting the second pixel of the first, second, third, and fourth rows at the next time point, and so on, until all pixels of the image to be processed have been input. If the data input mode is a multi-column point-by-point input mode, the pixel set consists of multiple pixels with the same row number and consecutive column numbers. That is, inputting a portion of pixels from multiple consecutive columns can be done by inputting the first pixel of the first, second, third, and fourth columns at a time, then inputting the second pixel of the first, second, third, and fourth columns at the next time point, and so on, until all pixels of the image to be processed have been input. Specifically, the data input mode can be determined according to the actual situation and is not limited here. It is understood that the data input mode includes, but is not limited to, the four data input modes mentioned above, and can also be other input modes, as long as the input pixels are multiple consecutive pixels, it should be within the scope of this embodiment. In this embodiment, by providing multiple data input modes, the options are greater and the flexibility is more pronounced.

[0103] In one embodiment, before performing parallel interpolation on the pixel set, the method further includes: filtering out the valid pixels to be interpolated in the pixel set to obtain a valid pixel set; step 200 includes: step 220, performing parallel interpolation on the valid pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the valid pixel set, wherein the number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0104] In practical applications, not all input pixels require interpolation calculations. Therefore, before performing parallel interpolation, the pixel set can be filtered to identify the valid pixels to be interpolated, resulting in a valid pixel set. Then, parallel interpolation is performed on this valid pixel set. In this embodiment, the pixel set is filtered, which saves unnecessary interpolation processing and improves data processing speed.

[0105] like Figure 3 As shown, in one embodiment, before performing parallel interpolation processing on the pixel set, the method further includes: step 120, filtering out the valid pixels to be interpolated and the invalid pixels that do not need interpolation in the pixel set; step 200 includes: step 220, turning off the first type of interpolation processing kernel that corresponds to the invalid pixels, so that the remaining first type of interpolation processing kernel performs parallel interpolation processing on the valid pixel set according to the first pixel arrangement direction, to obtain the current interpolation result corresponding to the valid pixel set, wherein the number of elements in the current interpolation result is linearly related to the number of elements in the valid pixel set.

[0106] In practical applications, multiple first-type interpolation kernels can be used to perform parallel interpolation processing on the input pixels, and multiple second-type interpolation kernels can be used for secondary parallel interpolation processing. The number of first-type interpolation kernels is equal to the number of second-type interpolation kernels, and the number is determined by the maximum magnification that the image processing end can support. For example, if the maximum magnification is 16x, then 16 first-type interpolation kernels and 16 second-type interpolation kernels are set to support parallel interpolation processing of pixels. In this embodiment, for ease of understanding, the first-type interpolation kernel is used as a horizontal interpolation kernel, and the second-type interpolation kernel is used as a vertical interpolation kernel for explanation. Before performing parallel interpolation on pixels, valid pixels to be interpolated and invalid pixels that do not require interpolation can still be filtered out. Since the number of built-in horizontal and vertical interpolation kernels is preset, if an invalid pixel that does not require interpolation is detected, the corresponding horizontal interpolation kernel that was previously assigned to process the invalid pixel can be disabled via an enable signal, stopping its processing or disabling its output validity, so that parallel interpolation can be performed on the valid pixels. Because the horizontal and vertical interpolation kernels are arranged in a one-to-one correspondence, if a horizontal interpolation kernel does not need to work, its corresponding vertical interpolation kernel can be disabled via an enable signal. This saves on the corresponding interpolation kernels.

[0107] In one embodiment, the data input mode includes a point-to-line input mode, a point-to-column input mode, a multi-line point-to-point input mode, or a multi-column point-to-point input mode, and the image data to be processed includes the image scaling ratio, as well as the row position information and / or column position information of each pixel in the pixel set.

[0108] Step 120 includes:

[0109] Step 122: If the data input mode is point-by-point and line-by-line input mode, select the effective pixels to be interpolated in the pixel set according to the image scaling ratio and the row position information of each pixel in the pixel set.

[0110] Step 124: If the data input mode is point-by-point and column-by-column input mode, select the effective pixels to be interpolated in the pixel set according to the image scaling ratio and the column position information of each pixel in the pixel set.

[0111] Step 126: If the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, then select the effective pixels to be interpolated in the pixel set according to the image scaling ratio, row position information and column position information.

[0112] In practice, the selection of pixels to be interpolated depends on the chosen data input mode. Specifically, it can be as follows: If the data input mode is point-by-point and row-by-row input, the effective pixels to be interpolated in the pixel set are selected based on the image scaling ratio and the row position information (which can be considered as row numbers) of each pixel in the pixel set. If the data input mode is point-by-point and column-by-column input, the effective pixels to be interpolated in the pixel set are selected based on the image scaling ratio and the column position information of each pixel in the pixel set. If the data input mode is multi-row point-by-point input or multi-column point-by-point input, the effective pixels to be interpolated in the pixel set are selected based on the image scaling ratio, row position information, and column position information. Specifically, it can be determined whether a pixel needs interpolation based on whether the product of the image scaling ratio and the row number of a pixel increases by an integer. For example, if the data input mode is point-by-point and line-by-line, and if pixel 1's row number is 9, pixel 2's row number is 10, and the image scaling ratio is 1 / 5, then the product of pixel 1 and the image scaling ratio is 9 / 5, and the product of pixel 2 and the image scaling ratio is 2. The product jumps from 9 / 5 to 2, a transition to an integer at pixel 2. Therefore, pixel 1 is an invalid pixel, and pixel 2 is a valid pixel. Similarly, for other data input modes, valid pixels can be filtered in the same way. In this embodiment, different filtering methods for valid pixel sets are set according to different data input modes, enabling quick filtering of valid pixels and providing greater flexibility and convenience.

[0113] like Figure 4As shown, in one embodiment, step 200 includes: step 222, obtaining the position information of each pixel in the pixel set, obtaining multiple first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table based on the position information of each pixel, and performing parallel interpolation processing on the pixel set according to the first pixel arrangement direction based on the multiple first-type interpolation coefficient arrays, thereby obtaining the first-type interpolation coefficient table containing the mapping relationship between the position information of the pixel to be interpolated and the first-type interpolation coefficient array, wherein the number of interpolation coefficients in the first-type interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

[0114] Since parallel interpolation is performed on the input pixel set, multiple interpolation coefficients need to be retrieved at once for interpolation. In this embodiment, the interpolation coefficients can be obtained using a pre-calculated mapping table. Specifically, the interpolation coefficient table can be calculated and stored based on the size information of the input image (including the width and height of the input image, and the width and height of the scaled image). The mapping relationship is that the interpolation coefficients are mapped to the positions of the input interpolation pixels. In this embodiment, bilinear interpolation is split into horizontal linear interpolation and vertical linear interpolation, so there are two types of interpolation coefficient tables: a horizontal linear interpolation coefficient table and a vertical linear interpolation coefficient table. The relationship between the interpolation coefficients and the interpolation positions is as follows: the interpolation coefficient array is bound to the position of the first pixel among the two input pixels involved in the interpolation, that is, one position corresponds to multiple interpolation coefficients cached. Since pixel interpolation is generally performed on two consecutive pixels, and the position of the first pixel among the two input pixels involved in the interpolation is bound to the interpolation coefficient array, the first type of interpolation coefficient array can be obtained based on the position of the first pixel among the two input pixels involved in the interpolation. When caching the interpolation coefficient table, the number of coefficients and multiple coefficients are cached together for convenient mapping and lookup. Cache calculations are performed using one horizontal row and one vertical column, with a consistent amount of data cached at each position. This data is determined by the maximum supported magnification factor (the maximum magnification factor is an integer; otherwise, it is rounded up) + 1, used to store the coefficients and the number of coefficients. The scaling factor determines how many pixels will be directly interpolated between two input interpolation positions. When scaling down, some positions may not participate in interpolation; in this case, both the number of cached coefficients and the number of coefficients are 0. When scaling up, the magnification factor k is not necessarily an integer, and the number of coefficients between two input pixel positions may vary between rounding down and rounding up k. Furthermore, the number of interpolation coefficients for each pixel to be interpolated is determined based on the pixel's position; that is, the number of interpolation coefficients for a pixel is calculated from the position of the input pixel, unlike the traditional method of calculating the number of interpolation coefficients based on the position of the output pixel. In this embodiment, by pre-calculating the interpolation coefficients and caching them in the form of an interpolation coefficient table, the interpolation coefficients can be retrieved accurately and quickly, facilitating interpolation calculations.

[0115] In one embodiment, parallel interpolation processing of a pixel set based on multiple first-type interpolation coefficient arrays includes: starting with each pixel in the pixel set, taking adjacent pixels to obtain adjacent pixel groups, sequentially inputting each adjacent pixel group into each built-in first-type interpolation processing kernel, and inputting the first-type interpolation coefficient array into the corresponding first-type interpolation processing kernel according to the position information of the first pixel in the adjacent pixel group, so that the first-type interpolation processing kernel performs interpolation processing on each of the input adjacent pixel groups according to the input first-type interpolation coefficient array to obtain the interpolation array corresponding to each adjacent pixel group.

[0116] In this embodiment, the first type of interpolation processing kernel is also used as an example of the horizontal interpolation processing kernel. It can be that each pixel in the pixel set is taken as the starting point, and adjacent pixels are obtained to form adjacent pixel groups. Each adjacent pixel group is then sequentially input into the built-in first type of interpolation processing kernel. Based on the position information of the first pixel in each adjacent pixel group, the first type of interpolation coefficient array is input into the corresponding first type of interpolation processing kernel, so that each first type of interpolation processing kernel performs interpolation processing on the input adjacent pixel groups according to the received first type of interpolation coefficient array, obtaining the interpolation array corresponding to each adjacent pixel group. For example, to illustrate, let's consider four consecutive pixel inputs. If P5, P6, P7, and P8 represent four connected pixels in a row, then the last pixel input at the previous time point, P4 and P5, are input to the first horizontal interpolation kernel; P5 and P6 to the second; P6 and P7 to the third; and P7 and P8 to the fourth. The corresponding interpolation coefficient arrays are then input to the respective horizontal interpolation kernels. Each horizontal interpolation kernel, upon receiving the interpolation coefficient array, performs interpolation calculations based on the horizontal position of the received pixel (e.g., row number). Using the last pixel from the previous input pixel set and the first pixel of the current input as adjacent pixel groups ensures the continuity of the interpolation process. It's understandable that if it's the first round of pixel input, the last pixel from the previous time point is empty; in this case, the first horizontal interpolation kernel doesn't need to perform interpolation. In this embodiment, by inputting adjacent pixel groups and corresponding horizontal interpolation coefficient arrays into the corresponding horizontal interpolation processing kernels, parallel computation of horizontal interpolation can be achieved, thereby improving the real-time performance of image processing.

[0117] like Figure 4As shown, in one embodiment, step 300 includes: step 320, obtaining the starting row and column positions of the pixel set and the number of rows and columns involved in the interpolation process; obtaining multiple second-type interpolation coefficient arrays from a preset second-type interpolation coefficient table according to the starting row and column positions and the number of rows and columns involved in the interpolation; and performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction based on the multiple second-type interpolation coefficient arrays. The second-type interpolation coefficient table contains the mapping relationship between the position information of the pixels to be interpolated and the second-type interpolation coefficient arrays, and the number of interpolation coefficients in the second-type interpolation coefficient arrays is determined based on the position information of each pixel in the pixel set.

[0118] In this embodiment, the second type of interpolation coefficient table is used as an example of a vertical interpolation coefficient table, and the second type of interpolation processing kernel is used as an example of a vertical interpolation processing kernel. The vertical interpolation coefficient array can be obtained as follows: obtain the starting row and column positions of the input pixels, and the number of rows and columns involved in the interpolation processing. If the data input mode is a point-by-point and row-by-row input mode or a multi-row and point-by-point input mode, then obtain the starting row position of the input pixels and the number of rows involved in the interpolation. Starting from the starting row position, according to the number of rows involved in the interpolation, continuously obtain the vertical interpolation coefficient array of the corresponding number of rows. Then, based on the multiple vertical interpolation coefficient arrays, perform parallel vertical interpolation processing on the previous interpolation result and the current interpolation result. Similarly, if the data input mode is point-by-point or multi-column point-by-point input, the starting row and column positions of the input pixels, as well as the number of rows and columns involved in the interpolation process, are obtained. If the data input mode is point-by-line or multi-row point-by-point input, the starting column position of the input pixels, as well as the number of columns involved in the interpolation, are obtained. Starting from the starting column position, the vertical interpolation coefficient arrays of the corresponding number of columns involved in the interpolation are continuously retrieved. Then, based on the multiple second-type interpolation coefficient arrays, the previous interpolation result and the current interpolation result are subjected to parallel interpolation processing. In this embodiment, by pre-calculating the interpolation coefficients and caching them in the form of an interpolation coefficient table, the interpolation coefficients can be retrieved accurately and quickly, facilitating the calculation of the interpolation process.

[0119] In one embodiment, the previous interpolation result and the current interpolation result include multiple interpolation arrays; performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to multiple second-type interpolation coefficient arrays includes: sequentially extracting the first interpolation array from the previous interpolation result and the second interpolation array from the current interpolation result, combining the first interpolation array and the second interpolation array to obtain multiple target interpolation arrays, and inputting the target interpolation arrays and the second-type interpolation coefficient arrays into each built-in second-type interpolation processing kernel, so that each second-type interpolation processing kernel performs interpolation processing on the input target interpolation array according to the input second-type interpolation coefficient array.

[0120] Following the above embodiments, after obtaining multiple vertical interpolation coefficient arrays, the process can start from the beginning, taking the first interpolation array from the previous interpolation result and the second interpolation array from the current interpolation result one by one, and combining the first and second interpolation arrays to obtain multiple target interpolation arrays. Specifically, if the data input mode is point-by-point row-by-row or point-by-point column-by-column input, since the input pixels have a temporal order, they can share a single kernel for interpolation processing. The horizontal interpolation result cache is based on two consecutive rows or two columns of interpolation results. Therefore, the horizontal interpolation results of the input pixels at the current time point and the previous time point can be stored continuously in one cache area, and the interpolation arrays can be taken directly from the same cache area and combined to obtain the target interpolation array. If the data input mode is multi-row point-by-point or multi-column point-by-point input, the horizontal interpolation results of the input pixels at the current time point and the previous time point are not necessarily stored in one cache area. In this case, the current interpolation result and the previous interpolation result are taken from the corresponding cache area. Then, the target interpolation array and the vertical interpolation coefficient array are input to the built-in vertical interpolation processing kernels respectively. It should be noted that the vertical interpolation coefficients received by each vertical interpolation processing kernel are the same. Each vertical interpolation processing kernel can select the corresponding vertical interpolation coefficient array from the received vertical interpolation coefficient array according to the relative row number or relative column number. The relative row number and relative column number are determined by the difference between the row number and column number of the pixel received by the vertical interpolation processing kernel and the minimum row number or minimum column number of the pixel in the pixel set. If the row numbers of the input pixels are row 5, row 6, row 7 and row 8 respectively, and the row number of the pixel received by the second vertical interpolation processing kernel is row 6, then the relative row number of the second vertical interpolation processing kernel is 2, and the vertical interpolation coefficient array in the second position of the received vertical interpolation coefficient array is taken accordingly. If the input consists of pixels from the same row or column, the relative row number of the vertical interpolation kernel is 0. In this case, the vertical interpolation coefficient array is directly selected based on the row or column number of the received interpolation result for vertical interpolation calculation. In practice, the vertical and horizontal interpolation kernels are essentially the same interpolation kernels, distinguished only by their function. Each vertical interpolation kernel contains m interpolation kernels, where m is the maximum vertical magnification factor. The vertical interpolation kernel splits the input horizontal interpolation result array and inputs it into each interpolation kernel for interpolation processing, ultimately merging the outputs of all the vertical interpolation kernels. In this embodiment, by inputting the target interpolation array and the vertical interpolation coefficient array into the corresponding vertical interpolation kernels, parallel calculation of vertical interpolation can be achieved, improving the real-time performance of image processing.

[0121] To clearly illustrate the image data processing method provided in this application, a specific embodiment is described below. In this embodiment, the data input mode is described point by point and line by line, specifically including the following:

[0122] The staff selects the data input mode as point-by-point and line-by-line input mode on the host computer, extracts the row pixels of the image to be processed line by line to obtain the pixel set, sends the size information, scaling ratio and other parameters of the image to be processed to the image processing terminal, and transmits the extracted pixel set to the image processing terminal in sequence according to the time point.

[0123] The image processing unit receives a pixel set consisting of the first row of pixels in the image to be processed, along with its size information, scaling factor, and other parameters, transmitted from the host computer at the current time. Then, starting with each pixel in the pixel set, it takes adjacent pixels to obtain adjacent pixel groups. Each adjacent pixel group is then sequentially input into the built-in horizontal interpolation processing kernels. Based on the position of the first pixel among the two input pixels involved in the interpolation, it obtains the horizontal interpolation coefficient array and inputs it into each horizontal interpolation processing kernel. Then, based on the image scaling ratio and the row position information (which can be regarded as row number) of each pixel in the pixel set, it filters out the valid pixels to be interpolated and the invalid pixels that do not need interpolation. The horizontal interpolation processing kernels that process invalid pixels are closed accordingly, so that the remaining horizontal interpolation processing kernels can perform parallel horizontal interpolation processing on the received valid pixels based on the received horizontal interpolation coefficient arrays, obtain the horizontal interpolation processing result corresponding to the valid pixel set (including the horizontal interpolation processing section of multiple adjacent pixel groups), and cache the horizontal interpolation processing result. When the next time point arrives, perform parallel horizontal interpolation on the pixels in the second row as described above, obtain the horizontal interpolation result corresponding to the pixels in the second row, and cache it. At this time, the cached data contains the horizontal interpolation result of the pixels in the previous row of the current row, which means that vertical interpolation is required.

[0124] The vertical interpolation process can involve obtaining the starting row position of the input pixels and the number of rows involved in the interpolation. Starting from the starting row position, the vertical interpolation coefficient arrays of a corresponding number of rows are continuously retrieved according to the number of rows involved in the interpolation. Then, based on the multiple vertical interpolation coefficient arrays, parallel vertical interpolation processing is performed on the previous interpolation result and the current interpolation result. Specifically, the parallel vertical processing can be as follows: the first interpolation array from the previous interpolation result and the second interpolation array from the current interpolation result are extracted sequentially. The first interpolation array and the second interpolation array are combined to obtain multiple target interpolation arrays. The target interpolation arrays and the vertical interpolation coefficient arrays are input to the built-in vertical interpolation processing kernels respectively. Each vertical interpolation processing kernel can select the corresponding vertical interpolation coefficient array from the received vertical interpolation coefficient arrays according to the relative row number or relative column number. Then, based on the selected vertical interpolation coefficients, the received target interpolation arrays are subjected to vertical interpolation processing to obtain the interpolation result corresponding to the adjacent pixel set. Finally, the interpolation results corresponding to each set of adjacent pixels of the image to be processed are collected, and the interpolation results corresponding to each set of adjacent pixels of the image to be processed are combined and processed according to the pixel position to obtain the complete scaled image information.

[0125] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0126] Based on the same inventive concept, this application also provides an image data processing apparatus for implementing the image data processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image data processing apparatus embodiments provided below can be found in the limitations of the image data processing method described above, and will not be repeated here.

[0127] In one embodiment, such as Figure 5 As shown, an image data processing apparatus is provided, comprising: a data receiving module 510, a first parallel interpolation processing module 520, a second parallel interpolation processing module 530, and a data stitching and merging module 540, wherein:

[0128] The data receiving module 510 is used to receive image data to be processed. The image data to be processed includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction.

[0129] The first parallel interpolation processing module 520 is used to perform parallel interpolation processing on the pixel set according to the first pixel arrangement direction, obtain the current interpolation result corresponding to the pixel set, and cache it. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set.

[0130] The second parallel interpolation processing module 530 is used to perform parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction when there is a previous interpolation result in the cached data. This process yields the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0131] The data splicing and merging module 540 is used to collect the interpolation results corresponding to each set of adjacent pixels of the image to be processed, and to splice and merge the interpolation results corresponding to each set of adjacent pixels of the image to be processed to obtain the processed image data.

[0132] The aforementioned image data processing device, on the one hand, takes consecutive pixels extracted from the image to be processed as input and performs parallel interpolation processing on the input pixels. This eliminates the need for image caching and segmentation, achieving a "pipeline"-style real-time image processing approach where processing begins immediately upon input, thus resulting in higher real-time performance. On the other hand, since the input pixel set has a temporal order and is not input simultaneously, parallel interpolation processing of pixel sets input at different times along the first pixel arrangement direction can share a single interpolation processing kernel. Therefore, the number of interpolation processing kernels used can be reduced, saving hardware resources. In summary, this device achieves the effect of low hardware resource consumption and high real-time image processing performance.

[0133] In one embodiment, the first parallel interpolation processing module 520 is further configured to obtain the position information of each pixel in the pixel set; according to the position information of each pixel, obtain a plurality of first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table; and perform parallel interpolation processing on the pixel set according to the plurality of first-type interpolation coefficient arrays; wherein, the first-type interpolation coefficient table contains a mapping relationship between the position information of the pixel to be interpolated and the first-type interpolation coefficient array, and the number of interpolation coefficients in the first-type interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

[0134] In one embodiment, the first parallel interpolation processing module 520 is further configured to take each pixel in the pixel set as a starting point, take adjacent pixels to obtain adjacent pixel groups, input each adjacent pixel group sequentially to each built-in first type interpolation processing kernel, and input the first type interpolation coefficient array to the corresponding first type interpolation processing kernel according to the position information of the first pixel in the adjacent pixel group, so that the first type interpolation processing kernel performs interpolation processing on each of the input adjacent pixel groups according to the input first type interpolation coefficient array to obtain the interpolation array corresponding to each adjacent pixel group.

[0135] In one embodiment, the second parallel interpolation processing module 530 is further configured to obtain the starting row and column positions of the pixel set and the number of rows and columns participating in the interpolation processing; according to the starting row and column positions and the number of rows and columns participating in the interpolation, obtain multiple second-type interpolation coefficient arrays from a preset second-type interpolation coefficient table; and perform parallel interpolation processing on the previous interpolation result and the current interpolation result according to the multiple second-type interpolation coefficient arrays; wherein, the second-type interpolation coefficient table contains the mapping relationship between the position information of the pixels to be interpolated and the second-type interpolation coefficient array, and the number of interpolation coefficients in the second-type interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

[0136] In one embodiment, the previous interpolation result and the current interpolation result include multiple interpolation arrays;

[0137] The second parallel interpolation processing module 530 is also used to sequentially extract the first interpolation array from the previous interpolation result and the second interpolation array from the current interpolation result, combine the first interpolation array and the second interpolation array to obtain multiple target interpolation arrays; and input the target interpolation arrays and the second type interpolation coefficient arrays into the built-in second type interpolation processing cores respectively, so that each second type interpolation processing core performs interpolation processing on the input target interpolation arrays according to the input second type interpolation coefficient arrays.

[0138] like Figure 6 As shown, in one embodiment, the device further includes a pixel filtering module 550, used to filter out the valid pixels to be interpolated in the pixel set to obtain a valid pixel set; the first parallel interpolation module 520 is also used to perform parallel interpolation processing on the valid pixel set.

[0139] In one embodiment, the pixel filtering module 550 is further configured to filter out valid pixels to be interpolated and invalid pixels that do not require interpolation from the pixel set; the first parallel interpolation module 520 is further configured to disable the built-in first type interpolation processing kernel that corresponds to the invalid pixels, so that the remaining first type interpolation processing kernels can perform parallel interpolation processing on the valid pixels.

[0140] In one embodiment, the data input mode includes a point-to-line input mode, a point-to-column input mode, a multi-line point-to-point input mode, or a multi-column point-to-point input mode, and the image data to be processed includes the image scaling ratio, as well as the row position information and / or column position information of each pixel in the pixel set.

[0141] The pixel filtering module 550 is also used to filter out the valid pixels to be interpolated in the pixel set according to the image scaling ratio and the row position information of each pixel in the pixel set if the data input mode is a point-by-point input mode; if the data input mode is a point-by-point column input mode, it filters out the valid pixels to be interpolated in the pixel set according to the image scaling ratio and the column position information of each pixel in the pixel set; if the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, it filters out the valid pixels to be interpolated in the pixel set according to the image scaling ratio, row position information and column position information.

[0142] Each module in the aforementioned image data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0143] In one embodiment, such as Figure 7 As shown, this application also provides an image data processing device, which includes a data filtering module 710, a first type interpolation module 720, a second type interpolation module 730, a data cache management module 740, and a data merging module 750.

[0144] The data filtering module 710 receives the pixel set extracted from the image to be processed according to the selected data input mode at the current time point, and inputs the pixel set to the first type interpolation module 720. The first type interpolation module 720 performs parallel interpolation processing on the received pixel set according to the first pixel arrangement direction to obtain the current interpolation result corresponding to the pixel set, and sends the current interpolation result to the data cache management module 740 for caching. When the data cache management module detects that there is a previous interpolation result of the current interpolation result in the cached data, it will store the current interpolation result and the previous interpolation result in the cached data. The previous interpolation result is sent to the second type interpolation module 730. The second type interpolation module 730 performs parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The interpolation result corresponding to the adjacent pixel set is then output to the data merging module 750. The data merging module 750 collects the interpolation result corresponding to each group of adjacent pixel sets of the image to be processed, and performs splicing and merging processing on the interpolation result corresponding to each group of adjacent pixel sets of the image to be processed to obtain the processed image data and output the processed image data.

[0145] The pixel set consists of multiple consecutive pixels located in the first pixel arrangement direction. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

[0146] In practice, the data filtering module 710 is used to filter out the valid pixels to be interpolated and the invalid pixels that do not need to be interpolated from the pixel set based on the image scaling size information.

[0147] The aforementioned image data processing device, on the one hand, takes consecutive pixels extracted from the image to be processed as input and performs parallel interpolation processing on the input pixels. This eliminates the need for image caching and segmentation, achieving a "pipeline" real-time image processing approach where processing begins immediately upon input. This results in higher real-time performance for image data processing. On the other hand, since the input pixel set has a temporal order and is not input simultaneously, parallel interpolation processing of pixel sets input at different times along the first pixel arrangement direction can share a single interpolation core. Therefore, the number of interpolation cores used can be reduced, saving hardware resources. In summary, this device achieves the effect of low hardware resource consumption and high real-time image processing performance.

[0148] like Figure 8 As shown, in one embodiment, the device further includes a scaling factor calculation module 760, which is used to calculate the horizontal interpolation factor and the vertical interpolation factor, obtain the horizontal interpolation factor table and the vertical interpolation factor table, and provide functions such as querying information on interpolation factors and scaling dimensions.

[0149] In this embodiment, the horizontal interpolation coefficient table and the vertical interpolation coefficient table are pre-calculated by the scaling factor calculation module, which enables the interpolation coefficients to be quickly obtained during the interpolation process, thereby improving the processing speed of interpolation calculation.

[0150] To provide a clearer explanation of the image data processing apparatus provided in this application, the following description is provided in conjunction with... Figure 9 Explanation:

[0151] The input signals of the data filtering module 710 include Input mode, Clk, Src_w, Src_h, Dst_w, Dst_h, P1, P2, P3, and P4. Input mode is the mode of input data, which supports point-by-point row-by-row input mode, point-by-point column-by-column input mode, multi-row point-by-point input mode, and multi-column point-by-point input mode. Taking four pixels as an example, the input of point-by-row and point-by-column input modes refers to four consecutive pixels in memory located in the same pixel arrangement direction (including horizontal or vertical). The multi-row point-by-point input mode or multi-column point-by-point input mode represents synchronous point-by-point pixels in multiple consecutive rows or columns of the image located in the same pixel arrangement direction. For ease of description, the data input modes described below will exemplify point-to-point and line-by-line input modes and multi-line point-to-point input modes. P1, P2, P3, and P4 are four consecutive pixels located in horizontally arranged memory, or synchronous point-to-point pixels across multiple consecutive lines of the image; Clk is the synchronization signal; Src_w and Src_h represent the width and height of the input image, respectively; Dst_w and Dst_h represent the width and height of the scaled image, respectively; P1, P2, P3, and P4 are four consecutive horizontal pixel values ​​of the image in space; the output signals include vaild1 and vail... d2, vaild3, vaild4, K0, K1, K2, K3, P1, P2, P3, P4, Row_flag, where vaild1, vaild2, vaild3, and vaild4 indicate whether the four input pixels need interpolation calculation results, which are used to control the subsequent four horizontal interpolation modules. K0, K1, K2, K3, P1, P2, P3, and P4 represent the four pairs of pixels to be interpolated. Row_flag represents the row flag, which records the current interpolation row number and the row number of the four interpolation kernels relative to the current interpolation row (i.e., the relative row number mentioned above).

[0152] Specifically, the specific structure of the data filtering module is as follows: Figure 10As shown, the data filtering module includes four sub-modules: a counting sub-module 810, a row and column position calculation sub-module 820, an interpolation filtering sub-module 830, and a data definition sub-module 840. The counting sub-module 810 records and outputs the number of pixel inputs and also has a reset function, resetting during frame changes. The row and column position calculation sub-module 820 calculates the position based on src_w and count and outputs the row and column positions of the four currently input pixels. The calculation is processed differently depending on the data input mode: when the data input mode is row-by-row and point-by-point, the four positions increment sequentially by 4 each time; when the data input mode is 4-row point-by-point, the four positions are the same and increment by 1 each time. It also has a reset function. The data definition sub-module 840's function is also processed differently depending on the data input mode: when the data input mode is row-by-row and point-by-point, K0 = P4 from the previous time step, K1 = P1, K2 = P2, K3 = P3; when the data input mode is 4-row point-by-point, K0 = ... At a given moment, P1 and K1 equal P2, K2 equal P3, and K3 equal P4 from the previous moment. Finally, P1, P2, P3, P4, K0, K1, K2, and K3 are output together. The interpolation filtering submodule 830 processes data based on the input mode. When the input mode is row-by-row and pixel-by-pixel, it calculates and detects whether interpolation is needed for the row positions of the four currently input pixels using the scaled size information. When the input mode is 4-row and pixel-by-pixel, it directly determines whether interpolation is needed for the row and column positions. Simultaneously, it calculates the row number of the current interpolation row and the relative row numbers of the four vertical interpolation kernels based on the row and column. Finally, it controls the subsequent interpolation modules through four output signals: vaild1, vaild2, vaild3, and vaild4.

[0153] Specifically, the input signals of the scaling factor calculation module 760 include Src_w, Src_h, Dst_w, Dst_h, Row, col_pos1, col_pos2, col_pos3, and col_pos4, where Src_w and Src_h represent the width and height of the input image, Dst_w and Dst_h represent the width and height of the scaled image, Row represents the current interpolation row number, and col_pos1, col_pos2, col_pos3, and col_pos4 represent the positions of the input pixels of the four horizontal interpolation kernels, respectively; the output signals include Coef_arr1, Coef_arr2, Coef_arr3, and Co_pos4. ef_arr4, Coef_cnt1, Coef_cnt2, Coef_cnt3, Coef_cnt4, Row_coef_cnt, Row_coef_arr, where Coef_arr1, Coef_arr2, Coef_arr3, and Coef_arr4 represent the interpolation coefficient arrays required for the four horizontal interpolation kernels, Coef_cnt1, Coef_cnt2, Coef_cnt3, and Coef_cnt4 represent the effective lengths of the four horizontal interpolation coefficient arrays, and Row_coef_cnt and Row_coef_arr represent the vertical interpolation coefficient array and the effective number of vertical interpolation coefficient arrays, respectively.

[0154] Specifically, the internal structure of the scaling factor calculation module is as follows: Figure 11 As shown, the module includes an interpolation coefficient calculation submodule 762 and a horizontal interpolation coefficient caching submodule 764, specifically comprising horizontal interpolation coefficient caching submodules 1, 2, 3, and 4, vertical interpolation coefficient caching submodule 766, and an interpolation coefficient output submodule 768. The interpolation coefficient calculation submodule 762 calculates and caches the horizontal and vertical interpolation coefficients; the horizontal interpolation coefficient caching submodule 764 caches the horizontal interpolation coefficients and is used in conjunction with the four horizontal interpolation modules for reading and writing; the vertical interpolation coefficient caching submodule 766 caches the vertical interpolation coefficients. The interpolation coefficient output submodule 768 reads and outputs the horizontal interpolation coefficients based on the horizontal position of the pixel, and reads two rows of valid vertical interpolation coefficient groups each time based on the vertical position of the pixel. For the last row, the second row reads and outputs the vertical interpolation coefficient groups from the first row.

[0155] The first interpolation module 720 and the second interpolation module 730 can be either horizontal or vertical interpolation modules. If the first interpolation module is a horizontal interpolation module, then the second interpolation module is a vertical interpolation module. The essence of both the vertical and horizontal interpolation modules can be interpolation kernels. The following explanation uses the first interpolation module 720 as a horizontal interpolation module and the second interpolation module 730 as a vertical interpolation module. Specifically, the horizontal interpolation module performs horizontal interpolation on the received pixels to obtain the current interpolation result, while the vertical interpolation module performs vertical interpolation on the received current interpolation result and the previous interpolation result to obtain the interpolation result corresponding to the adjacent pixel set. Specifically, this embodiment uses four pixel inputs, so there are four horizontal interpolation modules processing them in parallel. The input signals of the entire module include vaild1, vaild2, vaild3, vaild4, col1, col2, col3, col4, P0, P1, P2, P3, P4, Coef_arr1, Coef_arr2, Coef_arr3, Coef_arr4, Coef_cnt1, Coef_cnt2, Coef_cnt3, and Coef_cnt4, where vaild1, vaild2, vaild3, and vaild4 are the input signals of the four pixels. ild4 represents the enable control signal for the four lateral interpolation modules; col1, col2, col3, and col4 represent the lateral positions of the input pixels for the four lateral interpolation modules, and the lateral positions of the pixels are used to obtain the interpolation coefficient array; P0, P1, P2, P3, and P4 represent the filtered pixels to be interpolated; the eight signals Coef_arr1, Coef_arr2, Coef_arr3, Coef_arr4, Coef_cnt1, Coef_cnt2, Coef_cnt3, and Coef_cnt4 are the signals described in the scaling factor calculation module above, and are used for lateral interpolation calculation.

[0156] The specific internal structure of a single lateral interpolation module is as follows: Figure 12As shown, it includes a horizontal interpolation coefficient splitting submodule 722, a processing kernel validity control submodule 724, a synchronization processing submodule 726, an n-pixel interpolation submodule 728, and a data merging submodule 729. The synchronization processing submodule 726 synchronizes the data, synchronizing the input `valid` signal with the output `P_out_arr` and `P_out_count` signals, and transmitting the `col` signal to the scaling factor calculation module to obtain the horizontal interpolation coefficient array. The horizontal interpolation coefficient splitting submodule 722 splits the horizontal interpolation coefficients. The processing kernel validity control submodule 724 controls the enabling of the interpolation kernel based on the number of valid interpolation coefficients. The data merging submodule 729 merges all pixels from the horizontal interpolation. The pixel interpolation submodule 728 is the pixel interpolation kernel, where `n` is the number of pixel interpolation kernels, determined by the horizontal magnification factor supported by the system of the designed device (n = horizontal magnification factor rounded up). Its structure is as follows... Figure 13 As shown, the design uses the RGB (Red, Green, Blue, three primary colors) color model. P1 and P2 represent the RGB color data of two pixels participating in the interpolation. After channel splitting, P1 is obtained as R1, G1, and B1, and P2 is obtained as R2, G2, and B2. Core_coef represents the interpolation coefficient group of the processing kernel. After splitting, two coefficients, coef1 and coef2, for linear interpolation are obtained. The three groups of linear interpolation processes are obtained as RO, GO, and BO. Core_vaild represents the interpolation kernel enable control signal, which controls the channel merging process and transmits the output to the vaild signal. It includes a channel splitting processing subunit 820, an interpolation coefficient splitting processing subunit 822, a linear interpolation kernel 824, and a channel merging processing module 826.

[0157] The cache management module 740 is used to cache the current interpolation results sent by the first interpolation module and the second interpolation module, the interpolation results corresponding to adjacent pixel sets, the horizontal interpolation coefficient table, the vertical interpolation coefficient table, and other data. It also provides data output functionality, such as outputting the interpolation results corresponding to two consecutive time points when vertical interpolation calculation is required. Specifically, its input signals include PO_arr1, PO_arr2, PO_arr3, PO_arr4, PO_cnt1, PO_cnt2, PO_cnt3, PO_cnt4, vaild_d1, vaild_d2, vaild_d3, vaild_d4, and Row_flag; where PO_arr1, PO_arr2, PO_arr3, and PO_arr4 represent the pixel arrays of the four horizontal interpolation results, i.e., the horizontal interpolation arrays, and PO_cnt1, PO_cnt2, PO_cnt3, and PO_cnt4 represent the pixel arrays of the four horizontal interpolation results. The number of valid pixels in the pixel array; valid_d1, valid_d2, valid_d3, and valid_d4 indicate the validity of the horizontal interpolation result; Row_flag represents the row flag, containing the newline position and row number information; the output signals include 2P_arr1, 2P_arr2, 2P_arr3, 2P_arr4, 2P_cnt1, 2P_cnt2, 2P_cnt3, 2P_cnt4, row1, row2, row3, row4, valid_d21, valid_d22, valid_d23, valid_d24, and Row. Wherein, 2P_arr1, 2P_arr2, 2P_arr3, and 2P_arr4 represent two rows of pixel arrays generated by the four horizontal interpolation modules; 2P_cnt1, 2P_cnt2, 2P_cnt3, and 2P_cnt4 represent the effective number of the four pixel arrays; row1, row2, row3, and row4 represent the row numbers relative to the current row (as described in the above embodiment), obtained by parsing Row_flag, and are used to select the corresponding vertical interpolation coefficients from the vertical interpolation coefficient array; vaild_d21, vaild_d22, vaild_d23, and vaild_d24 represent the enable signals and row number information of the four vertical interpolation modules, respectively; Row represents the starting row number of the current interpolation; and Num represents the number of rows participating in the interpolation, used to obtain the vertical interpolation coefficients. Each time, Num rows of consecutive effective vertical interpolation coefficient arrays are obtained starting from Row.

[0158] The internal structure of the data cache management module 740 is as follows: Figure 14As shown, it includes a row flag parsing submodule 742, four interpolation result caching submodules 744, and four interpolation array combination submodules 746. The row flag parsing submodule 742 parses the Row_flag signal to obtain four relative row numbers and the current interpolation row number Row. It then uses these four relative row numbers to set the relative row numbers of the four cache management modules. These relative row numbers are calculated by the preceding modules and are used to select the acquired vertical interpolation coefficients. The interpolation result caching submodule 744 caches two rows of interpolation results and reads the previous row of interpolation results. Its internal structure is as follows: Figure 15 As shown, `vaild` represents the valid enable control signal, `cnt` and `2P_cnt` represent the number of interpolation results, `arr` and `P_arr` represent the horizontal interpolation result array, `W_addr` is the write address, `R_addr` is the read address, and `Prew row data` represents the interpolation result data of the previous row corresponding to the current position. This includes a synchronization processing submodule 726, a cache submodule 750, a cache write submodule 752, a ping-pong processing control submodule 754 for vertical interpolation data retrieval and horizontal interpolation result writing, and a cache read submodule 756. The synchronization processing submodule 726 is used to synchronize the valid signal `vaild`; the cache submodule 750 is used to cache two rows of horizontal interpolation results in the cache area; the ping-pong processing control module for vertical interpolation data retrieval and horizontal interpolation result writing operates on one memory location when reading data and on another memory location when writing data, ultimately storing the pixel data at the current position. The horizontal interpolation result of the previous row position pixel is output; the interpolation array combination submodule 746 is used to combine two horizontal interpolation arrays from three horizontal interpolation arrays according to the data input mode to obtain the target interpolation array for output. When the data input mode is point-by-point and line-by-line input, the current row interpolation array P_arr(n) and the interpolation array Prev_arr(n) of the row pixels input at the previous time step are combined. When the data input mode is four-line point-by-point input, the previous row interpolation array P_arr(n-1) and the current row interpolation array P_arr(n) are combined. Since the horizontal interpolation result may be cached in different inter caches, and the first vertical interpolation kernel needs to combine the interpolation result obtained at the previous time step for vertical interpolation processing, in this embodiment, the fourth inter cache kernel can output the interpolation result Prev_arr at the previous time step, and finally output 2P_arr.

[0159] The vertical interpolation module 730 is used to perform vertical interpolation calculations on the interpolation array obtained from the previous horizontal interpolation. Its input signals include 2P_arr1, 2P_arr2, 2P_arr3, 2P_arr4, 2P_cnt1, 2P_cnt2, 2P_cnt3, 2P_cnt4, vaild_d21, vaild_d22, vaild_d23, and vaild_d24. Their meanings have been given in the introduction of the cache management module above and will not be repeated here. Additionally, the two inputs, Row_coef_arr and Row_coef_cnt, represent the vertical interpolation coefficient arrays and the number of arrays for the current and next interpolation rows, respectively. row1, row2, row3, and row4 represent the interpolation ordinate positions of the four vertical interpolation modules. The output signals include: 2P_out_arr1, 2P_out_arr2, 2P_out_arr3, 2P_out_arr4, 2P_out_cnt1, 2P_out_cnt2, 2P_out_cnt3, 2P_out_cnt4, vaild_d31, vaild_d32, and vaild_d. 33, vaild_d34, where 2P_out_arr1, 2P_out_arr2, 2P_out_arr3, and 2P_out_arr4 represent the interpolation result arrays corresponding to the adjacent pixel sets output by the four vertical interpolation modules, 2P_out_cnt1, 2P_out_cnt2, 2P_out_cnt3, and 2P_out_cnt4 represent the number of valid pixels in the interpolation result arrays corresponding to the four adjacent pixel sets, and vaild_d31, vaild_d32, vaild_d33, and vaild_d34 represent the valid signals in the interpolation result arrays corresponding to the four adjacent pixel sets.

[0160] The internal structure of the vertical interpolation module is as follows: Figure 16As shown, the system includes an interpolation kernel control submodule 732, which controls the enabling of m linear interpolation kernels 824 (such as vertical interpolation kernels) based on the input start kernel count P_out_count. Only the first P_out_count kernels are active among the m linear interpolation kernels. A vertical interpolation coefficient quantity parameter splitting submodule 734 is used to split the vertical interpolation coefficient quantity parameter between two rows. A vertical interpolation coefficient array splitting submodule 736 is used to split the vertical interpolation coefficient array between two rows. The split arrays and the interpolation coefficient quantity are then further processed according to... The input row signal is selectively distributed to m linear interpolation kernels. The selection method is that the first row is selected when row=0, and the second row is selected when row=1. The interpolation array segmentation submodule 738 is used to segment the interpolation array into m pairs of interpolated pixels. The vertical interpolation module has the same function as described above. The data merging submodule 729 is used to merge the effective pixels output by all effective linear interpolation kernels for output. The synchronization processing submodule 726 is used for synchronization processing, matching the valid enable signal with the output P_out_arr and P_out_count signals.

[0161] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the image data processing method described above.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the image data processing method described above.

[0163] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the image data processing method described above.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0165] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image data processing method, characterized in that, The method includes: Receive image data to be processed, the image data to be processed includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction. The pixels of the image to be processed are continuously input in the form of a data stream according to the selected data input mode. The data input mode includes point-by-point line-by-line input mode, point-by-point column-by-column input mode, multi-line point-by-point input mode, or multi-column point-by-point input mode. According to the first pixel arrangement direction, the position information of each pixel in the pixel set is obtained. Based on the position information of each pixel, multiple first-type interpolation coefficient arrays are obtained from a preset first-type interpolation coefficient table. Parallel interpolation processing is performed on the pixel set according to the multiple first-type interpolation coefficient arrays to obtain the current interpolation result corresponding to the pixel set. The current interpolation result is cached in the interpolation result cache area. The interpolation result cache area caches at least two rows or two columns of interpolation results each time. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set. When there is a previous interpolation result of the current interpolation result, parallel interpolation processing is performed on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction. The interpolation results corresponding to each set of adjacent pixels of the image to be processed are collected, and the interpolation results corresponding to each set of adjacent pixels of the image to be processed are combined and processed to obtain the processed image data.

2. The image data processing method according to claim 1, characterized in that, The first type of interpolation coefficient table contains the mapping relationship between the position information of the pixel to be interpolated and the first type of interpolation coefficient array. The number of interpolation coefficients in the first type of interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

3. The image data processing method according to claim 2, characterized in that, The parallel interpolation processing of the pixel set based on multiple first-type interpolation coefficient arrays includes: Starting with each pixel in the pixel set, adjacent pixels are taken to obtain adjacent pixel groups. Each adjacent pixel group is sequentially input into each built-in first-type interpolation processing kernel. Based on the position information of the first pixel in the adjacent pixel group, the first-type interpolation coefficient array is input into the corresponding first-type interpolation processing kernel, so that the first-type interpolation processing kernel performs interpolation processing on each of the input adjacent pixel groups according to the input first-type interpolation coefficient array to obtain the interpolation array corresponding to each adjacent pixel group.

4. The image data processing method according to claim 1, characterized in that, The parallel interpolation process for the previous interpolation result and the current interpolation result includes: Obtain the starting row and column positions of the pixel set and the number of rows and columns involved in the interpolation process; Based on the starting row and column positions and the number of rows and columns involved in the interpolation, multiple arrays of second-type interpolation coefficients are obtained from a preset second-type interpolation coefficient table. Based on multiple second-type interpolation coefficient arrays, parallel interpolation processing is performed on the previous interpolation result and the current interpolation result; The second type of interpolation coefficient table contains the mapping relationship between the position information of the pixel to be interpolated and the second type of interpolation coefficient array. The number of interpolation coefficients in the second type of interpolation coefficient array is determined based on the position information of each pixel in the pixel set.

5. The image data processing method according to claim 4, characterized in that, The previous interpolation result and the current interpolation result include multiple interpolation arrays; The parallel interpolation processing of the previous interpolation result and the current interpolation result based on multiple second-type interpolation coefficient arrays includes: The first interpolation array from the previous interpolation result and the second interpolation array from the current interpolation result are extracted sequentially, and the first interpolation array and the second interpolation array are combined to obtain multiple target interpolation arrays; The target interpolation array and the second type interpolation coefficient array are respectively input to the built-in second type interpolation processing cores, so that each second type interpolation processing core performs interpolation processing on the input target interpolation array according to the input second type interpolation coefficient array.

6. The image data processing method according to any one of claims 1 to 5, characterized in that, Before performing parallel interpolation on the pixel set, the method further includes: Filter out the valid pixels to be interpolated from the pixel set to obtain the valid pixel set; The parallel interpolation process for the pixel set includes: Parallel interpolation processing is performed on the effective pixel set.

7. The image data processing method according to any one of claims 1 to 5, characterized in that, Before performing parallel interpolation on the pixel set, the method further includes: Filter out the valid pixels to be interpolated and the invalid pixels that do not need interpolation from the pixel set; The parallel interpolation process for the pixel set includes: The built-in first-type interpolation processing kernel that corresponds to the invalid pixel is deactivated, so that the remaining first-type interpolation processing kernels can perform parallel interpolation processing on the valid pixel.

8. The image data processing method according to any one of claims 1 to 5, characterized in that, If the data input mode is a point-by-point input mode, then the pixel set consists of multiple consecutive pixels located in the horizontal pixel arrangement direction; If the data input mode is a point-by-point column-by-column input mode, then the pixel set consists of multiple consecutive pixels located in the vertical pixel arrangement direction; If the data input mode is a multi-line point-by-point input mode, then the pixel set consists of multiple pixels with the same column number and consecutive row numbers; If the data input mode is a multi-column point-by-point input mode, then the pixel set consists of multiple pixels with the same row number and consecutive column numbers.

9. The image data processing method according to claim 6, characterized in that, The image data to be processed includes the image scaling ratio, as well as the row position information and / or column position information of each pixel in the pixel set; The valid pixels to be interpolated in the pixel set include: If the data input mode is a point-by-point and line-by-line input mode, the effective pixels to be interpolated in the pixel set are selected according to the image scaling ratio and the line position information of each pixel in the pixel set. If the data input mode is a point-by-point column-by-column input mode, the effective pixels to be interpolated in the pixel set are selected according to the image scaling ratio and the column position information of each pixel in the pixel set. If the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, then the effective pixels to be interpolated in the pixel set are selected according to the image scaling ratio, the row position information and the column position information.

10. An image data processing apparatus, characterized in that, The device includes: The data receiving module is used to receive image data to be processed. The image data to be processed includes a set of pixels extracted from the image to be processed according to the selected data input mode at the current time point. The set of pixels consists of multiple consecutive pixels located in the first pixel arrangement direction. The pixels of the image to be processed are continuously input in the form of a data stream according to the selected data input mode. The data input mode includes a point-by-point line-by-line input mode, a point-by-point column-by-column input mode, a multi-line point-by-point input mode, or a multi-column point-by-point input mode. The first parallel interpolation processing module is used to obtain the position information of each pixel in the pixel set according to the first pixel arrangement direction, obtain multiple first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table according to the position information of each pixel, perform parallel interpolation processing on the pixel set according to the multiple first-type interpolation coefficient arrays, obtain the current interpolation result corresponding to the pixel set, and cache the current interpolation result in the interpolation result cache area. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set. The interpolation result cache area caches at least two rows or two columns of interpolation results each time. The second parallel interpolation processing module is used to perform parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction when the cached data contains the previous interpolation result of the current interpolation result, so as to obtain the interpolation result corresponding to the adjacent pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction. The data merging module is used to collect the interpolation results corresponding to each set of adjacent pixels of the image to be processed, and to combine and process the interpolation results corresponding to each set of adjacent pixels of the image to be processed to obtain the processed image data.

11. An image data processing device, characterized in that, The device includes a data filtering module, a first type of interpolation module, a second type of interpolation module, a data cache management module, and a data merging module; The data filtering module receives a pixel set extracted from the image to be processed according to the selected data input mode at the current time point, and inputs the pixel set to the first type interpolation module. The first type interpolation module obtains the position information of each pixel in the pixel set according to the first pixel arrangement direction, and obtains multiple first type interpolation coefficient arrays from a preset first type interpolation coefficient table according to the position information of each pixel. It then performs parallel interpolation processing on the pixel set based on the multiple first type interpolation coefficient arrays to obtain the current interpolation result corresponding to the pixel set, and sends the current interpolation result to the data cache management module for caching. When the data cache management module detects that there is a problem in the cached data... When a previous interpolation result exists for the current interpolation result, the current interpolation result and the previous interpolation result are sent to the second type interpolation module. The second type interpolation module performs parallel interpolation processing on the previous interpolation result and the current interpolation result according to the second pixel arrangement direction to obtain the interpolation result corresponding to the adjacent pixel set. The interpolation result corresponding to the adjacent pixel set is then output to the data merging module. The data merging module collects the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and performs concatenation and merging processing on the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain the processed image data, and outputs the processed image data. The pixel set consists of multiple consecutive pixels located in the first pixel arrangement direction. The number of elements in the current interpolation result is linearly related to the number of elements in the pixel set. The previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point. The first pixel arrangement direction is perpendicular to the second pixel arrangement direction. The pixels of the image to be processed are continuously input in the form of a data stream according to the selected data input mode. The data cache management module caches at least two rows or two columns of interpolation results each time. The data input mode includes point-by-point line-by-line input mode, point-by-point column-by-column input mode, multi-row point-by-point input mode, or multi-column point-by-point input mode.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images