Image caching method and circuit and computer readable storage medium

Abstract

The invention provides an image caching method and circuit and a computer readable storage medium, and relates to the technical field of digital image processing. The method comprises the steps that n line caches are provided, the number of pixels capable of being stored in each line cache is m, and the pixels of each line in an original image of a current picture are stored in the corresponding line cache; according to the row index value i (0 < = i < n) and the column index value j (0 < = j < m) of the current processing pixel, sequentially outputting the neighborhoods of the current processing pixel according to the neighborhood size of p * q; wherein the size of the original image is C * R, C is the total column number of the original image, and R is the total row number of the original image; wherein the size of the neighborhood on which the method depends is p * q, p is the column number of the neighborhood, and q is the row number of the neighborhood. The circular processing of the data is realized by circularly utilizing a small amount of line cache, so that the delay of each level of image processing algorithm is controlled within a plurality of lines, the data processing flow length is reduced, and the processing delay is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of digital image processing, and in particular to an image caching method, circuit, and computer-readable storage medium. Background Art

[0002] With the development of digital image processing, today's video stream processing is mainly based on a CPU plus memory architecture (which can also include GPU and video memory). Most video processing tasks, such as image filtering, color gamut conversion, image scaling, resampling, etc., require storing a complete image frame in memory. This video stream processing that requires frame buffering offers great flexibility and facilitates the implementation of various algorithms, but it also increases video processing latency because caching a frame is time-consuming. In addition, many video processing processes have long delays and often involve more than one or two levels of frame buffering, which significantly increases processing latency. Summary of the Invention

[0003] In view of this, an object of an embodiment of the present application is to provide an image caching method to improve the problem of long processing delay in the prior art.

[0004] Provide n row buffers, each of which can store m pixels, and store the pixels of each row in the original image of the current screen into the corresponding row buffer; output the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q according to the row index value i (0<i≤n) and column index value j (0<j≤m) of the currently processed pixel; wherein the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; wherein the neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood, and q is the number of rows of the neighborhood.

[0005] In the above implementation, n row buffers are used to store pixel input for each row of the original image of the current frame. Each row of pixels in the original image of the current frame is sequentially input and stored in the corresponding row buffer. The row buffer's index value, i, and column index value, j, indicate the location of each pixel within the row buffer. Note that the buffer area can be smaller than, equal to, or larger than the original image size, depending on the specific loop situation. When the row index i and column index j of the currently processed pixel in the first frame or the first start of storage in the buffer area meet the target conditions, the neighborhood of the currently processed pixel is sequentially output in a p×q neighborhood size. The p×q neighborhood corresponding to the currently processed pixel is then used for subsequent computations. Since only the required n rows of pixel data need to be retained, computations can be performed on the image data without having to store the entire image data in memory before recomputing. For large-scale image processing scenarios, this significantly reduces system memory usage, saving hardware costs and power consumption.

[0006] Optionally, the method outputs the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q based on the row index value i and column index value j of the currently processed pixel, including: judging whether the neighborhood of the currently processed pixel contains irrelevant pixels based on the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q; if it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of size p×q corresponding to the currently processed pixel; wherein the irrelevant pixels are parts of the neighborhood of the currently processed pixel that exceed the current picture or are not directly or indirectly adjacent to the current pixel; wherein replacing the pixel values ​​of the irrelevant pixels includes: the corresponding pixel value of the irrelevant pixels is one of the upper limit or lower limit of the allowable value range, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, and a preset specific value.

[0007] In the above implementation, if the currently processed pixel is located at the edge or corner of the image, its neighborhood may exceed the image range. Because the values ​​of these pixels outside the image range (irrelevant pixels) do not exist in the image, if these irrelevant pixels are not processed, the neighborhood of the currently processed pixel may be incorrectly output. By replacing the values ​​of irrelevant pixels, it is possible to ensure that the pixel values ​​involved in the calculation are valid, thus avoiding calculation errors.

[0008] Optionally, the method outputs the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q based on the row index value i and column index value j of the currently processed pixel, and further includes: calculating the number of input pixels based on the row and column index values ​​of the currently processed pixel; and calculating the position of the neighborhood corresponding to the currently processed pixel after the output based on the number of input pixels; wherein the number of input pixels is Q, and Q=i·m+j+1.

[0009] In the above implementation, images are typically stored in computers as one-dimensional arrays. Using the formula Q = i·m+j+1, a two-dimensional pixel position (i, j) is mapped to a one-dimensional pixel sequence, essentially converting a two-dimensional coordinate index into a one-dimensional pixel count. By calculating the number of input pixels based on the row and column indices of the currently processed pixel, the position of the output neighborhood can be accurately restored, facilitating the proper management and manipulation of pixel data during image processing.

[0010] Optionally, the calculating of the position of the neighborhood after the current processing pixel corresponds to the output according to the number of the input pixels includes: when the current processing pixel position is located in the center of the neighborhood and the number of the input pixels is In the case of , it is determined that the output pixel neighborhood of the r-1th cycle is calculated; among them, represents rounding down, and r is the number of cycles.

[0011] In the above implementation, if the cache area is full, the previous data must be overwritten when new data is stored, and r represents the number of times the cache area is overwritten. Note that the cache area can be smaller than, equal to, or larger than the original image size, and the specific loop situation requires specific analysis. Calculating the output pixel neighborhood for the r-1th loop when the conditions are met fully utilizes data relationships and information within or between frames, ensuring that output is not paused within or between frames, helping to improve the throughput of results in multi-frame image processing tasks. Based on the number and position of input pixels, the output pixel relationship between different frames or different cache areas is flexibly calculated, making it suitable for multi-frame image processing tasks such as video processing and image sequence analysis, and expanding the scope of the method. Calculating the output position based on the number and position of input pixels enables more precise control and management of data flow and output results during image processing, ensuring the accuracy and consistency of processing results.

[0012] Optionally, the step of calculating the position of the neighborhood corresponding to the output of the current processing pixel according to the number of the input pixels further comprises: when the current processing pixel position is located at the center of the neighborhood and the input pixel In the case of , it is determined that the output pixel neighborhood of the rth cycle is calculated; among them, represents rounding down, and r is the number of cycles.

[0013] In the above implementation process, when the cache area is full, the previous data needs to be overwritten when new data is stored, and r here represents the number of times the cache area is cyclically overwritten. When the conditions are met, the output pixel neighborhood of the rth cycle is calculated, which makes full use of the data relationship and information within or between frames, and helps to improve the accuracy and reliability of the results in multi-frame image processing tasks. According to the number and position of the input pixels, the output pixel relationship between different frames or different cache areas is flexibly calculated, which is suitable for multi-frame image processing tasks such as video processing, image sequence analysis, etc., and enhances the scope of application of the method. The calculation of the output position based on the number and position of the input pixels can more accurately control and manage the data flow and result output during the image processing process, ensuring the accuracy and consistency of the processing results.

[0014] Optionally, the calculating the position of the neighborhood after the output corresponding to the currently processed pixel based on the number of input pixels also includes: performing operations based on the row and column index values ​​of the currently processed pixel to obtain the coordinates of the output neighborhood corresponding to the currently processed pixel.

[0015] In the above implementation process, by rationally utilizing the number of input pixels and row and column index values, the position of the central pixel of the output neighborhood corresponding to the current processing pixel can be calculated efficiently, flexibly and accurately, providing a basis for image processing tasks.

[0016] Optionally, storing pixels of each row in the original image of the current screen into the corresponding row cache includes: when the row cache area is filled and the total number of rows of the current screen is R (R≥n), when the pixel is input into the i-th row cache, the row cache of the i-n+Rth row of the filled row cache area is overwritten.

[0017] In the implementation described above, the core of this overwriting strategy lies in dynamically managing cache space to ensure efficient data processing within limited storage resources. Each frame of pixel data typically contains a large number of rows, and cache space is limited. Overwriting old data prevents the cache from being occupied by useless data, thereby improving storage efficiency. Overwriting old data reduces frequent memory allocation and deallocation operations, thereby reducing processing latency and improving overall performance. The overwriting strategy dynamically adapts to changes in the number of rows between frames, ensuring that the cache is always in an optimal state when processing multiple frames of data.

[0018] Optionally, storing the pixels of each row in the original image of the current screen into the corresponding row cache further includes: when the total number of input rows is the (rn+i)th row of pixels, covering the ((r-1)n+i)th row of the total number of rows; wherein r is the number of loops; when the (rn+i)th row of pixels is input and the row cache is covered, recording the covering order of the row cache and the row number corresponding to the pixel input in the covered row cache in the original image; and reordering the row outputs in the row cache according to the row index value of the currently processed pixel.

[0019] In this implementation, by recording the order in which row buffers are overwritten and the row numbers of the pixels in the overwritten row buffer in the original image, cache data can be managed more efficiently. When overwriting a row buffer, complex operations on the entire cache are eliminated; simply recording the overwrite information improves cache management efficiency. Output is reordered based on the index value of the currently processed pixel, ensuring the correctness and consistency of the output data and avoiding data confusion and errors caused by cache overwrites.

[0020] An embodiment of the present application further provides a circuit, which includes a programmable or non-programmable execution unit, and the execution unit executes any of the above-mentioned image cache processing methods based on automatic circuit action, predetermined functions during design, or an executable program therein.

[0021] During the above implementation process, the circuit can flexibly implement the image caching method through programming, is not restricted by fixed hardware logic, can adapt to image processing requirements in different scenarios, and is universal and scalable.

[0022] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a processor, the steps in any one of the above methods are executed.

[0023] In the above implementation process, computer program instructions in a computer-readable storage medium are used in the image caching method to ensure the continuity and stability of image processing in the time dimension and improve the overall image quality and visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1A first flow chart of the image caching method provided in an embodiment of the present application;

[0026] Figure 2 A second flow chart of the image caching method provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of irrelevant pixels provided in an embodiment of the present application;

[0028] Figure 4 A third flow chart of the image caching method provided in an embodiment of the present application;

[0029] Figure 5 A fourth flow chart of the image caching method provided in an embodiment of the present application;

[0030] Figure 6 A fifth flow chart of the image caching method provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of a circuit provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of a circuit structure provided in an embodiment of the present application;

[0033] Figure 9 A schematic diagram of the processing process of a 3×3 neighborhood at the row scale provided in an embodiment of the present application.

[0034] Icons: 11-row cache unit; 12-data input module; 13-output module; 14-control module; 15-boundary processing module; 16-row re-indexing module. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of them. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0036] The present application provides an image caching method applicable to image or video processing, aiming to optimize the storage and access efficiency of image data while reducing processing latency and improving overall performance. Through a rational caching strategy, this method effectively manages the storage, access, and update of image data, ensuring a fast system response and a smooth user experience when processing large amounts of image or video data.

[0037] Specifically, it involves two parts: the first part is to output the pixel neighborhood when the output conditions are met; the second part is to reorder the output pixel neighborhood. The corresponding embodiment of the present application will specifically describe the pixel neighborhood output and reordering process.

[0038] First, see Figure 1 , Figure 1 This is a first flow chart of the image caching method provided in an embodiment of the present application.

[0039] The method includes: providing n row caches, each row cache being able to store m pixels, and storing the pixels of each row in the original image of the current screen into the corresponding row cache; wherein the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, outputting the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q; wherein the neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood, and q is the number of rows of the neighborhood.

[0040] In the above implementation, n row buffers are prepared to store pixel data for the original image of the current screen or frame (when the original image size matches the cache area size). Each row buffer corresponds to a row of pixels in the image, ensuring rapid access and storage of pixel information during processing. The pixels for each row of the current screen or frame in the original image, which is C columns and R rows in size, are stored in a specific order in the corresponding row buffer. There are n row buffers in total, each of which can accommodate m pixels. Therefore, it is easy to understand that all row buffers form an m-column, n-row area for storing pixels in the original image data. Given the aforementioned m-column, n-row area, to determine the cache location of a stored pixel, each row buffer is numbered, i.e., row index i, and each column buffer is numbered, i.e., column index j. The value range of i is 0 ≤ i < n, and the value range of j is 0 ≤ j < m. n is the number of row buffers, and m is the number of pixels that can be stored in each row buffer, i.e., the number of columns that can be stored. If both row index value i and column index value j meet the target conditions, a p×q pixel neighborhood is output in sequence with a pixel neighborhood size of p×q, where p is the number of columns in the neighborhood size and q is the number of rows in the neighborhood size. Finally, the output p×q pixel neighborhood is processed using a preset pixel neighborhood processing algorithm.

[0041] Optionally, the target conditions are not specifically limited, and it is feasible to output a suitable pixel neighborhood according to the requirements for processing to obtain the corresponding effect. At the same time, the preset pixel neighborhood processing algorithm can be a mean filter algorithm, a Gaussian filter algorithm, a median filter algorithm, a Laplace sharpening algorithm, and other algorithms that can be selected for image processing according to the requirements.

[0042] Optionally, the cache area here can be smaller than the size of the original image, or equal to or larger than the size of the original image. The number of rows in the cache area can be equal to the number of rows in the original image, but the number of columns in the cache area can be smaller than, equal to, or larger than the number of columns in the original image. Similarly, the number of columns in the cache area can be equal to the number of columns in the original image, but the number of rows in the cache area can be smaller than, equal to, or larger than the number of rows in the original image. The number of rows and columns in the cache area can both be smaller than and equal to the number of rows and columns in the original image. The number of rows and columns in the cache area can also be equal to and equal to the corresponding number of rows and columns in the original image. Alternatively, the number of rows and columns in the cache area can both be larger than and equal to the corresponding number of rows and columns in the original image.

[0043] Further, see Figure 2 , Figure 2 This is a second flow chart of the image caching method provided in an embodiment of the present application.

[0044] The method also includes: outputting the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q according to the row index value i and column index value j of the currently processed pixel, including: judging whether the neighborhood of the currently processed pixel contains irrelevant pixels according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q; if it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the pixel neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are the parts of the neighborhood of the currently processed pixel that exceed the current frame or current picture or are not directly or indirectly adjacent to the current pixel.

[0045] In the above implementation process, when processing the neighborhood of the edge pixels of the image, some parts of the neighborhood may exceed the image boundary, resulting in the appearance of irrelevant pixels. According to the row and column index values ​​of the currently processed pixel, it is determined whether it is at the boundary position of the image or the neighborhood exceeds the image range, thereby judging whether the current pixel contains irrelevant pixels. The irrelevant pixel is the part of the neighborhood of the currently processed pixel that exceeds the current frame or is not directly or indirectly adjacent to the current pixel. Here, it can be interpreted as the part of the currently processed pixel in the neighborhood that exceeds the current frame, and this part includes two cases, one is in the current frame but not in the neighborhood calculation area, and the other is neither in the current frame nor in the neighborhood calculation area. In the case of irrelevant pixels, the pixel value of the irrelevant pixel is replaced.

[0046] Optionally, the pixel values ​​assigned to irrelevant pixels include the upper or lower limit of the corresponding pixel's possible value range, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above processing.

[0047] See also Figure 3 , Figure 3 A schematic diagram of irrelevant pixels provided in an embodiment of the present application.

[0048] Figure 3 There are 3 row buffers in , each of which can store 4 data. Here, the original image size is also 3 rows and 4 columns. At this time, taking the output neighborhood size of 3×3 as an example, the light gray part of the background is the pixels of the previous frame, and the dark gray part of the background is the pixels of the current frame.

[0049] When the first pixel of the first row of pixels of the current frame is being input to the first row, the currently processed pixel is the last pixel of the second row of the previous frame. At this time, the gray digital part pixels include one current frame pixel and two previous frame pixels, that is, irrelevant pixels, and do not participate in the calculation, and need to be replaced with the upper or lower limit value of the allowable value range, the value of the currently processed pixel, the pixel value at the symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above processing.

[0050] When the pixel input reaches the second pixel in the first row of the current frame and the first pixel in the third row of the previous frame, the current processing pixel is the first pixel in the third row of the previous frame. At this time, the gray digital pixels include two current frame pixels and three previous frame pixels. They are not included in the calculation and need to be replaced. The replacement rules are the same. For example, for an 8-bit grayscale image, the pixel value range is 0 to 255. The irrelevant pixels can be assigned 0 (lower limit) or 255 (upper limit).

[0051] Based on the above premise, the present application provides the following embodiments to describe the pixel neighborhood output and reordering process.

[0052] Example 1

[0053] Please combine Figure 1 、 Figure 2 and Figure 3 , see Figure 4 , Figure 4 This is the third flow chart of the image caching method provided in an embodiment of the present application.

[0054] In the first embodiment of the present invention, n row caches are provided, and the number of pixels that can be stored in each row cache is m. The pixels of each row in the original image of the current frame are stored in the corresponding row cache; wherein the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q; wherein the neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood, and q is the number of rows of the neighborhood. According to the row index value i and the column index value j of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q, including: according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; when it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are portions of the neighborhood of the currently processed pixel that exceed the current frame or are not directly or indirectly adjacent to the current pixel; wherein the pixel values ​​of the irrelevant pixels are replaced, including: the upper limit or lower limit of the value range of the corresponding pixel value of the irrelevant pixel, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset characteristic value, and other specific values ​​that can be used for the above-mentioned processing.

[0055] It should be noted that in this embodiment, the values ​​of n and m can be smaller than, equal to, or greater than C and R, respectively. In the output pixel neighborhood, the value range of p is 0 < p ≤ n, and the value range of q is 0 < q ≤ m. That is, the size of the output neighborhood is smaller than the n-row, m-column area formed by the row buffer. The size relationship of other parameters is not further limited. In other words, the cache area here can be smaller than the size of the original image, or equal to or greater than the size of the original image. The specific loop situation requires specific analysis.

[0056] Based on the row and column index values ​​of the currently processed pixel, calculate the number of input pixels, where Q = i·m+j+1. Based on the number of input pixels, calculate the position of the neighborhood of the currently processed pixel after the output.

[0057] In the above implementation process, when an image is stored in a row cache, it has a row cache index i and a column cache index j, and in computer logic, the index value usually starts at 0 (that is, if the index value starts at 1, the same idea can be used and the corresponding formula can be used for calculation). At the same time, knowing that each row of the row cache can cache m data, the formula Q = i·m+j+1 can be used to map the two-dimensional pixel position (i, j) corresponding to the current input pixel to a one-dimensional pixel sequence, thereby obtaining the number of current input pixels. By calculating the number of input pixels based on the row and column index values ​​of the currently processed pixel, the position of the p×q pixel neighborhood can be accurately output, which helps to correctly manage and operate pixel data when processing images.

[0058] It can be understood that there must be a corresponding relationship between the number of input pixels and the position of the pixel neighborhood of the output p×q corresponding to the current processing pixel. Under this corresponding relationship, the position of the pixel neighborhood of the output p×q corresponding to the current processing pixel can be obtained according to the number of input pixels.

[0059] Example 2

[0060] Please combine Figure 1 、 Figure 2 and Figure 3 , see Figure 5 , Figure 5 This is the fourth flow chart of the image caching method provided in an embodiment of the present application.

[0061] In the second embodiment, n row caches are provided, and the number of pixels that can be stored in each row cache is m. The pixels of each row in the original image of the current frame are stored in the corresponding row cache; wherein the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q; wherein the neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood, and q is the number of rows of the neighborhood. According to the row index value i and the column index value j of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q, including: according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; when it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are portions of the neighborhood of the currently processed pixel that exceed the current frame or are not directly or indirectly adjacent to the current pixel; wherein the pixel values ​​of the irrelevant pixels are replaced, including: the upper limit or lower limit of the value range of the corresponding pixel value of the irrelevant pixel, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above-mentioned processing.

[0062] When the m×n area is smaller than the C×R area and m≥C (one row of the cache area can hold one row of the original image), one cache area cannot store and process a frame of image, then a row cache overwrite mechanism is introduced for the same frame of image (similarly, when the m×n area is smaller than the C×R area and n≥R (one column of the cache area can hold one column of the original image), a column cache overwrite mechanism can be introduced for the same frame of image; and when the m×n area is smaller than the C×R area, m<C, n<R, both rows and columns can have overwrite mechanisms, the principles are similar, so they are not repeated here).

[0063] It can be understood that the overwriting is not limited to the same frame image, but can also be the overwriting between different consecutive frames. Here, each row and / or column can be stored in the cache area in sequence according to the frame number. When the row and / or column fills the cache area, it starts to be overwritten in sequence, that is, the first one stored will be overwritten first.

[0064] At this time, in the process of reordering the output pixel neighborhood, the data reordering process can be assisted while reusing the row buffer.

[0065] One case is, S1: when the row buffer area is filled and the total number of rows of the current picture is R (R≥n), when the pixel is input into the i-th row buffer, the row buffer of the i-n+Rth row of the filled row buffer area is overwritten.

[0066] In the above implementation process, when a pixel is input into the i-th row cache, it is determined whether the total number of rows R of the current picture is greater than or equal to the cache size n. If so, an overwrite operation exists for the current picture. The row index to be overwritten is calculated by the formula i-n+R. This ensures that the overwrite operation is based on the row relationship between the current input pixel and the pixel stored in the cache area in the previous processing process. The i-th row data of the current processing process is written into the cache, and the i-n+R-th row data of the pixel stored in the cache area in the previous processing process is overwritten. The row cache management adopts a row round-robin mechanism, which has a round-robin mechanism at the edge of the frame, between frames, and within the frame. When the number of rows of the original image exceeds the cache capacity n (R≥n), the row index is automatically converted through the index calculation of i-n+R. This mapping relationship ensures the dynamic overwrite update of historical data while maintaining the fixed capacity constraint of the n-row cache.

[0067] Another case is S2: when the (rn+i)th row of pixels is input, the ((r-1)n+i)th row in the current processing process is covered, where r is the number of loop coverages and r≥1.

[0068] In the above implementation process, when the (rn+i)th row of pixels is input, it is first determined whether the total number of rows R of the current frame is greater than or equal to the cache size n. If so, an overwrite operation occurs. The row index to be overwritten in the current frame is calculated using the formula ((r-1)n+i). The (rn+i)th row of data of the current frame is written to the cache, overwriting the ((r-1)n+i)th row of data of the current frame. In the case of overwriting the row cache, the overwriting order of the row cache and the row number corresponding to the pixel input in the overwritten row cache in the original image are recorded. The row outputs in the row cache are reordered according to the row index value of the currently processed pixel. For example, assuming the total number of rows R of the current frame is 100, the cache size n is 10, the row index currently input is i=5, and the loop count r=2: the input row index is (2×10+5)=25. The row index to be overwritten is ((2-1)×10+5)=15. The 25th row of data of the current frame will overwrite the 15th row of data of the current frame.

[0069] In the above implementation process, a coverage trajectory tracking mechanism is also provided. By recording the mapping relationship of the original image row number each time. It ensures that after the row cache is covered, the pixel arrangement of the output image can reflect the row number sequence of the original image, so as to present the correct row order in the output image, avoiding image content dislocation or data distortion caused by row number confusion. In the output process, since the input of data enters the row cache one by one in sequence, the positional relationship of the same row of data will not change during output. Therefore, when the row cache needs to be covered, only the row order may be disordered. At this time, it is only necessary to record the coverage order of the row cache and the row number corresponding to the pixel input in the covered row cache in the original image. This allows the output to maintain the spatial structure of the original image while supporting seamless connection of cache rows across multiple frame cycles, meeting the strict row synchronization timing requirements in real-time video stream processing.

[0070] It can be seen that when the m×n area is smaller than the C×R area, the rows and columns of the original image can be subjected to the same cyclic coverage mechanism when inputting the cache area to save storage space.

[0071] It can be seen that when the m×n area is equal to the C×R area, if the size of a frame image is C×R, the image input to the line buffer processing is the size of a frame of the original image, and a frame of the original image can be quickly processed during output.

[0072] Example 3

[0073] Please combine Figure 1 and Figure 2 , see Figure 6 , Figure 6 This is the fifth flow chart of the image caching method provided in an embodiment of the present application.

[0074] In the third embodiment, n row caches are provided, and the number of pixels that can be stored in each row cache is m. The pixels of each row in the original image of the current screen are stored in the corresponding row cache; wherein the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q; wherein the pixel neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood size, and q is the number of rows of the neighborhood size. According to the row index value i and the column index value j of the currently processed pixel, the neighborhood of the current pixel is output in sequence with a neighborhood size of p×q, including: according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; when it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are portions of the neighborhood of the currently processed pixel that exceed the current frame or are not directly or indirectly adjacent to the current pixel; wherein the pixel values ​​of the irrelevant pixels are replaced, including: the upper limit or lower limit of the value range of the corresponding pixel value of the irrelevant pixel, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above-mentioned processing.

[0075] Let the m×n region be smaller than the C×R region, meaning that an m×n region is insufficient to form a single frame of pixels. It's easy to see that, when i ≤ n and R ≥ n are satisfied, a single cache region cannot store and process a single frame of image data. Therefore, a row cache overwrite mechanism is introduced for the same frame of image data, reordering the output pixel neighborhood. For the input pixel row (rn+i), the ((r-1)n+i)th row of the current processing is overwritten, where r is the number of loops.

[0076] When the current processing pixel position is in the center of the pixel neighborhood and the number of input pixels is In the case of , it is determined that the calculation is for the output pixel neighborhood of the r-1th cycle; when the current processing pixel position is located in the center of the pixel neighborhood and the number of input pixels is In the case of , it is determined that the output pixel neighborhood of the rth cycle is calculated; among them, = represents floor, and r is the number of loops. In this case, the coordinates of the pixel at the center of the output pixel neighborhood corresponding to the currently processed pixel are calculated based on the row and column index values ​​of the currently processed pixel. This embodiment can be understood as combining all frames and all rows into a large square matrix, which enters the corresponding output channel.

[0077] Example 4

[0078] In the fourth embodiment of the present invention, n row caches are provided, and the number of pixels that can be stored in each row cache is m. The pixels of each row in the original image of the current screen are stored in the corresponding row cache; wherein the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q; wherein the pixel neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood, and q is the number of rows of the neighborhood. According to the row index value i and the column index value j of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q, including: according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; when it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are portions of the neighborhood of the currently processed pixel that exceed the current frame or are not directly or indirectly adjacent to the current pixel; wherein the pixel values ​​of the irrelevant pixels are replaced, including: the upper limit or lower limit of the value range of the corresponding pixel value of the irrelevant pixel, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above-mentioned processing.

[0079] Let the m×n area be equal to the C×R area, that is, an m×n area constitutes a frame of pixels. It is not difficult to know that at this time, when C≥m and R>n are satisfied, a row cache coverage mechanism is introduced for different frame images (different from the row cache coverage mechanism objects of Examples 2 and 3, the row cache coverage mechanism of Examples 2 and 3 is "for the same frame image", here it is "for different frame images"), and the output pixel neighborhood is reordered. When the (kn+i)th row of pixels is input and the total number of rows of the current frame is R, the ((k-1)n+i)th row of the current frame will be overwritten, where k is the number of frames of the original image data.

[0080] When the current processing pixel position is in the center of the pixel neighborhood and the number of input pixels is In the case of , calculate the output pixel neighborhood of the k-1th frame; when the current processing pixel position is located in the center of the pixel neighborhood and the number of input pixels , calculate the output pixel neighborhood of the kth frame; where, In this case, the coordinates of the pixel at the center of the output pixel neighborhood corresponding to the current processing pixel can be obtained based on the row and column index values ​​of the current processing pixel.

[0081] In the above implementation process, the number of input pixels Q and By comparison, it is not difficult to understand that the center position of the pixel neighborhood (i.e. the position of the current processing pixel) is determined based on the row index value i and column index value j of the current input pixel, combined with the size of the neighboring pixels. When the current processing pixel is located in the output pixel neighborhood of the k-1th frame; the number of current input pixels is greater than or equal to , the currently processed pixel is located in the output pixel neighborhood of the kth frame. Next, using the number of input pixels and their row and column indices, the coordinates of the pixel at the center of the output pixel neighborhood corresponding to the currently processed pixel can be calculated, thereby outputting the location of the p×q pixel neighborhood. In other words, based on the row and column indices of the currently processed pixel, the output can be retrieved sequentially, avoiding image content misalignment or data distortion caused by a disordered output order.

[0082] Example 5

[0083] In the fifth embodiment, n row caches are provided, and the number of pixels that can be stored in each row cache is m. The pixels of each row in the original image of the current frame are stored in the corresponding row cache; wherein, the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q; wherein, the pixel neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood size, and q is the number of rows of the neighborhood size. According to the row index value i and the column index value j of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q, including: according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; when it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are portions of the neighborhood of the currently processed pixel that exceed the current frame or are not directly or indirectly adjacent to the current pixel; wherein the pixel values ​​of the irrelevant pixels are replaced, including: the upper limit or lower limit of the value range of the corresponding pixel value of the irrelevant pixel, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above-mentioned processing.

[0084] If the size of a frame of image is C×R (C columns and R rows), the number n of row buffers is equal to q, and the number m of pixels that each row buffer can store is equal to C. That is, the number m of pixels that each row of the row buffer can accommodate is consistent with the horizontal size of the original image (i.e., the parameter C in the size of the original image), then there is a formula Q = i·C + j + 1, which is convenient for more convenient calculation of the position of the corresponding output neighborhood. Similarly, if the number n of row buffers is consistent with the height size of the original image (i.e., the parameter R in the size of the original image), it will also simplify the calculation of the corresponding position of the output neighborhood to a certain extent. The size of the neighborhood on which the output depends is p×q (p columns and q rows, and usually p and q are odd numbers). The k-th frame, the i-th row, and the j-th pixel are being input, and the current processing pixel can also be understood as the pixel to be calculated (in accordance with the computer field convention, k, i, and j all start from 0):

[0085] For the case where the current processing pixel is at the center of the neighborhood, the delay Among them, the delay D represents the number of pixels that need to be accumulated to start outputting the current frame. It can be understood that this delay corresponds to the row and column offsets required by the neighborhood center. It can be known that the neighborhood is completely within the current frame, that is, the delay D ≤ 0 (no need to wait for the data of the previous frame). At the same time, after starting the output, for each input pixel of each frame, the neighborhood of the corresponding current processing pixel will be output at one time with a pixel size of p×q. The frame number and coordinates of the pixel at the center of the output pixel neighborhood corresponding to the current processing pixel are represented as the k ,

[0090] ,

[0089] , , , , ,

[0091] ,

[0088] , , -th frame, the i ′ -th row, and the j ′ -th pixel (new coordinate position), then there are:

[0086] If iC + j < D, then the previous frame is being calculated:

[0087]

[0088] If iC + j ≥ D, then the current frame is being calculated:

[0089]

[0090] Among them, mod is the modulo operation, which represents the floor function.

[0091] Based on the row index value i and the column index value j, combined with the neighborhood pixel size, judge the center position of the pixel neighborhood to determine whether the current frame output condition is reached. Here, it means that the number of pixels to be accumulated reaches the delay Only after the current frame is output, will the pixel neighborhood of the currently processed pixel be output in p × q pixels for each pixel in each frame until data processing is complete. By limiting the neighborhood range or adjusting the timing, each output pixel can be ensured to use only input pixels from the current frame. For example, edge padding (such as mirroring or duplication) can be used to avoid cross-frame dependencies, or sufficient data can be cached in advance through pipeline settings.

[0092] Example 6

[0093] In the sixth embodiment, n row caches are provided, each row cache can store m pixels, and the pixels of each row in the original image of the current frame are stored in the corresponding row cache; wherein, the size of the original image is C×R, C is the total number of columns of the original image, and R is the total number of rows of the original image; according to the row index value i (0≤i<n) and column index value j (0≤j<m) of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q; wherein, the pixel neighborhood size on which the method depends is p×q, p is the number of columns of the neighborhood size, and q is the number of rows of the neighborhood size. According to the row index value i and the column index value j of the currently processed pixel, the neighborhood of the currently processed pixel is output in sequence with a neighborhood size of p×q, including: according to the row and column index values ​​of the currently processed pixel, when the neighborhood size of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; when it is determined that the neighborhood of the currently processed pixel contains irrelevant pixels, replacing the pixel values ​​of the irrelevant pixels and outputting the neighborhood of the currently processed pixel with a size of p×q; wherein the irrelevant pixels are portions of the neighborhood of the currently processed pixel that exceed the current frame or are not directly or indirectly adjacent to the current pixel; wherein the pixel values ​​of the irrelevant pixels are replaced, including: the upper limit or lower limit of the value range of the corresponding pixel value of the irrelevant pixel, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, a preset specific value, and other specific values ​​that can be used for the above-mentioned processing.

[0094] If the size of a frame image is C×R (C columns and R rows), the number of row buffers n is equal to q, and the number of pixels that each row buffer can store is m, which is equal to C. That is, the number of pixels m that each row of the row buffer can accommodate is consistent with the horizontal size of the original image (i.e., the parameter C in the original image size). Therefore, the formula Q = i·C+j+1 is used, which makes it easier to calculate the position of the corresponding output neighborhood. Similarly, if the number of row buffers n is consistent with the height size of the original image (i.e., the parameter R in the original image size), this will also simplify the calculation of the corresponding position of the output neighborhood to a certain extent.

[0095] Moreover, the size of the neighborhood on which the output depends is p×q (p columns and q rows, and usually p and q are odd numbers). When inputting the pixel at the i-th row and j-th column of the k-th frame, the currently processed pixel, which can also be understood as the pixel to be calculated (in accordance with the computer neighborhood convention, k, i, and j all start from 0), Embodiment 6 focuses on the following changes in the case where the currently processed pixel is not at the center of the neighborhood and p and q are not limited to odd or even numbers:

[0096] If the position of the currently processed pixel in the neighborhood is at the c-th column and r-th row, then the delay D = (q - r - 1)·C + (p - c - 1). The frame number and coordinates of the pixel at the center of the neighborhood of the output pixel corresponding to the currently processed pixel are represented as the k-th ′ frame, the i-th ′ row, and the j-th ′ pixel. Then, there is:

[0097] If iC + j < D, then the previous frame is being calculated:

[0098]

[0099] If iC + j ≥ D, then the current frame is being calculated:​​​​​​​​​​​​​​​​​​​​As shown, the circuit includes a row cache unit 11, a data input module 12, an output module 13, a control module 14, a boundary processing module 15, and a row re-indexing module 16. The row cache unit 11 is used to store the row pixels of the original image; the data input module 12 is used to store each row of pixels of the original image in the corresponding row cache unit 11 in sequence; the output module 13 is used to dynamically calculate the pixels of the output image based on the data in the row cache; the control module 14 is used to control the inter-row loop (the loop after the row is covered) and the intra-row loop (the loop within a single loop); the boundary processing module 15 is used to assign pixel values ​​to irrelevant pixels when the pixel neighborhood size in the row cache is less than n×n; and the row re-indexing module 16 is used to adjust the row order after the inter-row loop to ensure the correctness of the output image.

[0105] Optionally, in an actual circuit, the video image cache processing device may be implemented based on different hardware platforms, such as an FPGA (field programmable gate array), a microprocessor or other digital IC.

[0106] Optionally, the row cache unit 11 may be implemented using an SRAM, a register group, or other storage circuits.

[0107] Alternatively, the data input module 12 may be implemented using a data bus, a shift register or other data transmission circuits.

[0108] Alternatively, the output module 13 may be implemented using a logic circuit, an arithmetic logic unit (ALU) or other computing circuits.

[0109] Alternatively, the control module 14 may be implemented using a state machine, a counter, or other control circuits.

[0110] Optionally, the boundary processing module 15 can be implemented using a logic judgment circuit, a constant generation circuit or other assignment circuits.

[0111] Optionally, the row re-indexing module 16 may be implemented using an index mapping table, an address generation circuit, or other row order adjustment circuits.

[0112] Optionally, a loop counter may be added to count the number of loops. Similarly, during the loop process, multiple loop counters may be used to count different loop parameters.

[0113] Similarly, in combination with the aforementioned image buffering method, a circuit embodiment is described with a 3×3 neighborhood output pixel as an example: Figure 8 , Figure 8 A schematic diagram of a circuit structure provided in an embodiment of the present application.

[0114] There are three line caches cr0, cr1, and cr2, and three switches sw0, sw1, and sw2.

[0115] The switches sw0, sw1, and sw2 control the flow of data to the row caches cr0, cr1, and cr2, respectively. A low position indicates that the switch is on, allowing data to flow to the corresponding row cache. A high position indicates that the switch is off, preventing data from flowing to the corresponding row cache, but allowing data within the row to circulate.

[0116] When the first line is input, sw0 is placed in the lower position and data flows to cr0; sw1 and sw2 are placed in the upper position and data does not flow to cr1 and cr2.

[0117] Similarly, when the second row is input, sw1 is placed in the lower position and the data flows to cr1; sw0 and sw2 are placed in the upper position, and the data will not flow to cr0 and cr2. At this time, the intra-row loop begins.

[0118] Similarly, when the third row is input, sw2 is placed in the lower position and the data flows to cr2; sw0 and sw1 are placed in the upper position and the data does not flow to cr0 and cr1.

[0119] Similarly, when the fourth row is input, sw0 is placed in the lower position again, and the data flows to cr0; sw1 and sw2 are placed in the upper position, and the data will not flow to cr1 and cr2. At this time, the inter-row loop begins.

[0120] Similarly, when the fifth row is input, sw1 is placed in the lower position again, and the data flows to cr1; sw0 and sw2 are placed in the upper position, and the data will not flow to cr0 and cr2.

[0121] Similarly, when the sixth row is input, sw2 is placed in the lower position again, and the data flows to cr2; sw0 and sw1 are placed in the upper position, and the data will not flow to cr0 and cr1.

[0122] Corresponding to the aforementioned image caching method, cr0, cr1, and cr2 are three line caches, and reg0, reg1, and reg2 are the 3×3 pixel neighborhood of the currently processed pixel.

[0123] In the structure of this circuit, a 3×3 neighborhood output pixel is still taken as an example.

[0124] When the first row of the first frame image is input, the input image is sent to cr0, sw0 is placed in the lower position, and 640 data are input. Here, 640 corresponds to the number of columns C of the original image pixels. Figure 8 Among them, three data are being processed and the remaining 637 data are waiting to be processed.

[0125] When inputting in the second row, the input image is sent to cr1, sw1 is placed in the lower position, and sw0 is placed in the upper position. When data is input in cr1, the data in cr0 is also circulated together. This is an intra-row loop.

[0126] When the second pixel in the second row is input, there are 4 valid data points. The other 5 data points are replaced with irrelevant pixel values ​​during the calculation process, and the first pixel in the first row of the output image can be calculated. Similarly, when the third pixel in the second row is input, there are 6 valid data points. The other 3 data points are replaced with irrelevant pixel values ​​during the calculation process, and the second pixel in the first row of the output image can be calculated. After that, when each pixel is input, there are 6 valid data points, and the calculation of other pixels in the first row of the output image can continue. When calculating the pixels at the end of the row, the pixel values ​​of irrelevant pixels other than the valid data are replaced in the same way.

[0127] When inputting in the third line, the input image is sent to cr2, sw2 is placed in the lower position, sw0 and sw1 are placed in the upper position, and when data is input in cr2, the data in cr0 and cr1 are also circulated together.

[0128] When the first pixel is input in the third row, the last pixel of the first row of the output image can be calculated. At this time, 5 pixels can be replaced by irrelevant pixel values ​​during the calculation process; when the second pixel is input in the third row, the first pixel of the second row of the output image can be calculated. At this time, 3 pixels can be replaced by irrelevant pixel values ​​during the calculation process; when the third pixel is input in the third row, the second pixel of the second row of the output image can be calculated. At this time, all 9 pixels are valid pixels within the current frame boundary. After that, when each pixel is input, there are 9 valid pixels, and the calculation of other pixels in the second row of the output image can continue.

[0129] When inputting the 4th row, the input image is sent to cr0 again. This is an inter-row cycle. sw0 is placed at the lower position, and sw1 and sw2 are placed at the upper position. When data is input into cr0, the data in cr1 and cr2 are also circulated together.

[0130] Similarly, when the first pixel in the 4th row is input, the last pixel in the 2nd row of the output image can be calculated; when the second pixel is input, the first pixel in the 3rd row of the output image can be calculated.

[0131] Similarly, when inputting in the 5th line, the input image is sent to cr1 again, sw1 is placed in the lower position, sw2 and sw0 are placed in the upper position, and when data is input in cr1, the data in cr2 and cr0 are also circulated together.

[0132] Similarly, when inputting in the 6th line, the input image is sent to cr2 again, sw2 is placed in the lower position, sw0 and sw1 are placed in the upper position, and when data is input in cr2, the data in cr0 and cr1 are also circulated together.

[0133] Repeat this process until the last row of input is completed, and the second-to-last row and the second-to-last pixel of the output image can be calculated.

[0134] When the second frame, first row, and first pixel are input, the second-to-last row and the first-to-last pixel of the output image of the first frame can be calculated; when the second pixel is input, the first-to-last row and the first pixel of the output image of the first frame can be calculated; when the second frame, second row and the first pixel are input, the first-to-last row and the first-to-last pixel of the output image of the first frame can be calculated; when the second pixel is input, the first pixel of the first row of the second frame image can be calculated. At this time, one frame cycle is completed.

[0135] The final row re-indexing is required when the algorithm is sensitive to pixel position. It re-indexes the row order that has been "disrupted" by the inter-row loop into the correct row order.

[0136] Specifically, the corresponding relationship is as follows Figure 8 As shown, the final row order is based on a modular operation, that is, calculating the corresponding values ​​of cr[(r+0)%3], cr[(r+1)%3], and cr[(r+2)%3] can obtain their corresponding relationship, where r is the row index value.

[0137] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a processor, the steps in any one of the above methods are executed.

[0138] In the above implementation process, computer program instructions in a computer-readable storage medium are used in the image caching method, and the following steps are specifically performed: storing the pixels of each row in the original image of the current frame into the corresponding row cache, and when the output condition is met, outputting the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q and reordering the output pixel neighborhood.

[0139] See also Figure 9 , Figure 9 A schematic diagram of the processing process of a 3×3 neighborhood at the row scale provided in an embodiment of the present application.

[0140] Taking a 3×3 pixel neighborhood as an example, in digital logic, image pixels are fed into the processing unit one by one or in groups, driven by a clock signal. However, when the first row of pixels is input, the algorithm cannot perform calculations due to a lack of sufficient neighborhood data.

[0141] However, when the second pixel in the second row is input, the processing unit has accumulated enough data to form a complete 3×3 neighborhood, so that it can start calculating the first pixel of the output image. After that, each pixel input can be calculated one pixel at a time because the required neighborhood data is already in place.

[0142] When the last pixel of the last row is input, the processing unit can calculate the second to last pixel of the second to last row. When the first row of pixels of the next frame is input, although the output of the next frame cannot be calculated immediately, the last row of pixels of the previous frame can be calculated using the newly input data.

[0143] When the first pixel of the second row of the next frame is input, the processing unit can calculate the last pixel of the last row of the previous frame. Finally, when the second pixel of the second row of the next frame is input, the processing unit starts calculating the first pixel of the next frame, thus achieving continuous image processing.

[0144] Optionally, the time of each row cache overwrite event and the physical storage address of the overwritten row are recorded in a log. The log is stored in a ring buffer structure and indexed by timestamps to support fast backtracking queries. This ensures that the original spatial position information of the overwritten row can be restored based on the log in the subsequent pixel processing stage, providing data traceability for time-related image analysis (such as motion compensation and multi-frame fusion).

[0145] In summary, the present application provides an image caching method and circuit, which relates to the neighborhood of digital image processing technology. The method includes: providing n row caches, storing the pixels of each row in the original image of the current frame into the corresponding row cache; according to the row index value i (0<i≤n) and column index value j (0<j≤m) of the currently processed pixel, outputting the neighborhood of the currently processed pixel in sequence with a neighborhood size of p×q; wherein the size of the original image is C×R, C is the total number of columns, and R is the total number of rows; wherein the neighborhood size on which the method depends is p×q, p is the number of columns in the neighborhood, and q is the number of rows in the neighborhood. By cyclically utilizing a small amount of row cache to realize cyclic processing of data, the delay of each level of image processing algorithm is controlled within a few rows, which reduces the length of the data processing process and reduces the processing delay.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed devices can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices according to the multiple embodiments of the present application. In this regard, each box in the block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram, and the combination of the block diagrams, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0148] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0149] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0150] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, elements defined by the phrase "comprises..." do not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the elements.

Claims

1. An image caching method, characterized in that: The method comprises: Provide n row buffers, each of which can store m pixels, and store the pixels of each row in the original image of the current screen into the corresponding row buffer; According to the row index value i (0≤i<n) and column index value j (0≤j<m) of the current processing pixel, the neighborhood of the current processing pixel is output in sequence with a neighborhood size of p×q; The size of the original image is C×R, where C is the total number of columns of the original image and R is the total number of rows of the original image. The method relies on a neighborhood size of p×q, where p is the number of columns in the neighborhood and q is the number of rows in the neighborhood.

2. The method according to claim 1, characterized in that The method of sequentially outputting the neighborhood of the currently processed pixel with a neighborhood size of p×q according to the row index value i and the column index value j of the currently processed pixel includes: According to the row and column index values ​​of the currently processed pixel, when the size of the neighborhood of the currently processed pixel is p×q, determining whether the neighborhood of the currently processed pixel contains irrelevant pixels; When it is determined that the neighborhood of the currently processed pixel includes an irrelevant pixel, the pixel value of the irrelevant pixel is replaced, and the neighborhood of the currently processed pixel with a size of p×q is output; The irrelevant pixel is a portion of the neighborhood of the currently processed pixel that exceeds the current picture or is not directly or indirectly adjacent to the current pixel; The replacement of the pixel value of the irrelevant pixel includes: the corresponding pixel value of the irrelevant pixel is one of the upper limit or lower limit of the value range, the value of the currently processed pixel, the pixel value at a symmetrical position in the neighborhood, and a preset value.

3. The method according to claim 1, characterized in that The method further includes: sequentially outputting the neighborhood of the currently processed pixel with a neighborhood size of p×q according to the row index value i and the column index value j of the currently processed pixel; Calculate the number of input pixels based on the row and column index values ​​of the currently processed pixel; According to the number of the input pixels, the position of the neighborhood after the output corresponding to the current processing pixel is calculated; wherein the number of the input pixels is Q, and Q=i·m+j+1.

4. The method according to claim 3, characterized in that Calculating the position of the neighborhood of the current processing pixel corresponding to the output according to the number of the input pixels includes: The current processing pixel position is located in the center of the neighborhood, and the number of input pixels In the case of , it is determined that the output pixel neighborhood of the r-1th cycle is calculated; among them, represents rounding down, and r is the number of cycles.

5. The method according to claim 3, characterized in that The step of calculating the position of a neighborhood corresponding to the output of the currently processed pixel according to the number of the input pixels further includes: The current processing pixel position is located in the center of the neighborhood, and the input pixel In the case of , it is determined that the output pixel neighborhood of the rth cycle is calculated; among them, represents rounding down, and r is the number of cycles.

6. The method according to claim 3, characterized in that The step of calculating the position of a neighborhood corresponding to the output of the currently processed pixel according to the number of the input pixels further includes: The operation is performed based on the row and column index values ​​of the currently processed pixel to obtain the coordinates of the pixel at the center of the output neighborhood corresponding to the currently processed pixel.

7. The method according to claim 1, characterized in that The step of storing pixels of each row of the original image of the current screen into the corresponding row buffer includes: When the line buffer area is filled and the total number of lines of the current picture is R (R≥n), when the pixel is input into the i-th line buffer, the line buffer of the i-n+R-th line of the filled line buffer area is overwritten.

8. The method according to claim 1, characterized in that The step of storing pixels of each row of the original image of the current screen into the corresponding row buffer further includes: When the (rn+i)th row of pixels is input, the ((r-1)n+i)th row in the current processing process is covered; where r is the number of cycles and r≥1; When inputting the (rn+i)th row of pixels and overwriting the row buffer, recording the overwriting order of the row buffer and the row number corresponding to the pixel input in the overwritten row buffer in the original image; Reorder the row outputs in the row buffer according to the row index value of the currently processed pixel.

9. A circuit, characterized in that: The circuit includes a programmable or non-programmable execution unit, and the execution unit executes the image caching method according to any one of claims 1 to 8 based on a predetermined function during design or an executable program therein.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the image caching method according to any one of claims 1 to 8 is executed.

Images