An image processing method and chip based on a fixed direction

Abstract

The present invention relates to an image processing method and a chip based on a fixed direction. The image processing method includes: before the sliding window starts the first translation in the image to be processed, and whenever the sliding window makes a translation in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, searching for neighboring pixel points in the image area currently framed by the sliding window along a pair of preset search directions, and obtaining the similarity of the image information between the neighboring pixel points searched in the corresponding preset search directions and the center pixel point; wherein, the storage order of each pixel point in the image to be processed in the memory is row-by-row storage; the pixel point covered by the center position of the sliding window is configured as the center pixel point. The number of pixel pairs participating in the calculation is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to an image processing method and a chip based on a fixed direction. Background Art

[0002] In the existing computer memory management mechanism, images are commonly stored in the form of matrices; whenever an image is mentioned, it generally refers to a two-dimensional matrix (array), and its storage in computer memory is a continuous address space, which can be determined by the storage addresses of the first pixel and the last pixel, or by the first pixel and the total number of pixels. The elements of the two-dimensional array used to represent an image are stored row by row in memory, that is, all the data of the first row are stored first in this continuous address memory space, then the memory data of the second row are stored, and so on until the last row of the image.

[0003] In the process of filtering an image in the prior art, such as in the process of performing filtering and denoising processing such as sobel filtering, Gaussian filtering, bilateral filtering, etc. on an image, when calculating the result corresponding to a pixel and its adjacent 8 pixels, cross-row operations will be involved. For example, when calculating the difference between the gray value of a currently traversed pixel and the gray value of the pixel in the same column in the previous row, a cross-row read operation needs to be performed. Especially when performing operations pixel by pixel (involving exponential operations, square operations, division operations), image data needs to be frequently read across rows. Since the cache space (Cache) of the system device is still too small compared to an image with millions of pixel points, it is often not enough to store a complete frame of the image. Therefore, the row image data in the conventional memory will frequently enter and exit the cache space (Cache), easily causing the problem of cache invalidation and affecting the reading of the CPU. Summary of the Invention

[0004] In order to overcome the above technical defects, the present invention discloses an image processing method and a chip based on a fixed direction, which can effectively solve the problems of cache invalidation and excessive calculation amount. The specific technical solutions are as follows:

[0005] An image processing method based on a fixed direction, the image processing method comprising: before the sliding window starts the first translation in the image to be processed, and whenever the sliding window makes a translation in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, searching for neighboring pixel points in the image area currently framed by the sliding window along a pair of preset search directions, and obtaining the similarity of image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point; wherein, the storage order of each pixel point in the image to be processed in the memory is row-by-row storage; the pixel point covered by the center position of the sliding window is configured as the center pixel point; and the neighboring pixel points are the pixel points located within the neighborhood of the center pixel point.

[0006] Compared with the prior art, when it is necessary to calculate and obtain the similarity of image information (the gap in spatial distance and the gap in pixel values) between pixel point pairs each time, pixel point pairs in a fixed search direction are used to obtain representative similarity of image information, forming a traversal operation of fixed pixel positions, reducing the number of pixel points required for calculating the similarity of image information under the condition of a single translation of the sliding window, and reducing the number of pixel pairs participating in the calculation; the image processing method based on a fixed direction provided by this technical solution can significantly reduce the amount of calculation required for a filtering operation when it is applied to an image filtering operation, thereby avoiding the cache miss problem caused by frequent CPU cross-row access to the image, increasing the probability of the accessed image data in the cache space (Cache), and then improving the speed of the CPU accessing (traversing) the image data, thus improving the execution speed of image processing.

[0007] Further, the image processing method further comprises: whenever the sliding window covers all pixel points in the image to be processed, it is determined that the sliding window has completed one round of cyclic traversal of the image to be processed; when the sliding window performs the preset number of rounds of cyclic traversal of the image to be processed, the similarity of image information between the neighboring pixel points in a preset number of pairs of different preset search directions and the matching center pixel points is obtained, wherein, the pair of preset search directions along which the image to be processed is traversed in different rounds of cyclic traversal is different, the pair of preset search directions along which the image to be processed is traversed in the same round of cyclic traversal remains the same, and the pair of preset search directions along which the image to be processed is traversed in the same round of cyclic traversal are two different preset search directions.

[0008] Compared with the prior art, in this technical solution, by performing a preset number of cyclic traversals on the neighborhood pixel points in a pair of preset search directions within the same image to be processed, it is possible to ensure that in each sliding operation during each round of cyclic traversal, a fixed number of image rows in a fixed direction are read, reducing the number of times image data enters and exits the cache space. Thus, the image traversal speed is increased by reducing the number of pixel points for search calculation. At the same time, it is possible to set a pair of preset search directions with regional representativeness and set the pair of preset search directions corresponding to each round of cyclic traversal to be different, taking into account the comprehensiveness of the traversed image area.

[0009] Further, during the process of the same round of cyclic traversal, when the sliding window frames a specific image area in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighborhood pixel points along the first preset search direction within the specific image area currently framed by the sliding window, and obtain the similarity of the image information between each searched neighborhood pixel point and the center pixel point; during the process of the same round of cyclic traversal, when the sliding window frames a specific image area in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighborhood pixel points along the second preset search direction within the image area currently framed by the sliding window, and obtain the similarity of the image information between each searched neighborhood pixel point and the center pixel point; during the process of the same round of cyclic traversal, a pair of preset search directions are the first preset search direction and the second preset search direction, and the first preset search direction and the second preset search direction are opposite; wherein, the distances between each successively searched neighborhood pixel point and the center pixel point in the same direction of the preset search direction are different, and the neighborhood pixel point closest to the center pixel point is located in the octal neighborhood of the center pixel point.

[0010] When setting a pair of preset search directions in this technical solution, the pixel points searched in the first preset search direction and the pixel points searched in the second preset search direction during the same round of cyclic traversal are centrosymmetric with respect to the pixel point currently covered by the center position point of the sliding window, taking into account the distribution symmetry and relative fixity of the distribution positions of the pixel points to be searched within the image area covered by the sliding window.

[0011] Further, during the process of the same round of cyclic traversal, whenever the sliding window is translated by a preset step length within the image to be processed, the first preset search direction along which neighboring pixel points are searched within the image region currently framed by the sliding window is the same as the first preset search direction along which neighboring pixel points were searched within the image region previously framed by the sliding window, and the second preset search direction along which neighboring pixel points are searched within the image region currently framed by the sliding window is the same as the second preset search direction along which neighboring pixel points were searched within the image region previously framed by the sliding window. Among them, neighboring pixel points with the same orientation refer to neighboring pixel points located on the same preset search direction and at equal distances from the corresponding central pixel point. This ensures that the neighboring pixel points with the same orientation searched before and after the sliding window is translated by the preset step length are all within the same row of the image to be processed. Since the image to be processed is stored row by row in a specific memory, it is convenient for the CPU to read the pixel points of the image to be processed row by row from the memory, or for the CPU to store the pixel points of the image to be processed row by row into the cache space. This further reduces the number of cross-row reads and improves the hit rate of the pixel points accessed by the CPU in the cache space.

[0012] Further, during each round of cyclic traversal, an equal number of pixel points are searched along a pair of preset search directions within the image region framed by the sliding window. Among them, whenever the sliding window is translated to an image region of the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, the number of pixel points searched along the first preset search direction within the image region framed by the sliding window is equal to the number of pixel points searched along the second preset search direction within the image region framed by the sliding window.

[0013] This controls the entire translation of the sliding window within the image to be processed, which can not only ensure that the region framed by the sliding window is entirely within the image to be processed, but also ensure the symmetry of searching for neighboring pixel points along a pair of preset search directions.

[0014] Further, the preset quantity is 4; when the first preset search direction is the upper left diagonal direction, the second preset search direction is the lower right diagonal direction; wherein, the upper left diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the upper left corner point of the image area framed by the sliding window; the lower right diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the lower right corner point of the image area framed by the sliding window; when the first preset search direction is the lower left diagonal direction, the second preset search direction is the upper right diagonal direction; wherein, the lower left diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the lower left corner point of the image area framed by the sliding window; the upper right diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the upper right corner point of the image area framed by the sliding window; when the first preset search direction is the horizontal left direction, the second preset search direction is the horizontal right direction; wherein, the horizontal left direction starts from the pixel point covered by the center position of the sliding window and extends leftward along the horizontal axis direction of the image area framed by the sliding window; the horizontal right direction starts from the pixel point covered by the center position of the sliding window and extends rightward along the horizontal axis direction of the image area framed by the sliding window; when the first preset search direction is the vertical upward direction, the second preset search direction is the vertical downward direction; wherein, the vertical upward direction starts from the pixel point covered by the center position of the sliding window and extends upward along the vertical axis direction of the image area framed by the sliding window; the vertical downward direction starts from the pixel point covered by the center position of the sliding window and extends downward along the vertical axis direction of the image area framed by the sliding window.

[0015] This technical solution realizes cycling through the image to be processed four times along symmetric preset search directions, respectively traversing and calculating (including processing sequentially or simultaneously) the pixel pairs in the horizontal left - right direction, the upper left - lower right diagonal direction, the vertical up - down direction, and the lower left - upper right diagonal direction, reducing the computational amount of pixels in a single sliding operation and improving the image processing speed.

[0016] Further, the sliding window is represented by a square window, and the side length of this square window is 2×W + 1, where W is a positive integer, such that the pixel point covered when the center position of the sliding window slides within the image to be processed becomes the center pixel point, and when the sliding window translates by a preset step length, at most 2W neighboring pixel points can be framed in a pair of preset search directions.

[0017] Further, before the sliding window starts the first translation in the image to be processed, and whenever the sliding window translates a preset step length within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions, and obtain the similarity of the image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point, to determine that the sliding window performs a sliding operation within the image to be processed; whenever all the image areas framed by the sliding window have covered all the pixel points within the image to be processed, and the similarities of the image information between all the neighboring pixel points in all pairs of preset search directions within all the image areas framed by the sliding window and the corresponding center pixel points are obtained, it is determined that the sliding window has completed a round of cyclic traversal of the image to be processed.

[0018] This technical solution determines that the sliding window has performed a sliding operation within the image to be processed according to the traversal method of neighboring pixel points in a specific direction and the acquisition method of the similarity of their image information, and then determines a round of cyclic traversal of the image to be processed according to the full-coverage method of the sliding window for the image to be processed, forming a cyclic traversal mechanism for a preset number of pairs of neighboring pixel points in the preset search directions within the sliding window, where different pairs of preset search directions are different from each other.

[0019] Further, the method for obtaining the similarity of the image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point includes: whenever the sliding window translates a preset step length within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions, and calculate the similarity of the image information between the currently searched neighboring pixel point and the center pixel point without repetition; where the similarity of the image information includes the similarity of gray values and the spatial proximity; the similarity of gray values is used to represent the degree of difference between the gray values of two pixel points; the spatial proximity is used to represent the degree of difference in the spatial distance between two pixel points.

[0020] Compared with the prior art, during the translation of the sliding window within the image to be processed, this technical solution controls that there is no repeated calculation of the similarity of the image information between the same pair of pixel points, significantly reducing the number of calculations, and is applicable to the image processing of chips with a small cache space. Among them, this pair of pixel points includes the currently traversed neighboring pixel point and the center pixel point.

[0021] Further, the method for calculating the similarity of image information between the currently searched neighboring pixel points and the central pixel point without repetition includes: If the pixel point covered by the central position of the sliding window in the current sliding operation is updated from a neighboring pixel point searched along a pair of preset search directions in a previously executed sliding operation, the pixel point covered by the central position of the sliding window in the current sliding operation is set as the first reference central pixel point, and at the same time, it is determined that the pixel point covered by the central position of the sliding window in the previously executed sliding operation belongs to a neighboring pixel point that can be searched along a pair of preset search directions within the image area currently framed by the sliding window, and the pixel point covered by the central position of the sliding window in the previously executed sliding operation is set as the second reference central pixel point; wherein, in the previously executed sliding operation, the gray value similarity between the first reference central pixel point and the second reference central pixel point has been calculated; then, if a neighboring pixel point searched along a pair of preset search directions within the image area currently framed by the sliding window is updated from the central pixel point covered by the central position of the sliding window in the previously executed sliding operation, the gray value similarity between the aforementioned first reference central pixel point and the second reference central pixel point is directly updated to the gray value similarity between the currently searched neighboring pixel point and the pixel point covered by the central position of the sliding window in the current sliding operation.

[0022] This technical solution controls that only one gray value similarity calculation is performed between a pair of pixel points that may be repeatedly traversed and need to participate in the gray value similarity calculation. It can configure the gray value similarity calculated when the pair of pixel points is first traversed as the gray value similarity required when the pair of pixel points is repeatedly traversed later. Thus, compared with the existing repeated calculation phenomenon, the number of calculations is reduced by half, and the calculation speed of all pixel points of the image to be processed is increased.

[0023] Further, the method for calculating the similarity of image information between the currently searched neighboring pixel points and the central pixel point without repetition includes: when the sliding window has not started to translate within the image to be processed, within the initial image area framed by the entire sliding window within the image to be processed, starting from the pixel point currently covered by the central position of the sliding window, searching for neighboring pixel points within this initial image area along a pair of preset search directions, calculating the spatial proximity between the successively searched neighboring pixel points and the same central pixel point, and storing the calculated spatial proximities between the neighboring pixel points at different pixel positions within this initial image area and the same central pixel point into a preset memory space, so as to directly call the corresponding spatial proximities when the sliding window subsequently performs a sliding operation within the image to be processed; before the sliding window starts to translate within the image to be processed and in each sliding operation performed according to the preset step size, the spatial proximities between the neighboring pixel points with the same relative position relationship and the corresponding central pixel point within the image area framed by the sliding window within the image to be processed are equal.

[0024] In each round of the cyclic traversal, the fixed result of the spatial proximity calculated within the initial image area is saved for utilization in the subsequently shifted framed image, so as to more quickly call the spatial proximities at the corresponding position relationships calculated in advance and avoid repeated calculation of the spatial proximities.

[0025] Further, for the N corresponding neighboring pixel points traversed in a sliding operation performed by the sliding window, control the CPU to call N computing registers to calculate in parallel the similarity of image information between the N traversed neighboring pixel points and the same central pixel point in this sliding operation; wherein, each computing register is used to calculate the similarity of image information between a neighboring pixel point and the same central pixel point in a corresponding sliding operation; wherein, N is a positive integer greater than or equal to 2.

[0026] This technical solution uses a parallel computing method to improve the calculation speed of the similarity of image information corresponding to multiple neighboring pixel points, and further improve the calculation speed of the gray value similarity and its spatial proximity between multiple neighboring pixel points and a corresponding central pixel point, which can be used to accelerate the image processing speed.

[0027] Further, the N corresponding neighboring pixel points traversed in a sliding operation performed by the sliding window are the neighboring pixel points searched within the image area framed by the sliding window, starting from the central position of the sliding window, along the first preset search direction and / or the second preset search direction. This speeds up the operation speed of a sliding operation, and further improves the speed of a round of cyclic traversal of the image to be processed.

[0028] A chip for storing the program code corresponding to the described image processing method, which is used to reduce the number of times the pixel points of the image to be processed enter and exit the cache space when the image processing method is executed, where the cache space is set between the CPU and the memory for storing the image to be processed. Brief Description of the Drawings

[0029] Figure 1 The flowchart of an embodiment of the present invention discloses an image processing method based on a fixed direction.

[0030] Figure 2 The flowchart of another embodiment of the present invention discloses an image processing method based on a fixed direction (multiple rounds of cyclic traversal of the same image to be processed). Detailed Embodiments

[0031] Next, the technical solutions in the embodiments of the present invention will be described in detail with reference to the accompanying drawings in the embodiments of the present invention. It should be understood that the specific embodiments described below are only used to explain the present invention and are not used to limit the present invention. In the following description, specific details are given to provide a thorough understanding of the embodiments. However, those of ordinary skill in the art will understand that the embodiments can be implemented without these specific details. For example, the circuit can be shown in a block diagram to avoid obscuring the embodiments with unnecessary details. In other cases, well-known circuits, structures, and technologies can be not shown in detail to avoid confusing the embodiments.

[0032] The terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0033] It can be understood by those skilled in the art that the cache memory is a temporary memory located between the CPU and the memory (with a cache space Cache opened up), and its capacity is smaller than that of the memory. The data in the Cache is a small part of the data in the memory, and this small part is the data that the CPU will access in a short time. The CPU can directly call the data from the Cache instead of the memory, and such a reading mechanism makes the hit rate of the CPU reading the Cache very high. That is to say, about 90% of the data that the CPU will read next time is in the Cache, and only about 10% needs to be read from the memory, which greatly saves the time for the CPU to directly read the memory and also makes the CPU basically not need to wait when reading data. Generally speaking, the order for the CPU to read data is to read from the Cache first and then from the memory.

[0034] Since the Cache of a computer is still too small compared to an image frame with millions of pixel points and is not large enough to hold a complete image frame, when a data block in the memory enters the Cache at one time, the number of buffered rows is not many, and the hit rate of data in the Cache (the probability that the data required by the CPU is in the Cache) is not high. To meet the actual data volume read by the CPU, the row image data in the conventional memory needs to frequently enter and exit the Cache, which restricts the speed at which the CPU accesses data and affects the execution speed of the image processing algorithm.

[0035] It should be noted that the algorithm complexity does not exactly correspond to the execution speed. If the algorithm is not conducive to buffer optimization, even if the algorithm complexity is very low, the execution speed may not be fast. The influence of the hit rate of the buffer and the memory speed on the image processing speed cannot be underestimated. Moreover, the Cache has a great influence on the performance of the CPU, mainly caused by the data exchange order of the CPU and the bandwidth between the CPU and the Cache.

[0036] In the traditional method, it is necessary to calculate the similarity of the corresponding image information for each central pixel point and its eight adjacent neighborhood pixel points, which involves multiple cross-row operations and requires frequent cross-row reading of the image data in the memory into the cache space. For example, when calculating the similarity of the image information between the current pixel point and the pixel point in the same column of the previous row of the image to be processed, a cross-row read operation needs to be performed. When the number of executions of the cross-row read operation is large, it is easy to have the problem of cache invalidation in a small embedded device.

[0037] To overcome the foregoing technical deficiencies, an embodiment of the present invention discloses an image processing method based on a fixed direction, as Figure 1 shown, the image processing method includes:

[0038] Step S101, the sliding window starts to translate within the image to be processed to traverse the image to be processed, determines that the image to be processed starts to be traversed, and the traversal method is a translation method with the sliding window as the convolution kernel, and then enters step S102. In this embodiment, by executing step S101, an image area of a fixed size is framed, so as to search for pixel points that meet the expected processing target within the currently framed image area. Among them, the sliding window is a rectangular frame with a central position and used for image traversal, which is a specific term in the field of image processing. In this embodiment, the image to be processed is pre-saved in the image memory, such as a DDR memory electrically connected to the CPU.

[0039] Step S102: Before the sliding window starts the first translation in the image to be processed and every time the sliding window makes a translation in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions, and obtain the similarity of the image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point.

[0040] Specifically, after the sliding window makes a translation in the image to be processed, within the image area currently framed by the sliding window, starting from the pixel point currently covered by the center position of the sliding window, respectively along two preset search directions (i.e., a pair of preset search directions), sequentially traverse the neighboring pixel points, calculate the similarity of the image information between the neighboring pixel points searched in the currently traversed preset search direction and the center pixel point, and then store the calculated similarity of the image information, or directly retrieve the similarity of the image information between the corresponding pixel points from the image memory to determine the similarity of the image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point. The neighboring pixel points in the foregoing two preset search directions can be traversed simultaneously, or sequentially, or one of the preset search directions can be read in advance, and on this basis, the pixel points in the other preset search direction can be traversed. In some embodiments, the pixel point at the upper left corner of the image area currently framed by the sliding window is a neighboring pixel point searched along one of the pair of preset search directions, and it is necessary to calculate and obtain the similarity of the image information between this neighboring pixel point and the center pixel point as the similarity of the image information at the upper left corner position; the pixel point at the lower right corner of the image area currently framed by the sliding window is a neighboring pixel point searched along the other of the pair of preset search directions, and it is necessary to calculate and obtain the similarity of the image information between this neighboring pixel point and the center pixel point as the similarity of the image information at the lower right corner position.

[0041] Regarding step S102, it should be added that when the sliding window has not started to translate in the image to be processed, that is, in the initial state of image processing, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions, and start calculating the similarity of the image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point. At this time, the calculated similarity of the image information is the similarity of the image information between the currently searched neighboring pixel point and the center pixel point that was first covered, where the pixel point currently covered by the center position of the sliding window is used as the center pixel point that was first covered by the center position of the sliding window in the image to be processed; the neighboring pixel points are the pixel points within the neighborhood of the center pixel point.

[0042] It should be noted that the pixel points covered by the center position of the sliding window are configured as center pixel points; the neighborhood pixel points are the pixel points within the neighborhood of the center pixel point, including but not limited to the eight-neighborhood and 24-neighborhood, which are specifically related to the size of the sliding window; among them, the image information similarity includes the gray value similarity and the spatial proximity required by the filtering algorithm; the gray value similarity corresponds to the intensity difference in the pixel domain of the image to be processed, and the spatial proximity corresponds to the geometric gap in the spatial domain of the image to be processed. The storage order of each pixel point in the image to be processed in the memory is row-by-row storage; the pixel points covered by the center position of the sliding window are configured as center pixel points.

[0043] In summary, compared with the prior art, in this embodiment, when it is necessary to calculate and obtain the image information similarity (the gap in spatial distance and the gap in pixel values) between pixel point pairs each time, the pixel point pairs in a fixed search direction are used to obtain the representative image information similarity, forming a traversal operation of the fixed pixel positions, which is equivalent to taking out the region of the pixel positions of interest (certainly much smaller than the image to be processed) from the image to be processed in a single translation operation of the search window, reducing the number of pixel points required for calculating the image information similarity under the condition of a single translation of the sliding window, reducing the number of pixel pairs participating in the calculation, and since the pixel pairs participating in the calculation are in different two rows within the same sliding window. Therefore, when the CPU executes the calculation of the image information similarity once, it can avoid the operation of reading pixel points across rows in order to read a neighborhood pixel point and a corresponding center pixel point in different two rows; the image processing method based on a fixed direction provided in this embodiment can significantly reduce the calculation amount required for the filtering operation when applied to an image filtering operation, reduce the number of times the data blocks in the image memory enter and exit the Cache, and further avoid the cache miss problem caused by the CPU frequently accessing the image across rows, increasing the probability of the accessed image data in the cache space (Cache), thereby improving the speed of the CPU accessing (traversing) the image data, and thus improving the speed of image processing.

[0044] Preferably, in the corresponding preset search direction, the next row pixel point or the previous row pixel point adjacent to the row of pixel points currently read by the CPU can be pre-stored in the cache space. In this way, when the CPU needs to calculate the image information similarity between the row of pixel points currently read and the corresponding pixel points in the adjacent next row, the adjacent next row pixel point or the previous row pixel point is known and can be directly obtained from the cache space. Also, based on the fact that the image data in the image memory is always transmitted row by row, the CPU does not need to perform the operation of reading across rows, but instead performs the reading operation of two fixed rows each time.

[0045] As an embodiment, such asFigure 2 As shown, the image processing method includes:

[0046] Step S201: Whenever the sliding window makes a translation in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions. Specifically, traverse from the pixel point currently covered by the center position of the sliding window along the preset search direction to the corner point position in the preset search direction, that is, the corner point position of the image area currently framed by the sliding window in the preset search direction, and obtain the similarity of image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point. This includes calculating the similarity of image information between the position of the previously un-traversed neighboring pixel point and the position of the center pixel point, or directly reading from the storage space (such as an image memory, cache space, or other memory spaces) the similarity of image information between the position of the previously traversed neighboring pixel point and the position of the center pixel point; then proceed to step S202; where the pixel point currently covered by the center position of the sliding window is the newly set center pixel point.

[0047] It should be noted that when the sliding window makes a translation in the image to be processed, it means that the sliding window translates by one preset step length within the image to be processed. Then, whenever the sliding window translates by one preset step length within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions, and obtain the similarity of image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point. At this time, it is determined that the sliding window has performed a sliding operation within the image to be processed, which can be expressed as searching for all pixel points within the image area currently framed by the sliding window along a pair of preset search directions starting from the pixel point currently covered by the center position of the sliding window, denoted as neighboring pixel points (at least 2), and sequentially obtaining the similarity of image information between each neighboring pixel point searched in the corresponding preset search direction and the center pixel point. At this time, it is determined that the sliding window has completed a sliding operation. This is a sliding operation on the premise of extracting pixel points in a fixed search direction. Since the number of pixel points processed under a single translation operation becomes smaller, the hit rate of data in the Cache is improved.

[0048] Step S202: Repeat step S201 until the sliding window has covered all the pixel points in the image to be processed and the corresponding image information similarity is obtained, then it is determined that the sliding window has completed one round of cyclic traversal of the image to be processed; among them, for one round of cyclic traversal, a pair of preset search directions along which the sliding window moves on the image to be processed remains the same; then enter step S203; for step S202, whenever all the image regions framed by the sliding window have covered all the pixel points in the image to be processed, and the image information similarities between all the neighboring pixel points and the corresponding central pixel points in all the image regions framed by the sliding window in a pair of preset search directions are obtained, it is determined that the sliding window has performed one round of cyclic traversal of the image to be processed in the image to be processed, which can be expressed as: the sliding window undergoes multiple sliding operations and keeps a pair of preset search directions along which it moves the same in each sliding operation until all the image regions framed by the sliding window have covered all the pixel points in the image to be processed, and the image information similarities between all the neighboring pixel points and the corresponding central pixel points in all the image regions framed by the sliding window in a pair of preset search directions are obtained, then it is determined that the current round of cyclic traversal is completed, and then enter step S203 to prepare for starting the next round of cyclic traversal.

[0049] Step S203: Adjust a pair of preset search directions along which the current round of cyclic traversal moves on the image to be processed to obtain a new pair of preset search directions along which the next round of cyclic traversal moves on the image to be processed, and control that a pair of preset search directions along which different rounds of cyclic traversal move on the image to be processed are different, that is, configure the new pair of preset search directions along which the next round of cyclic traversal moves on the image to be processed to be different from the pair of preset search directions along which the current round of cyclic traversal moves on the image to be processed. Specifically, the new preset search directions along which the next round of cyclic traversal moves on the image to be processed are not the same as the preset search directions along which the current round of cyclic traversal moves on the image to be processed, but not opposite; then enter step S204. Among them, the image regions framed by the sliding window during the execution of the previous round of cyclic traversal and the image regions framed by the sliding window during the execution of the current round of cyclic traversal overlap, but the pair of preset search directions along which they move are different, so that the pixel points searched in these two rounds of cyclic traversal are partially the same.

[0050] Step S204: Repeat steps S201 to S203 until the sliding window completes the loop traversal of the preset number of rounds for the image to be processed, then the image information similarity between all neighborhood pixel points in the preset search direction and their corresponding central pixel points is obtained. In this embodiment, during the loop traversal of the preset number of rounds for the image to be processed, the two preset search directions along which different rounds of loop traversal are performed are different, that is, the two preset search directions along which the current round of loop traversal is performed are different from the two preset search directions along which the next round of loop traversal is performed; wherein, a pair of preset search directions along which the same round of loop traversal is performed on the image to be processed are two opposite preset search directions.

[0051] It should be noted that the sliding window of the prior art traverses all neighborhood pixel points comprehensively and completely in each convolution translation process, which requires calculation operations for each central pixel point and its 8 adjacent neighborhood pixels. Among them, there will be frequent cross-row operations, and each cross-row may be across three rows and multiple cross-row reading operations are adopted.

[0052] The image processing method described in the foregoing steps S201 to S204 can, by performing loop traversal of the preset number of times on the neighborhood pixel points in a pair of preset search directions within the same image to be processed, ensure that in each sliding operation during each round of loop traversal, the fixed number of image rows in the same direction is read, reducing the number of times the image data enters and exits the cache space. Then, by reducing the number of pixel points for search calculation, the image traversal speed is increased. At the same time, a pair of preset search directions with regional representativeness can be set, and the pair of preset search directions corresponding to different rounds of loop traversal are set to be different, taking into account the comprehensiveness of the traversed image area.

[0053] Based on the above embodiments, during the same round of cyclic traversal, when the sliding window frames a specific image region in the image to be processed, at this time, the sliding window may have performed one sliding operation or not started the sliding operation (in the initial position and not starting to translate). To facilitate the expansion of the search calculation to the directly above, directly below, horizontally to the left, horizontally to the right, upper left corner point, lower left corner point, upper right corner point, or lower right corner point of the specific image region, in this embodiment, starting from the pixel point currently covered by the center position of the sliding window, neighboring pixel points are searched within the specific image region currently framed by the sliding window along a first preset search direction, from the adjacent position of the pixel point currently covered by the center position of the sliding window to the corner point of the specific image region in the first preset search direction. At the same time, the similarity of the image information between each searched neighboring pixel point and the center pixel point is obtained. If the position where the searched neighboring pixel point is located and the position where the currently covered center pixel point is located are traversed for the first time, the similarity of the image information between this pair of pixel points is calculated; otherwise, the previously traversed pair of pixel points that have been obtained is directly called to obtain the similarity of the image information between them.

[0054] Similarly, during the same round of cyclic traversal, when the sliding window translates to a specific image region of the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, from the adjacent position of the pixel point currently covered by the center position of the sliding window to the corner point of the specific image region in a second preset search direction. At the same time, the similarity of the image information between each searched neighboring pixel point and the center pixel point is obtained. Therefore, in this embodiment, a pair of preset search directions are the first preset search direction and the second preset search direction, and the first preset search direction and the second preset search direction are opposite. When setting a pair of preset search directions in this embodiment, the pixel points searched in the first preset search direction and the pixel points searched in the second preset search direction during the same round of cyclic traversal are centrosymmetric with respect to the pixel point currently covered by the center position point of the sliding window, taking into account the distribution symmetry and relative fixity of the distribution positions of the pixel points to be searched within the image region covered by the sliding window.

[0055] It should be noted that the distances between each neighboring pixel point sequentially searched in the same preset search direction and the center pixel point are different. The neighboring pixel point closest to the center pixel point is located in the eight-neighborhood of the center pixel point, and the eight-neighborhood, 24-neighborhood, etc. of the center pixel point can be accommodated within the image region currently framed by the sliding window, which is specifically related to the size of the sliding window.

[0056] As an embodiment, during the same round of the loop traversal, whenever the sliding window is translated by a preset step size within the image to be processed, the first preset search direction along which neighboring pixel points are searched within the image region currently framed by the sliding window is the same as the first preset search direction along which neighboring pixel points were searched within the image region previously framed by the sliding window, and the second preset search direction along which neighboring pixel points are searched within the image region currently framed by the sliding window is the same as the second preset search direction along which neighboring pixel points were searched within the image region previously framed by the sliding window; thus, it is determined that the pair of preset search directions along which the image to be processed is traversed in the same round of loop traversal remains the same, that is, the first preset search direction along which each sliding operation is performed in the same round of loop traversal is the same, and the second preset search direction along which each sliding operation is performed in the same round of loop traversal is also the same. In this embodiment, neighboring pixel points with the same orientation are set to represent neighboring pixel points located in the same preset search direction and at equal distances from the corresponding central pixel point. Therefore, the neighboring pixel points with the same orientation searched by the sliding window before and after translating by the preset step size are all within the same row of the image to be processed. Based on the fact that the image to be processed is stored row by row in a specific memory, it is convenient for the CPU to read the pixel points of the image to be processed row by row from the memory, or for the CPU to store the pixel points of the image to be processed row by row into the cache space. Furthermore, the number of cross-row reads is reduced, and the hit rate of the pixel points accessed by the CPU in the cache space is increased.

[0057] Preferably, in each round of loop traversal, an equal number of pixel points are searched along a pair of preset search directions within the image region framed by the sliding window; specifically, in each sliding operation of each round of loop traversal, whenever the sliding window is translated to an image region of the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, the number of pixel points cumulatively searched along the first preset search direction within the image region framed by the sliding window is equal to the number of pixel points cumulatively searched along the second preset search direction within the image region framed by the sliding window. Of course, in some embodiments, in the initial state where the sliding window has not yet started performing image processing of the sliding operation, when the sliding window frames a specific image region within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, the number of pixel points passed through along the first preset search direction within the image region framed by the sliding window is equal to the number of pixel points passed through along the second preset search direction within the image region framed by the sliding window. Thus, it is ensured that the sliding window is fully translated within the image to be processed, which can not only ensure that the region framed by the sliding window is entirely within the image to be processed but also ensure the symmetry of searching for neighboring pixel points along a pair of preset search directions.

[0058] As an embodiment, the preset quantity is 4; then the following implementation manners exist:

[0059] When the first preset search direction is the upper left diagonal direction, the second preset search direction is the lower right diagonal direction; wherein, the upper left diagonal direction starts from the pixel point covered by the central position of the sliding window and points to the upper left corner point of the image area framed by the sliding window; the lower right diagonal direction starts from the pixel point covered by the central position of the sliding window and points to the lower right corner point of the image area framed by the sliding window; wherein, the first preset search direction and the second preset search direction are opposite.

[0060] When the first preset search direction is the lower left diagonal direction, the second preset search direction is the upper right diagonal direction; wherein, the lower left diagonal direction starts from the pixel point covered by the central position of the sliding window and points to the lower left corner point of the image area framed by the sliding window; the upper right diagonal direction starts from the pixel point covered by the central position of the sliding window and points to the upper right corner point of the image area framed by the sliding window; wherein, the first preset search direction and the second preset search direction are opposite.

[0061] When the first preset search direction is the horizontal left direction, the second preset search direction is the horizontal right direction; wherein, the horizontal left direction starts from the pixel point covered by the central position of the sliding window and extends leftward along the horizontal axis direction of the image area framed by the sliding window; the horizontal right direction starts from the pixel point covered by the central position of the sliding window and extends rightward along the horizontal axis direction of the image area framed by the sliding window; wherein, the first preset search direction and the second preset search direction are opposite.

[0062] When the first preset search direction is the vertical upward direction, the second preset search direction is the vertical downward direction; wherein, the vertical upward direction starts from the pixel point covered by the central position of the sliding window and points to, and extends upward along the vertical axis direction of the image area framed by the sliding window; the vertical downward direction starts from the pixel point covered by the central position of the sliding window and extends downward along the vertical axis direction of the image area framed by the sliding window. Wherein, the first preset search direction and the second preset search direction are opposite.

[0063] In summary, this embodiment realizes circularly traversing the image to be processed four times along mutually symmetric preset search directions, respectively traversing and calculating (including processing sequentially or simultaneously) the pixel pairs in the horizontal left - right direction, the pixel pairs in the upper left - lower right diagonal direction, the pixel pairs in the vertical up - down direction, and the pixel pairs in the lower left - upper right diagonal direction, reducing the calculation amount of the pixel points under a single sliding operation and improving the processing speed of the image.

[0064] Preferably, the sliding window is represented by a square window, and the side length of the square window is 2×W + 1, where W is a positive integer. When the center position of the sliding window slides within the image to be processed, the pixel points covered thereby become the central pixel points, and after the sliding window is translated by a preset step length each time, at most 2W neighboring pixel points can be framed in a pair of preset search directions. Correspondingly, when the first preset search direction is the upper left diagonal direction, the second preset search direction is the lower right diagonal direction, the acute angle formed by the upper left diagonal direction and the horizontal direction is 45 degrees, and the acute angle formed by the lower right diagonal direction and the horizontal direction is 45 degrees; when the first preset search direction is the lower left diagonal direction, the second preset search direction is the upper right diagonal direction, the acute angle formed by the lower left diagonal direction and the horizontal direction is 45 degrees, and the acute angle formed by the upper right diagonal direction and the horizontal direction is 45 degrees.

[0065] As an embodiment, the method for obtaining the similarity of image information between the neighboring pixel points searched in a corresponding preset search direction and the central pixel point includes: whenever the sliding window translates a preset step length within the image to be processed, starting from the pixel point currently covered by the central position of the sliding window, search for neighboring pixel points within the image area currently framed by the sliding window along a pair of preset search directions, and calculate the similarity of image information between the currently searched neighboring pixel point and the central pixel point without repetition; wherein, the similarity of image information includes gray value similarity and spatial proximity. The non-repetitive calculation method involved in this embodiment is the non-repetitive calculation of the positions of two pixel points, specifically, the similarity of image information between the positions of these two pixel points is not calculated repeatedly; in this embodiment, after the sliding window makes a translation within the image to be processed, within the image area currently framed by the sliding window, starting from the pixel point currently covered by the central position of the sliding window, search for neighboring pixel points in sequence along the preset search direction, and, on the premise of not repeating the calculation of a pair of pixel points (a neighboring pixel point and its corresponding central pixel point), calculate the similarity of image information between the currently searched neighboring pixel point and the central pixel point, and then store the calculated similarity of image information, so that when a searched neighboring pixel point is the original central pixel point (when repeating to traverse a pair of pixel points at a specific relative position), directly call the previously stored corresponding similarity of image information as the similarity of image information between the currently searched neighboring pixel point and the central pixel point; it should be noted that the translation direction of the sliding window remains unchanged before wrapping; the search direction for searching neighboring pixel points starting from the central position of the sliding window is the aforementioned pair of preset search directions, including the first preset search direction and the second preset search direction. Preferably, the pixel point covered by the central position of the sliding window is configured as the central pixel point; the neighboring pixel points are the pixel points located within the neighborhood of the central pixel point, including but not limited to the eight-neighborhood and 24-neighborhood, which are specifically related to the size of the sliding window; wherein, the similarity of image information includes the gray value similarity required for the bilateral filtering algorithm and the spatial proximity, both of which are used as the weight domains required for the bilateral filtering algorithm; the gray value similarity corresponds to the intensity difference in the pixel domain required for performing the bilateral filtering algorithm, sobel filtering, or Gaussian filtering, and the spatial proximity corresponds to the geometric spatial position difference in the spatial domain required for performing the bilateral filtering algorithm, sobel filtering, or Gaussian filtering.

[0066] As an embodiment, the method for non-repetitively calculating the similarity of image information between the currently searched neighboring pixel point and the central pixel point includes:

[0067] If the pixel point covered by the center position of the sliding window in the current sliding operation is updated from a neighboring pixel point searched along a pair of preset search directions in a previously executed sliding operation, that is, when it is detected that the pixel point currently covered by the center position of the sliding window is a neighboring pixel point searched in a previously executed sliding operation, then the pixel point covered by the center position of the sliding window in the current sliding operation is set as the first reference center pixel point. At the same time, it is determined that the pixel point covered by the center position of the sliding window in the previously executed sliding operation belongs to a neighboring pixel point that can be searched along a pair of preset search directions within the image area currently framed by the sliding window, and the pixel point covered by the center position of the sliding window in the previously executed sliding operation is set as the second reference center pixel point. Thus, a pair of pixel points that can be repeatedly traversed is formed, that is, the first reference center pixel point and the second reference center pixel point determined in the current sliding operation. Among them, in the previously executed sliding operation, the gray value similarity between the first reference center pixel point and the second reference center pixel point has been calculated.

[0068] In some embodiments, if the upper left corner point of the image area framed by the sliding window in the previous time is the first reference center pixel point, then the lower right corner point of the image area currently framed by the sliding window can be the lower right pixel point of the first reference center pixel point. Based on this, traverse along the preset search direction. If the lower right pixel point of the first reference center pixel point is the center pixel point of the image area framed by the sliding window in the previous time, it is recorded as the second reference center pixel point. Then the upper left corner point of the image area framed by the sliding window in the previous time is the upper left pixel point of the second reference center pixel point. In summary, the first reference center pixel point and the second reference center pixel point form a pair of pixel points with an upper left - lower right position relationship and support being repeatedly traversed to.

[0069] Then, if a neighboring pixel point searched along a pair of preset search directions within the image region currently framed by the sliding window is obtained by updating the central pixel point covered by the central position of the sliding window during a previously performed sliding operation, it is determined that the currently searched neighboring pixel point is the pixel point that was first covered by the central position of the sliding window during a previously performed sliding operation, that is, the second reference central pixel point. Then, the gray value similarity between the aforementioned first reference central pixel point and the second reference central pixel point is directly updated to the gray value similarity between the currently searched neighboring pixel point and the pixel point covered by the central position of the sliding window during the current sliding operation, without performing another calculation of the gray value similarity, thus avoiding repeating the calculation of the gray value similarity between the aforementioned first reference central pixel point and the second reference central pixel point. Consequently, it is determined that the corresponding gray value similarity already obtained with the previously covered central pixel point as the central pixel point is updated to the gray value similarity between the previously covered central pixel point and the pixel point currently covered by the central position of the sliding window. Among them, the gray value similarity between the aforementioned first reference central pixel point and the second reference central pixel point was calculated during a previously performed sliding operation and is the gray value similarity between the corresponding pixel point pair obtained most recently. Among them, the central pixel point covered by the central position of the sliding window during a previously performed sliding operation is the central pixel point previously covered by the central position of the sliding window. Among them, during the translation of the sliding window within the image to be processed, it is configured not to allow the repeated framing of the same specific image region, and the size of this specific image region is equal to the size of the sliding window. Preferably, the translation direction of the sliding window within the image to be processed is configured not to allow the repeated framing of the same specific image region, and the size of this specific image region is equal to the size of the sliding window.

[0070] In some embodiments, a 3x3 sliding window (a window with a side length of 3 pixel points) is used to perform a sliding operation on the image to be processed. Initially, pixel point A and pixel point B are in adjacent columns within the sliding window. If pixel point A is located at the second row and second column of the image to be processed and pixel point B is located at the third row and third column of the image to be processed, that is, pixel point B is located at the lower right corner point of the 3x3 sliding window, and pixel point A is located at the pixel center point (the pixel position at the second row and second column), and pixel point A and pixel point B are distributed along the lower right diagonal direction in adjacent rows. Then, after traversing the neighboring pixel points in the first column within the sliding window along the upper left diagonal direction and obtaining the corresponding image information similarity, and traversing the neighboring pixel points in the third column within the sliding window along the lower right diagonal direction and obtaining the corresponding image information similarity, the 3x3 sliding window is controlled to be translated once according to a preset step size.

[0071] When the 3x3 sliding window frames the second row to the fourth row of the image to be processed and the second column to the fourth column of the image to be processed, a new sliding operation starts. At this time, there is a partially overlapping image area between the image area currently framed by the 3x3 sliding window and the image area framed by the previous sliding operation. Among them, the partially overlapping image area includes pixel point A and pixel point B, which are located at the first row and first column and the second row and second column of the 3x3 sliding window respectively. Then, the similarity of the image information between pixel point A and pixel point B has been calculated in the previous sliding operation and does not need to be recalculated in the current sliding operation. Instead, the previous calculation result is directly used, especially the similarity of the gray values between pixel point A and pixel point B. The spatial proximity between pixel point A and pixel point B is determined by calculating all the relative position relationships between the central position and the adjacent pixel positions within the 3x3 sliding window before the sliding operation starts (understood as before the bilateral filtering operation starts) and will not change as the 3x3 sliding window translates in the image to be processed. Among them, pixel point A can be denoted as the aforementioned first reference central pixel point, and pixel point B can be denoted as the aforementioned second reference central pixel point. Therefore, in some embodiments, within an image area framed by a 3x3 sliding window (3x3 bilateral filtering kernel), only one calculation of the gray value similarity is required for a pair of 2 pixel pairs in a pair of preset search directions, and the other calculation of the gray value similarity has been calculated beforehand. It should be noted that the aforementioned 3x3 means 3 multiplied by 3, and the 3x3 sliding window represents a sliding window with 3 rows and 3 columns.

[0072] In summary, for a pair of pixel points that may be repeatedly traversed and need to participate in the calculation of the gray value similarity, controlling only one calculation of the gray value similarity between this type of pixel point pairs can configure the gray value similarity calculated when this pair of pixel points is first traversed as the gray value similarity required when this pair of pixel points is repeatedly traversed later. Thus, compared with the existing repeated calculation phenomenon, the number of calculations is reduced by half, and the calculation speed of all pixel points of the image to be processed is increased.

[0073] Based on any of the foregoing embodiments, the method for calculating the gray - scale value similarity between a neighboring pixel point and a corresponding central pixel point is as follows: calculate the square of the difference between the gray - scale value of the neighboring pixel point and the gray - scale value of a corresponding central pixel point to obtain the squared data of the gray - scale change value; wherein, the neighboring pixel point is located within the image area framed by a sliding window centered on a corresponding central pixel point, and the neighboring pixel points participating in the gray - scale value similarity calculation are symmetrically arranged around the central pixel point in the left - right, up - down directions within the image to be processed; then, take the ratio of the squared data of the gray - scale change value to twice the square of the pixel - domain parameter, and use the obtained ratio as a parameter to be input into a pre - configured exponential function, and then update the exponential function value calculated by the CPU as the gray - scale value similarity between a neighboring pixel point and a corresponding central pixel point; wherein, the pixel - domain parameter belongs to the Gaussian distribution parameter, which is used to limit the change range of the gray - scale value similarity and determine the difference between pixel values; the coefficient of the exponential part of the pre - configured exponential function is with a negative sign to make the exponential part a decreasing function, and the exponential part is used to accept the independent variable, and the independent variable comes from the ratio of the squared data of the gray - scale change value to twice the square of the pixel - domain parameter. Therefore, this embodiment uses the square of the difference between the gray - scale value of a neighboring pixel point and the gray - scale value of a corresponding central pixel point within the image area framed by the sliding window to describe the gray - scale value similarity between the neighboring pixel point and the corresponding central pixel point.

[0074] As an embodiment, the method for calculating the image information similarity between the currently searched neighboring pixel point and the central pixel point without repetition includes:

[0075] In each round of the loop traversal of the image to be processed, when the sliding window has not started to translate within the image to be processed, that is, when using the sliding window to process the initial state of the image to be processed, within the initial image area framed by the sliding window within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within this initial image area along a pair of preset search directions, calculate the spatial proximity between the successively searched neighboring pixel points and the same central pixel point, and store the calculated spatial proximities between the neighboring pixel points at different pixel positions within this initial image area and the same central pixel point into a preset memory space, so as to facilitate directly calling the corresponding spatial proximities when the sliding window subsequently performs a sliding operation within the image to be processed; at this time, through searching and calculation, the spatial proximities between the neighboring pixel points and the central pixel point along a pair of preset search directions within the initial image area framed by the sliding window within the image to be processed can be determined, and they belong to a fixed relative position relationship; since before the sliding window starts to translate within the image to be processed (which can be before the first sliding operation) and in each sliding operation performed according to the preset step size, the spatial proximities between the neighboring pixel points with the same relative position relationship and the corresponding central pixel point within the image area framed by the sliding window in real time within the image to be processed are equal and are fixed values, therefore, in each round of the loop traversal, the spatial proximities between the neighboring pixel points and the central pixel point only need to be calculated and saved before the sliding window starts to translate, that is, the spatial proximities between the neighboring pixel points searched along a pair of preset search directions and the corresponding central pixel point within the initial image area framed by the sliding window within the image to be processed, so that when the sliding window subsequently translates within the image to be processed, for the relative position relationship between the neighboring pixel points traversed in real time and the pixel point currently covered by the center position of the sliding window, the previously calculated spatial proximities at the corresponding position relationship can be called, avoiding repeated calculation of the spatial proximities.

[0076] In the above embodiments, the method for calculating the spatial proximity between a neighboring pixel point and a corresponding central pixel point is as follows: calculate the square of the difference between the abscissa of the neighboring pixel point and the abscissa of the corresponding central pixel point to obtain the squared horizontal pixel distance; wherein, the neighboring pixel point is located within the image area framed by a sliding window centered on the corresponding central pixel point, and the neighboring pixel points participating in the calculation of the spatial proximity are arranged symmetrically around the central pixel point in the left, right, up, and down directions within the aforementioned initial image area; calculate the square of the difference between the ordinate of the neighboring pixel point and the ordinate of the corresponding central pixel point to obtain the squared vertical pixel distance; then, take the ratio of the sum of the squared horizontal pixel distance and the squared vertical pixel distance to twice the square of the spatial domain parameter, and use the obtained ratio as a parameter to be input into a pre-configured exponential function. Next, set the exponential function value calculated by the CPU as the spatial proximity between a neighboring pixel point and a corresponding central pixel point; wherein, the spatial domain parameter belongs to the spatial domain parameter in the Euclidean distance formula, which is used to limit the change range of the spatial proximity and determines the change of the spatial distance. The coefficient of the exponential part of the pre-configured exponential function is negative to make the exponential part a decreasing function. The exponential part is used to accept the independent variable, and the independent variable is derived from the ratio of the sum of the squared horizontal pixel distance and the squared vertical pixel distance to twice the square of the spatial domain parameter. Therefore, in this embodiment, the squared distance between a neighboring pixel point and a corresponding central pixel point within the image area framed by the sliding window is used to describe the spatial proximity between the neighboring pixel point and the corresponding central pixel point.

[0077] Preferably, the shape of the sliding window is the same as the shape of the initial image area, and the area of the sliding window is equal to the area of the initial image area, so as to obtain a more complete spatial proximity between the pixel position and the corresponding central pixel point. The obtained spatial proximity is more representative within the image to be processed and fully describes the positional relationship between each neighboring pixel point in various neighborhoods and a central pixel point.

[0078] In one embodiment, when performing four rounds of the loop traversal on the image to be processed, the first round of loop traversal may be: the first preset search direction is the upper left diagonal direction, and the second preset search direction is the lower right diagonal direction; wherein, the upper left diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the upper left corner point of the image area framed by the sliding window; the position relationship of each neighboring pixel point searched along the upper left diagonal direction with respect to the center pixel point is different from each other during the same sliding operation; the lower right diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the lower right corner point of the image area framed by the sliding window; the position relationship of each neighboring pixel point searched along the lower right diagonal direction with respect to the center pixel point is different from each other during the same sliding operation and is symmetric with respect to the upper left diagonal direction; wherein, the first preset search direction and the second preset search direction are opposite to each other.

[0079] Then, the second round of loop traversal may be: the first preset search direction is the lower left diagonal direction, and the second preset search direction is the upper right diagonal direction; wherein, the lower left diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the lower left corner point of the image area framed by the sliding window; the position relationship of each neighboring pixel point searched along the lower left diagonal direction with respect to the center pixel point is different from each other during the same sliding operation; the upper right diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the upper right corner point of the image area framed by the sliding window; the position relationship of each neighboring pixel point searched along the upper right diagonal direction with respect to the center pixel point is different from each other during the same sliding operation and is symmetric with respect to the lower left diagonal direction.

[0080] Then, the third round of loop traversal may be: the first preset search direction is the horizontal left direction, and the second preset search direction is the horizontal right direction; wherein, the horizontal left direction starts from the pixel point covered by the center position of the sliding window and extends leftward along the horizontal axis direction of the image area framed by the sliding window; the position relationship of each neighboring pixel point searched along the direction of extending leftward along the horizontal axis direction with respect to the center pixel point is different from each other during the same sliding operation; the horizontal right direction starts from the pixel point covered by the center position of the sliding window and extends rightward along the horizontal axis direction of the image area framed by the sliding window; the position relationship of each neighboring pixel point searched along the direction of extending rightward along the horizontal axis direction with respect to the center pixel point is different from each other during the same sliding operation and is symmetric with respect to the horizontal left direction.

[0081] Then, the fourth round of loop traversal can be: the first preset search direction is the vertically upward direction, and the second preset search direction is the vertically downward direction; wherein, the vertically upward direction starts from the pixel points covered by the center position of the sliding window and extends upward along the longitudinal axis direction of the image area delimited by the sliding window; the positional relationships of each neighboring pixel point searched in the upward extension direction along this longitudinal axis direction with respect to the center pixel point are different from each other in the same sliding operation; the vertically downward direction starts from the pixel points covered by the center position of the sliding window and extends downward along the longitudinal axis direction of the image area delimited by the sliding window; the positional relationships of each neighboring pixel point searched in the downward extension direction along this longitudinal axis direction with respect to the center pixel point are different from each other in the same sliding operation and are symmetric with respect to the vertically upward direction.

[0082] Among the above four rounds of loop traversals, the positional relationships of each neighboring pixel point searched in different rounds of loop traversals with respect to the center pixel point are different from each other, and the spatial proximity degrees between a batch of neighboring pixel points searched and the corresponding center pixel points are different when the preset search directions are asymmetric; in each sliding operation within the same round of loop traversal, the relative positional relationships of each neighboring pixel point searched with respect to the corresponding center pixel point are the same. Therefore, it is only necessary to calculate the spatial proximity degrees between the neighboring pixel points searched in the first sliding operation of each round of loop traversal and the corresponding center pixel points.

[0083] As an embodiment, for the N corresponding neighborhood pixel points traversed in a sliding operation performed on the sliding window, the CPU is controlled to call N computing registers to calculate in parallel the similarity of the image information between the N neighborhood pixel points traversed and the same central pixel point in this sliding operation, and further calculate in parallel the product of the gray value similarity and the spatial neighborhood degree between the N corresponding neighborhood pixel points searched and a central pixel point in this sliding operation; preferably, the processor used in this embodiment has a multi-core (e.g., 8-core or 16-core) structure. Each core is a complete computing register, and multiple computing registers perform data calculations in parallel under the control of the Neon instruction set. Among them, each computing register is used to calculate the similarity of the image information between a neighborhood pixel point and the same central pixel point in a corresponding sliding operation, and further calculate the product of the gray value similarity and the spatial neighborhood degree between a neighborhood pixel point and the central pixel point in the corresponding sliding operation; independent calculation operations of the similarity of the image information or its corresponding product are supported between different computing registers; where N is a positive integer greater than or equal to 2, which is determined by a predefined search quantity and is associated with the requirement for the number of pixel points in the convolution filtering. This embodiment uses the parallel computing method to improve the calculation speed of the similarity of the image information corresponding to multiple neighborhood pixel points, and further improve the calculation speed of the gray value similarity and the spatial neighborhood degree between multiple neighborhood pixel points and a corresponding central pixel point, which can be used to accelerate the filtering processing speed of the image.

[0084] Preferably, the N corresponding neighborhood pixel points traversed in a sliding operation performed on the sliding window are the neighborhood pixel points searched from the central position of the sliding window along the first preset search direction and / or the second preset search direction within the image area framed by the sliding window. This can accelerate the operation speed of a sliding operation, and further improve the speed of a round of cyclic traversal of the image to be processed.

[0085] Based on the foregoing embodiment, a chip is also disclosed. This chip is used to store the program code corresponding to the image processing method. When the image processing method is executed, it is used to reduce the number of times the pixel points of the image to be processed enter and exit the cache space, where the cache space is set between the CPU and the memory for storing the image to be processed.

[0086] Especially when more data than the current demand is pre-stored in the cache space, pre-reading some data into the cache space can reduce the access to the storage medium, so that the CPU reads only two fixed rows from the cache space, which is more suitable for calculating the difference between the pixel points in the adjacent upper and lower rows, thereby improving the reading speed and avoiding problems caused by crossing rows.

[0087] When the CPU adopts a multi-core structure (for example, 8 cores or 16 cores), each core of the CPU is triggered to perform data processing in parallel; it is also possible to trigger the CPU to independently read and write each memory in the order of Cache first and then memory for different memories independently connected thereto.

[0088] The directional words such as "up (front)", "down (back)", "left", and "right" mentioned in the above embodiments refer to the up, down, left, and right directions of the drawings, and the vertical and horizontal directions refer to the vertical direction and the horizontal direction of the drawings, unless otherwise specified.

[0089] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. These programs can be stored in a computer-readable storage medium (such as various media that can store program codes, such as ROM, RAM, magnetic disks, or optical discs). When the program is executed, it performs the steps of the above method embodiments. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method based on a fixed direction, characterized in that, The image processing method includes: Before the sliding window starts the first translation in the image to be processed, and whenever the sliding window makes a translation in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points in the image area currently framed by the sliding window along a pair of preset search directions, and obtain the similarity of image information between the neighboring pixel points searched in the corresponding preset search direction and the center pixel point; Among them, the storage order of each pixel point in the image to be processed in the memory is row-by-row storage; the pixel point covered by the center position of the sliding window is configured as the center pixel point; the neighboring pixel points are the pixel points located in the neighborhood of the center pixel point; The image processing method further includes: Whenever the sliding window covers all the pixel points in the image to be processed, it is determined that the sliding window has completed one round of cyclic traversal of the image to be processed; When the sliding window makes a preset number of rounds of cyclic traversal of the image to be processed, the similarity of image information between the neighboring pixel points in all preset search directions and the matching center pixel points is obtained, where the preset search directions along which the image to be processed is traversed in different rounds of cyclic traversal are different, and each pair of preset search directions along which the traversal is made in the same round of cyclic traversal remains the same.

2. The image processing method according to claim 1, wherein During the process of the same round of cyclic traversal, when the sliding window frames a specific image area in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points in the specific image area currently framed by the sliding window along the first preset search direction, and obtain the similarity of image information between each searched neighboring pixel point and the center pixel point; During the process of the same round of cyclic traversal, when the sliding window frames a specific image area in the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points in the image area currently framed by the sliding window along the second preset search direction, and obtain the similarity of image information between each searched neighboring pixel point and the center pixel point; During the process of the same round of cyclic traversal, a pair of preset search directions are the first preset search direction and the second preset search direction, and the first preset search direction and the second preset search direction are opposite; Among them, the distances between each neighboring pixel point searched in sequence in the same direction of the preset search direction and the center pixel point are different, and the neighboring pixel point closest to the center pixel point is located in the octagon neighborhood of the center pixel point.

3. The image processing method according to claim 2, wherein During the process of the same round of cyclic traversal, whenever the sliding window translates a preset step length in the image to be processed, the first preset search direction for searching neighboring pixel points in the image area currently framed by the sliding window is the same as the first preset search direction for searching neighboring pixel points in the image area framed by the sliding window last time, and the second preset search direction for searching neighboring pixel points in the image area currently framed by the sliding window is the same as the second preset search direction for searching neighboring pixel points in the image area framed by the sliding window last time.

4. The image processing method according to claim 3, wherein During each round of loop traversal, an equal number of pixel points are searched along a pair of preset search directions within the image area framed by the sliding window. Among them, whenever the sliding window is translated to an image area of the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, the number of pixel points searched along the first preset search direction within the image area framed by the sliding window is equal to the number of pixel points searched along the second preset search direction within the image area framed by the sliding window.

5. The image processing method according to claim 2, wherein The preset number is 4; When the first preset search direction is the upper left diagonal direction, the second preset search direction is the lower right diagonal direction; among them, the upper left diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the upper left corner point of the image area framed by the sliding window; the lower right diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the lower right corner point of the image area framed by the sliding window; When the first preset search direction is the lower left diagonal direction, the second preset search direction is the upper right diagonal direction; among them, the lower left diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the lower left corner point of the image area framed by the sliding window; the upper right diagonal direction starts from the pixel point covered by the center position of the sliding window and points to the upper right corner point of the image area framed by the sliding window; When the first preset search direction is the horizontal left direction, the second preset search direction is the horizontal right direction; among them, the horizontal left direction starts from the pixel point covered by the center position of the sliding window and extends left along the horizontal axis direction of the image area framed by the sliding window; the horizontal right direction starts from the pixel point covered by the center position of the sliding window and extends right along the horizontal axis direction of the image area framed by the sliding window; When the first preset search direction is the vertical upward direction, the second preset search direction is the vertical downward direction; among them, the vertical upward direction starts from the pixel point covered by the center position of the sliding window and extends upward along the vertical axis direction of the image area framed by the sliding window; the vertical downward direction starts from the pixel point covered by the center position of the sliding window and extends downward along the vertical axis direction of the image area framed by the sliding window.

6. The image processing method according to claim 4, wherein The sliding window is represented by a square window, and the side length of the square window is 2×W + 1, where W is a positive integer.

7. The image processing method according to claim 3 or 6, characterized in that Before the sliding window starts the first translation in the image to be processed, and whenever the sliding window is translated by a preset step length within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighborhood pixel points along a pair of preset search directions within the image area currently framed by the sliding window, and obtain the similarity of the image information between the neighborhood pixel points searched in the corresponding preset search direction and the center pixel point, and determine that the sliding window performs a sliding operation on the image to be processed. When all the image regions framed by the sliding window have covered all the pixel points in the image to be processed, and all the similarity degrees of the image information between the neighborhood pixel points and the corresponding central pixel points in the preset search direction in all the image regions framed by the sliding window are obtained, it is determined that the sliding window has completed one round of cyclic traversal of the image to be processed; Among them, each pair of preset search directions along which all the sliding operations in the same round of cyclic traversal are the same.

8. The image processing method according to claim 2, wherein The method for obtaining the similarity degree of the image information between the neighborhood pixel points searched in the corresponding preset search direction and the central pixel points includes: When the sliding window translates a preset step length in the image to be processed, starting from the pixel point currently covered by the central position of the sliding window, search for neighborhood pixel points in the image region currently framed by the sliding window along a pair of preset search directions, and calculate the similarity degree of the image information between the currently searched neighborhood pixel points and the central pixel points without repetition; Among them, the similarity degree of the image information includes the similarity degree of the gray value and the spatial proximity degree; the similarity degree of the gray value is used to represent the degree of difference between the gray values of two pixel points; the spatial proximity degree is used to represent the degree of difference between the spatial distances between two pixel points.

9. The image processing method according to claim 8, wherein The method for calculating the similarity degree of the image information between the currently searched neighborhood pixel points and the central pixel points without repetition includes: If the pixel point covered by the central position of the sliding window in the current sliding operation is updated from a neighborhood pixel point searched along a pair of preset search directions in a previously executed sliding operation, the pixel point covered by the central position of the sliding window in the current sliding operation is set as the first reference central pixel point, and at the same time, it is determined that the pixel point covered by the central position of the sliding window in the previously executed sliding operation belongs to a neighborhood pixel point that can be searched along a pair of preset search directions in the image region currently framed by the sliding window, and the pixel point covered by the central position of the sliding window in the previously executed sliding operation is set as the second reference central pixel point; among them, in the previously executed sliding operation, the similarity degree of the gray value between the first reference central pixel point and the second reference central pixel point has been calculated; Then, if a neighborhood pixel point searched along a pair of preset search directions in the image region currently framed by the sliding window is updated from the central pixel point covered by the central position of the sliding window in the previously executed sliding operation, the similarity degree of the gray value between the aforementioned first reference central pixel point and the second reference central pixel point is directly updated to the similarity degree of the gray value between the currently searched neighborhood pixel point and the pixel point covered by the central position of the sliding window in the current sliding operation.

10. The image processing method according to claim 8, wherein The method for calculating the similarity degree of the image information between the currently searched neighborhood pixel points and the central pixel points without repetition includes: In each round of the loop traversal of the image to be processed, when the sliding window has not started to translate within the image to be processed, within the initial image region framed by the sliding window within the image to be processed, starting from the pixel point currently covered by the center position of the sliding window, search for neighboring pixel points within this initial image region along a pair of preset search directions, calculate the spatial proximity between the successively searched neighboring pixel points and the same central pixel point, and store the spatial proximities between the neighboring pixel points at different pixel positions within this initial image region and the same central pixel point into a preset memory space, so as to directly call the corresponding spatial proximities when the sliding window subsequently performs a sliding operation within the image to be processed; Before the sliding window starts to translate within the image to be processed and in each sliding operation performed according to the preset step size, the spatial proximities between the neighboring pixel points with the same relative position relationship and the corresponding central pixel point within the image region framed by the sliding window within the image to be processed are equal.

11. The image processing method according to claim 8, wherein For the N corresponding neighboring pixel points traversed in a sliding operation performed by the sliding window, control the CPU to call N computing registers to calculate in parallel the image information similarities between the N traversed neighboring pixel points and the same central pixel point in this sliding operation; Among them, each computing register is used to calculate the image information similarity between a neighboring pixel point and the same central pixel point in a corresponding sliding operation; Among them, N is a positive integer greater than or equal to 2.

12. The image processing method according to claim 11, wherein The N corresponding neighboring pixel points traversed in a sliding operation performed by the sliding window are the neighboring pixel points searched within the image region framed by the sliding window, starting from the center position of the sliding window, along the first preset search direction and / or the second preset search direction.

13. A chip, characterized in that, This chip is used to store the program code corresponding to the image processing method according to any one of claims 1 to 12, and is used to reduce the number of times the pixel points of the image to be processed enter and exit the cache space when executing the image processing method, where the cache space is set between the CPU and the memory for storing the image to be processed.

Images