Accelerated method for non-local means filtering denoising

Abstract

The application discloses a non-local mean filtering denoising acceleration method. The non-local mean filtering denoising acceleration method comprises the following steps: step 1, obtaining image information to be processed; step 2, pre-filtering the image information to be processed by using an NLM method in a kernel parallel acceleration mode to obtain pre-filtered data; step 3, performing feature construction conversion on the pre-filtered data by using the NLM method in the kernel parallel acceleration mode to obtain feature construction converted data; and step 4, processing the feature construction converted data by using the NLM method in the kernel parallel acceleration mode to obtain final denoised data. In the image denoising process, the application performs kernel parallel acceleration on hot spot calculation parts of pre-filtering, feature construction conversion and denoising reconstruction, effectively improves the parallel efficiency of the NLM algorithm, accelerates the denoising calculation process of the NLM algorithm, and effectively shortens the denoising time of the image without reducing the denoising effect.

Description

Technical Field

[0001] The present application relates to the technical field of image filtering and denoising, and in particular to an acceleration method for non-local mean filtering and denoising. Background Art

[0002] During the acquisition and transmission process, images are inevitably interfered with and a lot of noise is added, which seriously affects people's reading of the image information after transmission. Therefore, denoising images contaminated by noise through certain methods has always been an important issue in modern image research, and has attracted the attention and attention of more and more researchers. After studying the basis of classic denoising algorithms such as anisotropic diffusion, neighborhood filtering and total variation filtering, Buades et al. proposed an algorithm model of non-local means (NLM) and applied it to image denoising, achieving good denoising effect. NLM is one of the most widely studied image denoising methods so far. Its basic idea is to use a large amount of redundant information in the image, and calculate the average weighted grayscale values ​​of all pixels in the approximate neighborhood of the image according to the weight coefficient to obtain the grayscale estimate value of the noisy image pixel. The principle of NLM is as follows:

[0003] Given a discrete noisy image u = {u(i)|i∈I}, I∈N 2 For a certain pixel point i in the image, we can use the NLM algorithm to estimate the value of point i by taking the weighted average of all pixels in the image.

[0004]

[0005] The weight w(i,j) depends on the similarity between pixel i and pixel j and satisfies the following conditions:

[0006]

[0007] The similarity between pixels i and j is determined by their neighborhood N i and N j Theoretically, the neighborhoods of different pixels can have different sizes and shapes to better adapt to the image structure of the area they are in, but in practice, for simplicity and convenience of calculation, the neighborhood of a pixel is limited to a square shape with a fixed size. i , its gray value vector is defined as follows:

[0008] v(N i )={v(j)|j∈N i}

[0009] Neighborhood N i and Nj The similarity between them is expressed by their gray value vector v(N i ) and v(N j ) is measured using the Gaussian weighted Euclidean distance:

[0010]

[0011] Where a is the standard deviation of the Gaussian kernel, ||·|| 2,a Indicates l that has undergone Gaussian convolution 2 The reason why Gaussian weighting is performed when calculating the Euclidean distance of the neighborhood grayscale value vector is mainly to increase the proportion of the neighborhood center pixel in the entire similarity measurement, because the neighborhood center pixel is the one that ultimately participates in the weighted average denoising. i and N j The more similar they are, the gray value vector v(N i ) and v(N j ), the smaller the Gaussian weighted Euclidean distance d(i,j), the greater the weight value of pixel j.

[0012] Based on the similarity between neighbors, the weight value of the pixel is calculated as follows:

[0013]

[0014] Among them, h is the filtering parameter, which controls the attenuation of the weight function and determines the smoothness of the image after denoising. Z(i) is the normalization factor:

[0015]

[0016] In theory, the NLM method searches the entire image, but this results in excessively high algorithmic complexity. In practical denoising, a search window is often used to narrow the search range, reducing algorithmic complexity. The non-local means denoising method can effectively remove noise from an image, but the calculation of each pixel's value requires first calculating the similarity between the current pixel and other pixels in the image space, making the algorithm computationally very complex. To better utilize the NLM algorithm, it must be parallelized and accelerated.

[0017] (2) Supercomputer parallel computing

[0018] Supercomputers are an important indicator of a country's scientific and technological development level and comprehensive national strength. They are characterized by powerful functions, fast computing speed, and large storage capacity. They support the calculation of large and complex application topics and are mostly used in national high-tech fields and cutting-edge technology research. "Sunway TaihuLight" is the world's first supercomputer with a performance exceeding 100PFlops, and it is also my country's first supercomputer with completely independent intellectual property rights. The "Sunway TaihuLight" supercomputer uses the domestically produced SW26010 heterogeneous many-core processor, which adopts a heterogeneous many-core architecture that combines on-chip array clusters and distributed shared storage. Figure 2 As shown, a processor consists of four core groups, each consisting of a master core, an 8x8 slave core array, and a memory controller. The core groups of the SW26010 processor communicate via an on-chip network and can connect to external devices through system interfaces. A single SW26010 processor has a peak computing power of 3 TFlops and a memory access bandwidth of 130 GB / s. The Sunway TaihuLight supercomputer incorporates a total of 40,960 SW26010 heterogeneous many-core processors, enabling parallel computing with tens of millions of cores.

[0019] The SW26010 processor's memory consists of main memory and LDM local memory. Each core group has 8GB of main memory, using virtual addresses and divided into continuous and interleaved segments. The continuous segment is private to each core group and is continuously addressed within the core group. Both the master and slave cores can directly access the continuous segment of their own core group. The interleaved segment is shared between core groups on the same chip, and both the master and slave cores can access this distributed, shared area across the entire chip. The LDM local memory is a high-speed, local data storage space for each slave core. It is only 128KB in size but offers fast access speeds. The LDM is divided into a private segment, a continuous shared segment, and a data cache segment. The LDM private segment is a local, private space that is quickly accessed by the slave core. The LDM continuous shared segment is shared access within the slave array, and the LDM data cache segment is used for data cache access. Furthermore, slave cores can initiate DMA operations to transfer data between the LDM and main memory, improving the efficiency of slave core access to main memory.

[0020] The NLM algorithm can effectively remove noise from images. However, in the process of calculating each pixel valuation, it is necessary to first calculate the similarity between other pixels in the image space and the current pixel, which makes the computational complexity of the algorithm very high. In order to better apply the NLM algorithm to image denoising, the NLM algorithm must be parallelized and accelerated.

[0021] The existing NLM parallelization acceleration methods have the following problems:

[0022] (1) The algorithm complexity is reduced. Although the algorithm's computational processing speed is improved, the algorithm's denoising effect on the image is also reduced.

[0023] (2) The time-consuming processes such as feature extraction and structural transformation in the denoising process are not taken into account, and the parallel efficiency needs to be improved;

[0024] Therefore, it is hoped that there will be a technical solution to solve or at least alleviate the above-mentioned deficiencies in the prior art. Summary of the Invention

[0025] The object of the present invention is to provide an accelerated method for non-local mean filtering denoising to solve at least one of the above technical problems.

[0026] In one aspect of the present invention, a method for accelerating non-local means filtering denoising is provided, comprising:

[0027] Step 1: Obtain the image information to be processed;

[0028] Step 2: Use the NLM method to pre-filter the image information to be processed in a parallel accelerated manner from the core, thereby obtaining pre-filtered data;

[0029] Step 3: Using the NLM method to perform feature structure transformation on the pre-filtered data in a parallel acceleration manner from the kernel, thereby obtaining feature structure transformed data;

[0030] Step 4: Use the NLM method to process the data after feature construction transformation in a kernel parallel acceleration manner to obtain the final denoised data.

[0031] Optionally, the pre-filtering includes a shadow pre-filtering process, a feature pre-filtering process and a color pre-filtering process;

[0032] The step 2: using the NLM method to pre-filter the image information to be processed in a manner of parallel acceleration from the core, thereby obtaining pre-filtered data includes:

[0033] Step 21: In the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array;

[0034] Step 22: In the feature pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array;

[0035] Step 23: In the color pre-filtering process, parallel calculation of the NLM filtering algorithm is performed on the cumulative calculation part of the image information to be processed from the kernel array.

[0036] Optionally, step 21: in the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array includes:

[0037] Step 211: The main core obtains a set of pixels to be filtered and calculated in the image information to be processed;

[0038] Step 212: The master core transfers the pixel set to be filtered and calculated to the pixel pool via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the slave core array shared space;

[0039] Step 213: Each slave core in the slave core array retrieves the coordinates of the pixel to be filtered from the pixel pool, accesses the main memory, retrieves the pixel data of the area S centered on the pixel coordinates, and then performs NLM filtering.

[0040] Step 214: The pixel point that has completed the NLM filter calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for filter calculation from the pixel pool, and read the pixel data of the size of the main memory area S to perform the NLM filter calculation.

[0041] Step 215: Repeat step 214 until the pixel coordinates in the pixel pool are 0, and then terminate the parallel calculation from the kernel array.

[0042] Optionally, the step 3: performing feature structure conversion on the pre-filtered data using an NLM method in a manner of core parallel acceleration, thereby obtaining the feature structure converted data includes:

[0043] Step 31: The main core obtains a set of coordinates of the filter area to be constructed and transformed in the image information to be processed;

[0044] Step 32: The main core transfers the filter area coordinate set to be constructed and transformed to the filter area pool through DMA;

[0045] Step 33: Obtain corresponding filter area data from each slave core in the core array through the coordinates of the filter area pool to perform construction transformation calculation;

[0046] Step 34: The filter area that has been calculated is transferred back to the master core via DMA. The slave core that has completed the calculation takes the next filter area from the filter area pool for calculation.

[0047] Step 35: Repeat step 34 until there is no more filter area that needs to be calculated in the filter area pool, and then terminate the parallel calculation from the core array.

[0048] Optionally, the step 4: using the NLM method to process the data after feature structure conversion in a kernel parallel acceleration manner to obtain the final denoised data includes:

[0049] Step 41: The main core obtains a set of pixels to be denoised and reconstructed in the image information to be processed;

[0050] Step 42: The master core transfers the set of pixels to be denoised and reconstructed to the pool of pixels to be denoised and reconstructed via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the shared space of the slave core array;

[0051] Step 43: Each slave core in the core array retrieves the coordinates of the pixel to be denoised and reconstructed from the pool of pixels to be denoised and reconstructed, accesses the main memory, retrieves the pixel data of a region S centered on the pixel coordinates, and then performs NLM filtering calculations.

[0052] Step 44: The pixel point that has completed the NLM filtering calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for denoising and reconstruction calculation from the pool of pixels to be reconstructed, and read the pixel data of the size of the main memory area S to perform the NLM filtering calculation.

[0053] Step 45: Repeat step 44 until the coordinate of the pixel point in the pixel pool to be denoised and reconstructed is 0, and then terminate the parallel calculation from the kernel array.

[0054] Optionally, in the shadow pre-filtering process, the NLM filtering calculation process of each pixel point in the single slave kernel is similarity calculation, fuzzy filtering calculation and weight calculation.

[0055] Optionally, the NLM filtering algorithm process executed in a single slave core of the slave core array in the denoising and reconstruction process includes fuzzy filtering calculation, weight calculation and Gramian matrix construction.

[0056] Beneficial effects

[0057] The acceleration method of non-local mean filtering denoising in this application is based on the parallel acceleration advantage of the core array of the Sunway supercomputer. During the image denoising process, the hot spot calculation parts of the pre-filtering, feature structure conversion, denoising and reconstruction processes are accelerated in parallel from the core, which effectively improves the parallel efficiency of the NLM algorithm and accelerates the denoising calculation process of the NLM algorithm. It can effectively shorten the image denoising time without reducing the denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 4 is a flowchart of an acceleration method for non-local mean filtering denoising according to an embodiment of the present application.

[0059] Figure 2This is a schematic diagram of the SW26010 heterogeneous many-core processor architecture according to an embodiment of the present application.

[0060] Figure 3 Schematic diagram of pixel search during execution of the NLM algorithm according to an embodiment of the present application.

[0061] Figure 4 FIG. 1 is a detailed schematic diagram of a parallel NLM filtering and denoising process from a kernel array according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the implementation of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below in conjunction with the drawings in the embodiments of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be understood as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0063] Figure 1 4 is a flowchart of an acceleration method for non-local mean filtering denoising according to an embodiment of the present application.

[0064] like Figure 1 The acceleration methods for non-local means filtering denoising shown include:

[0065] Step 1: Obtain the image information to be processed;

[0066] Step 2: Use the NLM method to pre-filter the image information to be processed in a parallel accelerated manner from the core, thereby obtaining pre-filtered data;

[0067] Step 3: Using the NLM method to perform feature structure transformation on the pre-filtered data in a parallel acceleration manner from the kernel, thereby obtaining feature structure transformed data;

[0068] Step 4: Use the NLM method to process the data after feature construction transformation in a kernel parallel acceleration manner to obtain the final denoised data.

[0069] In this embodiment, the pre-filtering includes a shadow pre-filtering process, a feature pre-filtering process, and a color pre-filtering process;

[0070] The step 2: using the NLM method to pre-filter the image information to be processed in a manner of parallel acceleration from the core, thereby obtaining pre-filtered data includes:

[0071] Step 21: In the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array;

[0072] Step 22: In the feature pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array;

[0073] Step 23: In the color pre-filtering process, parallel calculation of the NLM filtering algorithm is performed on the cumulative calculation part of the image information to be processed from the kernel array.

[0074] In this embodiment, shadow pre-filtering is performed first, then feature pre-filtering is performed, and finally color pre-filtering is performed.

[0075] In this embodiment, step 21: in the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array includes:

[0076] Step 211: The main core obtains a set of pixels to be filtered and calculated in the image information to be processed;

[0077] Step 212: The master core transfers the pixel set to be filtered and calculated to the pixel pool via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the slave core array shared space;

[0078] Step 213: Each slave core in the slave core array retrieves the coordinates of the pixel to be filtered from the pixel pool, accesses the main memory, retrieves the pixel data of the area S centered on the pixel coordinates, and then performs NLM filtering.

[0079] Step 214: The pixel point that has completed the NLM filter calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for filter calculation from the pixel pool, and read the pixel data of the size of the main memory area S to perform the NLM filter calculation.

[0080] Step 215: Repeat step 214 until the pixel coordinates in the pixel pool are 0, and then terminate the parallel calculation from the kernel array.

[0081] In this embodiment, after the shadow pre-filtering process, the feature pre-filtering process is performed. In the feature pre-filtering process, the parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array includes:

[0082] The main core obtains the pixel set to be calculated for feature pre-filtering in the image information to be processed;

[0083] The master core transfers the pixel set to be calculated for feature pre-filtering to the pixel pool through DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the shared space of the slave core array;

[0084] Each slave core in the slave core array takes out the pixel coordinates to be filtered from the pixel pool, accesses the main memory space, takes out the pixel data of the area S with the pixel coordinates as the center, and then performs the NLM filtering calculation;

[0085] The pixel point that completes the NLM filter calculation will transfer the calculation result back to the main core through DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for filter calculation from the pixel pool, and read the pixel data of the size of the main memory area S to perform NLM filter calculation;

[0086] Repeat the above steps until the pixel coordinates in the pixel pool are 0, and then end the parallel calculation from the kernel array.

[0087] In this embodiment, after the feature pre-filtering process, the color pre-filtering process is performed. In the color pre-filtering process, the parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array includes:

[0088] The main core obtains a set of pixels to be color pre-filtered in the image information to be processed;

[0089] The master core transfers the pixel set to be calculated for color pre-filtering to the pixel pool through DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the shared space of the slave core array;

[0090] Each slave core in the slave core array takes out the pixel coordinates to be filtered from the pixel pool, accesses the main memory space, takes out the pixel data of the area S with the pixel coordinates as the center, and then performs the NLM filtering calculation;

[0091] The pixel point that completes the NLM filter calculation will transfer the calculation result back to the main core through DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for filter calculation from the pixel pool, and read the pixel data of the size of the main memory area S to perform NLM filter calculation;

[0092] Repeat the above steps until the pixel coordinates in the pixel pool are 0, and then end the parallel calculation from the kernel array.

[0093] In this embodiment, step 3: performing feature structure conversion on the pre-filtered data using the NLM method in a manner of core parallel acceleration, thereby obtaining the feature structure converted data includes:

[0094] Step 31: The main core obtains a set of coordinates of the filter area to be constructed and transformed in the image information to be processed;

[0095] Step 32: The main core transfers the filter area coordinate set to be constructed and transformed to the filter area pool through DMA;

[0096] Step 33: Obtain corresponding filter area data from each slave core in the core array through the coordinates of the filter area pool to perform construction transformation calculation;

[0097] Step 34: The filter area that has been calculated is transferred back to the master core via DMA. The slave core that has completed the calculation takes the next filter area from the filter area pool for calculation.

[0098] Step 35: Repeat step 34 until there is no more filter area that needs to be calculated in the filter area pool, and then terminate the parallel calculation from the core array.

[0099] In this embodiment, step 4: using the NLM method to process the data after feature structure conversion in a core parallel acceleration manner to obtain the final denoised data includes:

[0100] Step 41: The main core obtains a set of pixels to be denoised and reconstructed in the image information to be processed;

[0101] Step 42: The master core transfers the set of pixels to be denoised and reconstructed to the pool of pixels to be denoised and reconstructed via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the shared space of the slave core array;

[0102] Step 43: Each slave core in the core array retrieves the coordinates of the pixel to be denoised and reconstructed from the pool of pixels to be denoised and reconstructed, accesses the main memory, retrieves the pixel data of a region S centered on the pixel coordinates, and then performs NLM filtering calculations.

[0103] Step 44: The pixel point that has completed the NLM filtering calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for denoising and reconstruction calculation from the pool of pixels to be reconstructed, and read the pixel data of the size of the main memory area S to perform the NLM filtering calculation.

[0104] Step 45: Repeat step 44 until the coordinate of the pixel point in the pixel pool to be denoised and reconstructed is 0, and then terminate the parallel calculation from the kernel array.

[0105] In this embodiment, in the shadow pre-filtering process, the NLM filtering calculation process of each pixel point in the single slave kernel is similarity calculation, fuzzy filtering calculation and weight calculation.

[0106] In this embodiment, the NLM filtering algorithm process executed in a single slave core of the slave core array in the denoising and reconstruction process includes fuzzy filtering calculation, weight calculation and Gramian matrix construction.

[0107] The present application is further described in detail below in the form of examples. It should be understood that the examples do not constitute any limitation to the present application.

[0108] In this embodiment, the application platform of this application is a master-slave heterogeneous system based on the SW26010 processor, in which the master core is responsible for the construction of the image denoising data, and the slave core array takes advantage of parallel computing to perform slave core parallel acceleration on the hotspot calculation parts of pre-filtering, feature construction conversion, and denoising reconstruction in the NLM image denoising process. First, the master core constructs the image information to be processed, sets the denoising data cache, executes the pre-filtering process, and performs slave core array parallel acceleration on the hotspot calculation parts of shadow pre-filtering, feature pre-filtering, and color pre-filtering respectively; then, the master core executes the feature construction conversion process, starts the slave core array in its hotspot calculation part, and performs the calculation of the feature construction conversion of the filter area in parallel; finally, when the master core executes the denoising and reconstruction process, the cumulative calculation part in the reconstruction process is accelerated by the slave core array. By performing slave core array parallel acceleration on the intensive calculation parts of the three serial processes of pre-filtering, feature construction conversion, and denoising reconstruction in the NLM image denoising process, the parallel efficiency can be maximized and the image denoising time can be effectively shortened.

[0109] In this embodiment, step 1: obtaining the image information to be processed; specifically, the main core constructs the image information to be processed, starts the image denoising task process, sets the denoising data cache, etc.;

[0110] Step 2: Use the NLM method to pre-filter the image information to be processed in a parallel accelerated manner from the kernel to obtain the pre-filtered data. Specifically, pre-filtering includes three sub-processes: shadow pre-filtering, feature pre-filtering, and color pre-filtering. The computational hotspots of these three pre-filtering sub-processes are all NLM filtering algorithms. Therefore, when executing each pre-filtering sub-process, parallel calculation of the NLM filtering algorithm is initiated from the kernel array.

[0111] In this embodiment, step 2 includes:

[0112] Step 21: In the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array;

[0113] Step 22: In the feature pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array;

[0114] Step 23: In the color pre-filtering process, parallel calculation of the NLM filtering algorithm is performed on the cumulative calculation part of the image information to be processed from the kernel array.

[0115] In this embodiment, step 21: in the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array includes:

[0116] Step 211: The main core obtains a set of pixels to be filtered and calculated in the image information to be processed;

[0117] Step 212: The master core transfers the pixel set to be filtered and calculated to the pixel pool via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the slave core array shared space;

[0118] Step 213: Each slave core in the slave core array retrieves the coordinates of the pixel to be filtered from the pixel pool, accesses the main memory, retrieves the pixel data of the area S centered on the pixel coordinates, and then performs NLM filtering.

[0119] Step 214: The pixel point that has completed the NLM filter calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for filter calculation from the pixel pool, and read the pixel data of the size of the main memory area S to perform the NLM filter calculation.

[0120] Step 215: Repeat step 214 until the pixel coordinates in the pixel pool are 0, and then terminate the parallel calculation from the kernel array.

[0121] See also Figure 3, for example, the NLM filtering algorithm in the pre-filtering process uses parallel acceleration from the core array. The specific principle is as follows: Define the pixel pool as a set of X pixel coordinates to be filtered and calculated from the shared space of the core array (X < the size of the LDM space and X is configurable). The main core transfers the set information of the X coordinates of the pixels to the pixel pool through the DMA method, and at the same time transfers the values of the search box S (the size of S is configurable) and the neighborhood box N (the size of N is configurable) to the shared space of the core array. Each core in the core array takes out the pixel point coordinates to be filtered and calculated from the X in the pixel pool, accesses the main memory space, takes out the pixel data with the size of the S area centered on the pixel point coordinates, and then performs the filtering calculation of NLM. The pixel points that have completed the NLM filtering calculation are transferred back to the main core through the DMA method. Then, the core takes out the next pixel point coordinates waiting for filtering calculation from the X in the pixel pool, reads the pixel data with the size of the S area in the main memory, and performs the filtering calculation of NLM until X in the pixel pool is 0, ending the parallel calculation of the core array. The pixel point NLM filtering calculation process in a single core is similarity calculation, fuzzy filtering calculation, and weight calculation.

[0122] In other embodiments, it is possible that there are too many pixels, resulting in the situation that one pixel pool cannot hold them. Therefore, in this embodiment, it further includes:

[0123] After completing the NLM filtering calculation of the X pixels in the pixel pool, the main core will transfer the next batch of X pixel coordinates to the pixel pool through the DMA method, and repeat the above steps until the NLM filtering calculation of all pixels is completed.

[0124] In this embodiment, shadow pre-filtering, feature pre-filtering, and color pre-filtering all adopt the above method for filtering calculation.

[0125] In this embodiment, step 3: Use the NLM method to perform feature construction conversion on the data after pre-filtering in a parallel acceleration manner from the core, so as to obtain the data after feature construction conversion. The specific method is as follows:

[0126] Step 31: The main core obtains the set of coordinates of the filtering area to be constructed and converted in the image information to be processed;

[0127] Step 32: The main core transfers the set of coordinates of the filtering area to be constructed and converted to the filtering area pool through the DMA method;

[0128] Step 33: Each core in the core array respectively obtains the corresponding filtering area data through the coordinates in the filtering area pool for the calculation of construction conversion;

[0129] Step 34: The filtering area that has completed the calculation is transferred back to the main core through the DMA method, and the core that has completed the calculation takes out the next filtering area from the filtering area pool for calculation;

[0130] Step 35: Repeat step 34 until there is no filtering area in the filtering area pool that needs to be calculated, and end the parallel calculation of the core array.

[0131] For example, the main core executes the feature construction conversion process, defines the filtering area pool as the coordinate set of M filtering areas to be constructed and converted in the shared space of the core array (M < the size of the LDM space and M is configurable), and the main core transmits the coordinates of the M filtering areas to the filtering area pool through the DMA method. Each slave core in the slave core array obtains the corresponding filtering area data through the coordinates of the filtering area pool M for the calculation of construction and conversion. The completed filtering area is transmitted back to the main core through the DMA method, and then the slave core fetches the next filtering area from M in the filtering area pool for calculation until M in the filtering area pool is 0, and the parallel calculation of the slave core array ends. The filtering area calculation process in a single slave core is mainly feature calculation, and the features are written into the conversion matrix.

[0132] In this embodiment, according to the data divided by the filtering area, after the slave core obtains the data of a certain filtering area, the calculation performed is the calculation of feature construction conversion.

[0133] In other embodiments, it is possible that there are too many pixels, resulting in the situation that one pixel pool cannot hold them. Therefore, in this embodiment, it further includes:

[0134] After completing the calculation of the M filtering areas in the filtering area pool, the main core will transmit the next batch of M filtering areas to the filtering area pool through the DMA method, and repeat the above steps until all filtering areas are calculated.

[0135] In this embodiment, step 4: Process the data after feature construction conversion by the parallel acceleration method of the slave core using the NLM method to obtain the final denoised data, including:

[0136] Step 41: The main core obtains the set of pixels for denoising reconstruction calculation in the image information to be processed;

[0137] Step 42: The main core transmits the set of pixels for denoising reconstruction calculation to the denoising reconstruction pixel pool through the DMA method, and at the same time transmits the values of the search box S and the neighborhood box N to the shared space of the slave core array;

[0138] Step 43: Each slave core in the slave core array respectively fetches the pixel point coordinates for denoising reconstruction calculation from the denoising reconstruction pixel pool, accesses the main memory space, fetches the pixel data with the size of the S area centered on the pixel point coordinates, and then performs the NLM filtering calculation;

[0139] Step 44: The pixel point that has completed the NLM filtering calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for denoising and reconstruction calculation from the pool of pixels to be reconstructed, and read the pixel data of the size of the main memory area S to perform the NLM filtering calculation.

[0140] Step 45: Repeat step 44 until the coordinate of the pixel point in the pixel pool to be denoised and reconstructed is 0, and then terminate the parallel calculation from the kernel array.

[0141] Specifically, the denoising and reconstruction process of the present application has a principle of obtaining pixel data that is similar to the pre-filtering process. The difference is that after obtaining the pixel data, the denoising and reconstruction process executes the NLM filtering algorithm process in a single slave core of the slave core array, which is fuzzy filtering calculation, weight calculation and gramian matrix construction, and transfers the denoised and reconstructed pixel data back to the main core via DMA.

[0142] After executing the above steps, the main core writes the denoised data into the cache. After denoising is completed, the final denoised image is output.

[0143] This paper proposes an accelerated method for non-local mean filtering denoising, which has the following advantages:

[0144] a) Leveraging the high-performance parallel computing advantages of the SW26010 processor's core array, the core array is used to parallelly accelerate the computational hotspots of the pre-filtering, feature structure conversion, and denoising reconstruction processes in the NLM algorithm-based denoising process. This effectively improves the parallel efficiency of the NLM algorithm denoising and significantly shortens the image denoising time.

[0145] b) The resource pool method of the shared space of the slave core array enables each slave core to adaptively adjust the number of tasks it obtains based on its own computing capacity, ensuring load balancing of task parallelism among the slave cores;

[0146] c) By dynamically configuring the size of the search box S, we can achieve computational outputs of different algorithm complexities to meet the user's denoising accuracy requirements;

[0147] d) It can be expanded to large-scale denoising of multiple frames and multiple main cores, achieving parallel acceleration of multi-frame image denoising.

[0148] Although the present invention has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made based on the present invention. Therefore, such modifications and improvements, which do not depart from the spirit of the present invention, are intended to be within the scope of protection claimed herein.

Claims

1. An accelerated method for non-local mean filtering denoising, characterized in that: The acceleration method of non-local mean filtering denoising includes: Step 1: Obtain the image information to be processed; Step 2: Use the NLM method to pre-filter the image information to be processed in a parallel accelerated manner from the core, thereby obtaining pre-filtered data; Step 3: Using the NLM method to perform feature structure transformation on the pre-filtered data in a parallel acceleration manner from the kernel, thereby obtaining feature structure transformed data; Step 4: Use the NLM method to process the data after feature construction transformation in a parallel acceleration manner from the kernel to obtain the final denoised data; The pre-filtering includes a shadow pre-filtering process, a feature pre-filtering process and a color pre-filtering process; The step 2: using the NLM method to pre-filter the image information to be processed in a manner of parallel acceleration from the core, thereby obtaining pre-filtered data includes: Step 21: In the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array; Step 22: In the feature pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array; Step 23: In the color pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array; Step 21: In the shadow pre-filtering process, performing parallel calculation of the NLM filtering algorithm on the cumulative calculation part of the image information to be processed from the kernel array includes: Step 211: The main core obtains a set of pixels to be filtered and calculated in the image information to be processed; Step 212: The master core transfers the pixel set to be filtered and calculated to the pixel pool via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the slave core array shared space; Step 213: Each slave core in the slave core array retrieves the coordinates of the pixel to be filtered from the pixel pool, accesses the main memory, retrieves the pixel data of the area S centered on the pixel coordinates, and then performs NLM filtering. Step 214: The slave core that has completed the NLM filter calculation transfers the pixel calculation result back to the master core via DMA. The slave core that has completed the calculation takes the coordinates of the next pixel waiting for filter calculation from the pixel pool and reads the pixel data of the size of the main memory area S to perform the NLM filter calculation. Step 215: Repeat step 214 until the pixel coordinates in the pixel pool are 0, and then terminate the parallel calculation of the slave core array. The step 3: performing feature structure conversion on the pre-filtered data in a manner of parallel acceleration from the core using the NLM method, thereby obtaining the data after feature structure conversion, includes: Step 31: The main core obtains a set of coordinates of the filter area to be constructed and transformed in the image information to be processed; Step 32: The main core transfers the filter area coordinate set to be constructed and transformed to the filter area pool through DMA; Step 33: Obtain corresponding filter area data from each slave core in the core array through the coordinates of the filter area pool to perform construction transformation calculation; Step 34: The slave core that has completed the calculation transfers the calculated filter area back to the master core via DMA. The slave core that has completed the calculation takes the next filter area from the filter area pool for calculation. Step 35: Repeat step 34 until there is no more filter area that needs to be calculated in the filter area pool, and then terminate the parallel calculation from the core array; The step 4: using the NLM method to process the data after feature structure conversion in a kernel parallel acceleration manner to obtain the final denoised data includes: Step 41: The main core obtains a set of pixels to be denoised and reconstructed in the image information to be processed; Step 42: The master core transfers the set of pixels to be denoised and reconstructed to the pool of pixels to be denoised and reconstructed via DMA, and at the same time transfers the values ​​of the search box S and the neighborhood box N to the shared space of the slave core array; Step 43: Each slave core in the core array retrieves the coordinates of the pixel to be denoised and reconstructed from the pool of pixels to be denoised and reconstructed, accesses the main memory, retrieves the pixel data of a region S centered on the pixel coordinates, and then performs NLM filtering calculations. Step 44: The pixel point that has completed the NLM filtering calculation will transfer the calculation result back to the main core via DMA. The slave core that has completed the calculation will take out the coordinates of the next pixel point waiting for denoising and reconstruction calculation from the pool of pixels to be reconstructed, and read the pixel data of the size of the main memory area S to perform the NLM filtering calculation. Step 45: Repeat step 44 until the coordinate of the pixel point in the pixel pool to be denoised and reconstructed is 0, and then terminate the parallel calculation from the kernel array.

2. The acceleration method for non-local means filtering denoising according to claim 1, characterized in that: In the shadow pre-filtering process, the NLM filtering calculation process of each pixel point in the single slave kernel is similarity calculation, fuzzy filter calculation and weight calculation.

3. The acceleration method for non-local means filtering denoising according to claim 2, characterized in that: The NLM filtering algorithm process executed in a single slave core of the slave core array in the denoising and reconstruction process includes fuzzy filtering calculation, weight calculation and gramian matrix construction.

Images