FPGA-based non-local mean denoising device and method

Abstract

The application relates to a non-local mean denoising device and method based on FPGA, which comprises the following modules: a cache module, which is used for caching a target image into a plurality of first-in-first-out queues according to a similar frame, a search frame and a center frame determined by a preset pixel template; an index calculation module, which is used for calculating a plurality of similarity indexes of the most central pixel of each similar frame and the most central pixel of the corresponding center frame in parallel through the floating-point operation of FPGA and the plurality of first-in-first-out queues; a weight calculation module, which is used for calculating the sum of the plurality of similarity indexes through floating-point conversion and calculating the weight of each pixel in each similar frame in the non-local mean processing of each center frame; and a denoising module, which is used for performing non-local mean denoising on the target image according to the weight of each pixel in each similar frame in the non-local mean processing of each center frame. The non-local mean denoising of FPGA is realized through the conversion of the pixel template and floating-point calculation.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a nonlocal mean denoising device and method based on FPGA. Background Technology

[0002] In image processing, object detection, and other fields, image denoising is an essential preprocessing step. Common two-dimensional image denoising methods include median filtering, mean filtering, and morphological filtering. These traditional denoising methods are simple in principle and have low computational complexity, but they cannot preserve image texture and other detailed information. Non-local means denoising (NLM) uses the redundant information that is ubiquitous in natural images to denoise. It finds similar regions in the image on a patch-by-pattern basis and then averages these similar regions, achieving good results in both noise removal and preservation of texture details. However, the NLM algorithm has high computational complexity, often requiring more hardware resources and processing time.

[0003] An FPGA is a programmable logic device, a type of semi-custom circuit within the category of application-specific integrated circuits (ASICs). Its basic structure includes programmable input / output units, configurable logic blocks, a digital clock management module, embedded block RAM, routing resources, an embedded dedicated hard core, and underlying embedded functional units. It offers advantages such as low power consumption, low cost, flexible development, and high processing speed. FPGAs allow for the development of independent hardware circuit structures for each system, thus combining the characteristics of parallel structures and pipelined processing.

[0004] The NLM algorithm is characterized by high computational complexity, high computational density, and high computational latency. Therefore, how to efficiently implement the NLM algorithm on FPGA is the key to enabling the NLM algorithm to run on embedded platforms such as FPGA. Summary of the Invention

[0005] To implement the NLM algorithm on an FPGA platform, a first aspect of the present invention provides an FPGA-based nonlocal means denoising device, comprising: a caching module for caching a target image in a multi-path first-in-first-out queue based on similarity boxes, search boxes, and center boxes determined by a preset pixel template; an exponent calculation module for parallelly calculating multiple similarity indices between the center pixel of each similarity box and the center pixel of its corresponding center box using floating-point arithmetic on the FPGA and the multi-path first-in-first-out queue; a weight calculation module for calculating the sum of the multiple similarity indices through floating-point conversion, and calculating the weight of each pixel in each similarity box relative to the nonlocal means processing of each center box based on the sum of the multiple similarity indices; and a denoising module for performing nonlocal means denoising on the target image based on the weight of each pixel in each similarity box relative to the nonlocal means processing of each center box.

[0006] In some embodiments of the present invention, the index calculation module includes: a sum of squares calculation unit, used to calculate the sum of squares of the absolute values ​​of the pixel differences between each center box and its similar boxes in each first-in-first-out queue; a floating-point conversion unit, used to convert the sum of squares of the absolute values ​​into a half-precision floating-point number; a division unit, used to perform a division operation between the sum of squares of the absolute values ​​after floating-point conversion and a preset standard deviation parameter; and a floating-point index calculation unit, used to calculate the similarity index between the center pixel of each center box and the center pixel of its multiple corresponding similar boxes based on the calculation result of the division unit and through a preset IP or piecewise function.

[0007] Furthermore, the sum of squares calculation unit includes: an input unit for matching multiple similar boxes from one or more search boxes corresponding to each center box; and a first calculation unit for calculating the variance of the pixel values ​​of the corresponding pixels of each center box and the multiple similar boxes using a preset template.

[0008] In some embodiments of the present invention, the weight calculation module includes: a floating-point conversion unit for converting each similarity index from a floating-point number to a fixed-point number; a first calculation unit for calculating the sum of multiple similarity indices after floating-point conversion; a fixed-point conversion unit for converting multiple similarity indices and their sum into floating-point numbers, and normalizing them based on the similarity indices and their sum after fixed-point conversion; and a weight calculation unit for calculating the weight of each pixel within each similar bounding box relative to the non-local mean processing of each center box based on the normalized similarity indices and their sum.

[0009] Furthermore, the weight calculation unit also includes an enhancement unit for enhancing the calculation clock frequency of the weight calculation unit.

[0010] In the above embodiments, the denoising module includes: a fixed-point unit, used to perform fixed-point denoising on the weight of each pixel in each search box relative to the non-local mean processing of each center box; and a multiply-add unit, used to multiply the multiple fixed-point weight values ​​with the pixel values ​​of the corresponding pixels and sum them to obtain the pixel value of each center box after non-local mean denoising.

[0011] A second aspect of the present invention provides an FPGA-based nonlocal means denoising method, comprising: caching a target image in a multi-path first-in-first-out queue based on similarity boxes, search boxes, and center boxes determined by a preset pixel template; calculating, in parallel, multiple similarity indices between the center pixel of each similarity box and the center pixel of its corresponding center box using floating-point arithmetic of the FPGA and the multi-path first-in-first-out queue; calculating the sum of the multiple similarity indices using floating-point conversion, and calculating the weight of each pixel within each similarity box in the nonlocal means processing of each center box based on the sum of the multiple similarity indices; and performing nonlocal means denoising on the target image based on the weight of each pixel within each similarity box in the nonlocal means processing of each center box.

[0012] Furthermore, the parallel calculation of multiple similarity indices between the center pixel of each similar box and its corresponding center pixel using the FPGA's floating-point arithmetic and the multi-way FIFO queue includes: calculating the sum of squares of the absolute values ​​of the pixel differences between each center box and its corresponding similar boxes in each FIFO queue; converting the sum of squares of the absolute values ​​into a half-precision floating-point number; and, based on the calculation result of the division unit, calculating the similarity index between the center pixel of each center box and the center pixels of its corresponding multiple similar boxes using a preset IP or piecewise function.

[0013] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the FPGA-based nonlocal mean denoising method provided in the second aspect of the present invention.

[0014] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the FPGA-based nonlocal mean denoising method provided in the second aspect of the present invention.

[0015] The beneficial effects of this invention are:

[0016] This invention relates to an FPGA-based nonlocal mean denoising apparatus and method. The apparatus includes: a caching module for caching a target image in a multi-path first-in-first-out queue based on similarity boxes, search boxes, and center boxes determined by a preset pixel template; an index calculation module for parallelly calculating multiple similarity indices between the center pixel of each similarity box and the center pixel of its corresponding center box using floating-point arithmetic of the FPGA and the multi-path first-in-first-out queue; a weight calculation module for calculating the sum of the multiple similarity indices through floating-point conversion and calculating the weight of each pixel within each similarity box in the nonlocal mean processing of each center box; and a denoising module for performing nonlocal mean denoising on the target image based on the weight of each pixel within each similarity box in the nonlocal mean processing of each center box.

[0017] As can be seen, this invention addresses the challenge of high hardware implementation complexity of the NLM algorithm by providing an FPGA-based hardware implementation of the NLM algorithm. By leveraging the parallel pipelined computing characteristics of the FPGA device architecture, and while meeting the accuracy requirements of the NLM algorithm, it achieves a fully pipelined and efficient hardware computing architecture for the NLM algorithm based on pixel templates and the conversion between fixed-point and floating-point numbers, with a limited loss of accuracy. This reduces the complexity, computational density, and latency of the NLM algorithm. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the structure of an FPGA-based nonlocal mean denoising device in some embodiments of the present invention;

[0019] Figure 2 This is a schematic diagram of the specific structure of an FPGA-based nonlocal mean denoising device in some embodiments of the present invention;

[0020] Figure 3 This is a schematic diagram of the cache module of an FPGA-based nonlocal mean denoising device in some embodiments of the present invention.

[0021] Figure 4 This is a schematic diagram of a pixel template in some embodiments of the present invention;

[0022] Figure 5 This is a schematic diagram of the principle of an FPGA-based exponent calculation module in some embodiments of the present invention;

[0023] Figure 6 This is a schematic diagram illustrating the floating-point format representation in some embodiments of the present invention;

[0024] Figure 7 This is a schematic diagram illustrating the floating-point division calculation of IP in some embodiments of the present invention;

[0025] Figure 8This is a schematic diagram of the floating-point exponent calculation IP in some embodiments of the present invention;

[0026] Figure 9 This is a schematic diagram of the weight calculation module in some embodiments of the present invention and a schematic diagram of the FPGA NLM algorithm deployment resources;

[0027] Figure 10 These are FPGA simulation diagrams of the NLM algorithm in some embodiments of the present invention;

[0028] Figure 11 These are comparison images of FPGA NLM algorithm noise reduction in some embodiments of the present invention;

[0029] Figure 12 This is a schematic diagram of the basic process of the FPGA-based nonlocal mean denoising method in some embodiments of the present invention;

[0030] Figure 13 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation

[0031] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0032] refer to Figure 1 and Figure 2 In a first aspect of the present invention, an FPGA-based nonlocal mean denoising device 1 is provided, comprising: a caching module 11, configured to cache a target image in a multi-path first-in-first-out queue based on similarity boxes, search boxes, and center boxes determined by a preset pixel template; an index calculation module 12, configured to calculate, in parallel, multiple similarity indices between the center pixel of each similarity box and the center pixel of its corresponding center box using floating-point arithmetic of the FPGA and the multi-path first-in-first-out queue; a weight calculation module 13, configured to calculate the sum of the multiple similarity indices by floating-point conversion, and calculate the weight of each pixel in each similarity box relative to the nonlocal mean processing of each center box based on the sum of the multiple similarity indices; and a denoising module 14, configured to perform nonlocal mean denoising on the target image based on the weight of each pixel in each similarity box relative to the nonlocal mean processing of each center box.

[0033] It should be noted that the similarity index in this invention is based on the grayscale value of the pixel, or in other words, the pixel value in this invention is the grayscale value of the pixel; the similarity box, search box, and center box are used for the convenience of description, and the corresponding areas are represented by rectangular boxes in the examples or drawings; without loss of generality, the above-mentioned similarity box, search box, and center box can also be understood as the source area, search area, and target area, respectively corresponding to the center box, search box, and similarity box.

[0034] refer to Figures 2 to 4 In some embodiments of the present invention, the caching module 11 is used to cache the target image in a multi-way first-in-first-out queue based on the similarity box, search box and center box determined by the preset pixel template.

[0035] Specifically, such as Figure 4 As shown, since the radius of the similar bounding box is 2, its diameter is 5; and the radius of the search box is 4, its diameter is 9. Therefore, when calculating and comparing the center box and the similar bounding box, the image caching module needs to form a 13x13 pixel template. Twelve FIFOs are used for row caching of image data. Each row of image data flows into the next level FIFO after being cached by the current level FIFO. After 12 levels of FIFO caching, 13 rows of image data can be output simultaneously to form a 13x13 image template for subsequent calculation modules. The depth of the FIFO is determined based on the column width of the image.

[0036] refer to Figures 4 to 8 In some embodiments of the present invention, the index calculation module 12 includes: a sum of squares calculation unit, used to calculate the sum of squares of the absolute values ​​of the pixel differences between each center box and its similar boxes in each first-in-first-out queue; a floating-point conversion unit, used to convert the sum of squares of the absolute values ​​into a half-precision floating-point number; a division unit, used to perform a division operation between the sum of squares of the absolute values ​​after floating-point conversion and a preset standard deviation parameter; and a floating-point index calculation unit, used to calculate the similarity index between the center pixel of each center box and the center pixel of its corresponding multiple similar boxes based on the calculation result of the division unit and through a preset IP or piecewise function.

[0037] Furthermore, the sum of squares calculation unit includes:

[0038] The input unit is used to match multiple similar boxes from one or more search boxes corresponding to each center box; the first calculation unit is used to calculate the variance of the pixel values ​​of the corresponding pixels of each center box and multiple similar boxes using a preset template.

[0039] Specifically, it is necessary to calculate the sum of squares of the absolute values ​​of the corresponding pixels between the 5x5 center bounding box and a certain 5x5 similar bounding box. To save computational resources and implement pipelined computation when calculating the sum of squares of the absolute values ​​of the pixel differences between the center bounding box and the similar bounding box, it is not necessary to completely cache the pixels of the two 5x5 templates before calculation. Figure 3 As shown, input the corresponding 5x1 template pixels of the center box and similar boxes column by column, calculate the sum of squares of the differences between the 5x1 column data of the similar box and the 5x1 column data of the center box in turn, and then add the sum of squares of the 5 columns to get the sum of squares of the corresponding pixel differences between the 5x5 similar box and the 5x5 center box, which is denoted as d^2.

[0040] It is understandable that after obtaining the d^2 between the center bounding box and one of its similar bounding boxes, d^2 needs to be used as the dividend and divided by the input standard deviation parameter Std. Two aspects need to be considered when performing division: first, the precision of the division operation must be guaranteed; therefore, the division result cannot only take the integer part, but must guarantee a certain precision after the decimal point; second, the FPGA logic resources occupied by the division operation must be considered, as higher division precision consumes more computational resources. Considering both aspects, this invention uses a half-precision floating-point format for division calculation.

[0041] Therefore, a floating-point conversion unit is used to convert the sum of squares of the absolute values ​​into half-precision floating-point numbers; specifically, d^2 is first converted to the standard deviation parameter Std according to... Figure 5 The format shown converts a fixed-point number to a half-precision floating-point number. A half-precision floating-point number is 16 bits long, with the highest bit being the sign bit, the exponent occupying 5 bits, and the decimal places occupying 10 bits. This conversion does not require calling any IP address; it can be completed using FPGA logic.

[0042] Next, division is performed using the division unit; specifically, refer to... Figure 7 The division calculation can be completed by calling the floating-point calculation IP inside the FPGA.

[0043] It is understandable that, according to the calculation process of the NLM algorithm, after the division calculation is completed, the result of the division operation needs to be reversed and the logarithm of the reversed number needs to be calculated. Currently, most FPGAs have integrated floating-point format pointer arithmetic IP (circuit).

[0044] Therefore, the floating-point exponent calculation unit is used to calculate the similarity index between the center pixel of each center box and the center pixels of its corresponding multiple similar boxes, based on the calculation result of the division unit and through a preset IP or piecewise function. For example... Figure 8 As shown, the exponent calculation IP can be directly called for exponentiation. If the FPGA does not have an integrated exponent calculation IP, exponentiation can be achieved by generating a lookup table based on the value range or by fitting a piecewise function. The floating-point exponent calculation module adopts a pipelined operation. For a 9x9 search area, excluding the center box itself, a total of 80 exponent calculation modules are required. The number of exponent calculation modules required to be instantiated varies depending on the definition of the search box.

[0045] refer to Figure 9 In some embodiments of the present invention, the weight calculation module 13 includes:

[0046] The floating-point conversion unit is used to convert each similarity exponent from a floating-point number to a fixed-point number; the first calculation unit is used to calculate the sum of multiple similarity exponents after floating-point conversion;

[0047] Specifically, according to the 9x9 search area, each pixel can obtain 80 exponential results through the exponential calculation module. Since the d^2 value of the center box and itself is 1, its exponential value is 1. The weight calculation module sums the 80 exponential results with 1 to obtain the exponential sum.

[0048] The fixed-point conversion unit is used to convert multiple similar exponents and their sum into floating-point numbers, and to normalize them based on the similar exponents and their sum after fixed-point conversion.

[0049] Specifically, if floating-point format is used for summation, the exponents of the floating-point numbers need to be aligned and shifted. In addition, rounding errors will occur. Therefore, this scheme converts the floating-point exponents to fixed-point numbers for summation. For a 16-bit half-precision floating-point number, a 25-bit fixed-point number is used to represent it, ensuring no conversion error during the floating-point to fixed-point conversion. Specifically, a pipelined operation is used to add 80 fixed-point exponents to 1 to obtain the exponent sum.

[0050] The weight calculation unit is used to calculate the weight of each pixel within each similar bounding box relative to the non-local mean processing of each center box, based on the normalized similarity index and its sum. Each index calculated by the index calculation module 11 represents the similarity between the center pixels of the similar bounding boxes and the center pixels of the center boxes. Therefore, after obtaining the sum of the indices, it is necessary to further calculate the division between each index and the sum of the indices for normalization. The division is performed using a half-precision floating-point format, converting the fixed-point index sum to a floating-point number.

[0051] Furthermore, the weight calculation unit also includes an enhancement unit for enhancing the calculation clock frequency of the weight calculation unit.

[0052] Specifically, the 81 floating-point exponents are divided by the sum of floating-point exponents, and the result is denoted as the weight parameter w, which represents the proportion of each pixel in the search box in the final NLM-processed pixel composition. To save FPGA computing resources, this module increases the computation clock by 1.5 times; what originally required 81 division IPs now only requires 54 division IPs in the 1.5 times clock domain.

[0053] In the above embodiments, the denoising module 14 includes: a fixed-point unit, used to perform fixed-point denoising on the weight of each pixel in each search box relative to the non-local mean processing of each center box; and a multiply-add unit, used to multiply the multiple fixed-point weight values ​​with the pixel values ​​of the corresponding pixels and sum them to obtain the pixel values ​​of each center box after non-local mean denoising.

[0054] Specifically, after the weight calculation module obtains the weight values ​​of each pixel in the search box, it converts the floating-point weight values ​​to fixed-point values. Then, it multiplies the 81 fixed-point weight values ​​with the 81 corresponding pixels in the search box and sums them up. Finally, the NLM calculated pixel values ​​can be obtained. The multiplication and summation still adopt the pipelined calculation method.

[0055] Figure 9 The resource usage table is shown after deploying the NLM computing architecture of this solution on an Xlinx K7-325T FPGA. The logic resource LUT is used by 33%, the register resource FF is used by 22%, the block memory resource is used by only 11%, and the DSP is used by 60%, mainly for the exponential calculation module.

[0056] Figure 11 The FPGA simulation diagram of the NLM algorithm based on this application is shown, as well as the comparison diagram of PGA NLM noise reduction before and after.

[0057] Example 2

[0058] refer to Figure 12 A second aspect of the present invention provides a nonlocal means denoising method based on FPGA, comprising: S100. caching the target image in a multi-path first-in-first-out queue according to similar boxes, search boxes, and center boxes determined by a preset pixel template; S200. calculating in parallel multiple similarity indices between the center pixel of each similar box and the center pixel of its corresponding center box using floating-point arithmetic of the FPGA and the multi-path first-in-first-out queue; S300. calculating the sum of the multiple similarity indices by floating-point conversion, and calculating the weight of each pixel in each similar box relative to the nonlocal means processing of each center box based on the sum of the multiple similarity indices; S400. performing nonlocal means denoising on the target image according to the weight of each pixel in each similar box relative to the nonlocal means processing of each center box.

[0059] Furthermore, in step S200, the parallel calculation of multiple similarity indices between the center pixel of each similar bounding box and its corresponding center pixel of the central bounding box using floating-point arithmetic on the FPGA and the multi-way first-in-first-out queue includes:

[0060] S201. Calculate the sum of the squares of the absolute values ​​of the pixel differences between each center box and its similar boxes in each first-in-first-out queue;

[0061] S202. Convert the sum of squares of the absolute values ​​into a half-precision floating-point number;

[0062] S203. Based on the calculation result of the division unit, calculate the similarity index between the center pixel of each center box and the center pixels of its corresponding multiple similar boxes by using a preset IP or piecewise function.

[0063] Example 3

[0064] refer to Figure 13 A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method of the second aspect of the present invention.

[0065] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0066] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 13 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 13 Each box shown can represent a device or multiple devices as needed.

[0067] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0068] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to:

[0069] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A nonlocal mean denoising device based on FPGA, characterized in that, include: The caching module is used to cache the target image in a multi-way first-in-first-out queue based on the similarity box, search box and center box determined by the preset pixel template; The index calculation module is used to calculate multiple similarity indices between the center pixel of each similar box and the center pixel of its corresponding center box in parallel through the floating-point arithmetic of the FPGA and the multi-way first-in-first-out queue. The index calculation module includes: a sum of squares calculation unit, used to calculate the sum of squares of the absolute values ​​of the pixel differences between each center box and its similar boxes in each first-in-first-out queue; a floating-point conversion unit, used to convert the sum of squares of the absolute values ​​into a half-precision floating-point number; a division unit, used to perform a division operation on the sum of squares of the absolute values ​​after floating-point conversion with a preset standard deviation parameter; and a floating-point index calculation unit, used to calculate the similarity index between the center pixel of each center box and the center pixels of its multiple corresponding similar boxes based on the calculation result of the division unit and using a preset IP or piecewise function; the sum of squares calculation unit includes: an input unit, used to match multiple similar boxes from one or more search boxes corresponding to each center box; and a first calculation unit, used to calculate the variance of the pixel values ​​of the corresponding pixels of each center box and multiple similar boxes using a preset template; A weight calculation module is used to calculate the sum of the multiple similarity indices through floating-point conversion, and to calculate the weight of each pixel in each similar box relative to the non-local mean processing of each center box based on the sum of the multiple similarity indices. The weight calculation module includes: a floating-point conversion unit for converting each similarity index from a floating-point number to a fixed-point number; a first calculation unit for calculating the sum of the multiple similarity indices after floating-point conversion; a fixed-point conversion unit for converting the multiple similarity indices and their sum to floating-point numbers, and normalizing them based on the similarity indices and their sum after fixed-point conversion; and a weight calculation unit for calculating the weight of each pixel in each similar box relative to the non-local mean processing of each center box based on the normalized similarity indices and their sum. The weight calculation unit further includes: an boosting unit for boosting the calculation clock frequency of the weight calculation unit. The denoising module is used to perform non-local mean denoising on the target image based on the weight of each pixel in each similar bounding box relative to the non-local mean processing of each center box.

2. The FPGA-based nonlocal mean denoising device according to claim 1, characterized in that, The noise reduction module includes: The fixed-point unit is used to perform fixed-point calculation on the weight of each pixel in each search box relative to the non-local mean processing of each center box. The multiply-add unit is used to multiply multiple fixed-point weight values ​​with the corresponding pixel values ​​and sum them to obtain the pixel value of each center box after non-local mean denoising.

3. A nonlocal means denoising method based on FPGA, characterized in that, include: Based on the similarity box, search box, and center box determined by the preset pixel template, the target image is cached in a multi-way first-in-first-out queue; Using the FPGA's floating-point operations and the multi-path FIFO queue, multiple similarity indices are calculated in parallel between the center pixel of each similar bounding box and its corresponding center pixel. This involves calculating the sum of squares of the absolute values ​​of the pixel differences between each center box and its similar bounding boxes in each FIFO queue; converting the sum of squares of these absolute values ​​into half-precision floating-point numbers; and, based on the calculation results from the division unit, calculating the similarity index between the center pixel of each center box and the center pixels of its multiple corresponding similar bounding boxes using a preset IP or piecewise function. The sum of squares of the absolute values ​​of the pixel differences between each center box and its similar bounding boxes in each FIFO queue includes: matching multiple similar bounding boxes from one or more search boxes corresponding to each center box; and calculating the variance of the pixel values ​​of the corresponding pixels of each center box and its multiple similar bounding boxes using a preset template. The sum of the multiple similarity indices is calculated by converting floating-point numbers to fixed-point numbers. Based on the sum of the multiple similarity indices, the weight of each pixel within each similar bounding box in the non-local mean processing of each center box is calculated. The weight calculation includes: converting each similarity index from a floating-point number to a fixed-point number; calculating the sum of the multiple similarity indices after floating-point conversion; converting the multiple similarity indices and their sum back to floating-point numbers, and normalizing them based on the similarity indices and their sum after fixed-point conversion; calculating the weight of each pixel within each similar bounding box in the non-local mean processing of each center box based on the normalized similarity indices and their sum; and increasing the clock frequency of the weight calculation unit. Based on the weight of each pixel within each similar bounding box relative to the nonlocal mean processing of each center box, nonlocal mean denoising is performed on the target image.

4. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the FPGA-based nonlocal mean denoising method as described in claim 3.

5. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the FPGA-based nonlocal mean denoising method as described in claim 3.

Images