A lightweight network hardware accelerator for image defogging

Abstract

The application discloses a kind of lightweight network hardware accelerators for image defogging, comprising: fuzzy image storage module, two sampling modules, BLD-Net module, ASM module, BI module, address control module;Wherein, fuzzy image storage module stores image data and is transmitted to first sampling module for sampling, then the sampling data is transmitted to BLD-Net module processing, it is converted into image reconstruction parameter K (x), subsequently pass to ASM module;In ASM module, a combination method is used to calculate the transmission map and atmospheric light of given fuzzy image, to generate initial defogging image data;Initial defogging image data and fuzzy image data are subsequently sampled by second sampling module and transmitted to BI module;After being processed through BI module, the final defogging is reconstructed by merging image data. The application can improve the performance of image defogging accelerator, speed up the processing speed to improve the defogging effect.

Description

Technical Field

[0001] This invention belongs to the technical field of image dehazing and can be applied to image dehazing accelerators for automotive driver assistance, intelligent security monitoring, and industrial control monitoring. Background Technology

[0002] Due to the rapid advancement of artificial intelligence (AI) technology, the industrial sector is undergoing a transformative revolution in automation, reliability, and robustness. However, severe weather conditions can significantly degrade the visual quality of images, primarily due to the presence of polluted atmospheric particles, which adversely affects the entire monitoring system. Images captured in hazy environments are often affected by noise (haze), which significantly reduces their visibility. Reconstructing blurred images is a very tedious task, requiring additional information to remove hazy particles from the captured image. To address this issue, dehazing algorithms based on traditional prior models estimate atmospheric light and transmission maps by analyzing scene depth, polarization filtering, and the texture of the hazy image, ultimately achieving the reconstruction of the blurred image. For example, He et al. proposed a single-image haze removal method using the Dark Channel Prior (DCP) algorithm. They used the ASM model to estimate atmospheric light and transmission maps. The calculated transmission map and atmospheric light were used to obtain the final dehazed image. Wang et al. proposed a single-image haze removal algorithm that recovers haze-free images by utilizing the linear relationship between the estimated transmission map and the blurred image. This algorithm performs poorly in dehazing high-resolution images and exacerbates darkness, requiring further image processing techniques. While traditional dehazing algorithms are feasible, they perform poorly with outdoor, naturally hazy images exhibiting general fog density.

[0003] In addition to dehazing algorithms based on traditional prior models, various CNN-assisted single-image dehazing algorithms have emerged in the field of deep learning. For example, Li et al. proposed a learning-based single-hazy image reconstruction algorithm. They approximate atmospheric light and transmission maps by combining CNN and ASM models. Zhang et al. proposed a CNN-assisted image haze removal framework based on a perceptual pyramid deep network. Their method obtains the dehazed image by utilizing the local relationship between the hazy image and its corresponding depth image. While these deep learning-based algorithms effectively overcome the limitations of algorithms based on traditional prior models and demonstrate significant performance in image dehazing, many CNN-based dehazing methods have very complex architectures. These architectures increase the computational cost and time complexity of the overall blurred image reconstruction process, limiting their deployment on resource-constrained platforms and resulting in poor real-time performance. This makes these computationally expensive and memory-intensive methods unsuitable for deployment and processing on resource-constrained edge platforms. Summary of the Invention

[0004] To address the shortcomings of the existing technology, this invention proposes a lightweight network hardware accelerator for image dehazing, aiming to accelerate image processing, improve dehazing effect, and meet the requirements of real-time high-quality dehazing as much as possible.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a lightweight network hardware accelerator for image dehazing, which includes: a blurred image storage module, two sampling modules, a BLD-Net module, an ASM module, a BI module, and an address control module.

[0007] The blurred image storage module includes two image RAM modules, each used to store an identical blurred image. The first image RAM module transmits a blurred image to the first sampling module.

[0008] The first sampling module converts the received blurred image into a blurred digital image I, and detects whether any pixel I(x) in the blurred digital image I belongs to a preset sampling point. If it does, I(x) is recorded as pixel I2(x) and transmitted to the BI module; otherwise, I(x) is recorded as pixel I1(x) and transmitted to the BLD-Net module.

[0009] The BLD-Net module uses equation (1) to convert the received pixel I1(x) into image reconstruction parameters K(x) and passes them to the ASM module;

[0010]

[0011] In equation (1), t(x) represents the transfer function; A represents the atmospheric light value; and b represents the bias.

[0012] The ASM module includes: an ASM separation module, a first original code to two's complement module, a first two's complement to original code module, three sets of ASM shift registers, three ASM bias RAMs, an ASM merging module, and three ASM buffers;

[0013] The ASM separation module receives the image reconstruction parameters K(x) and decomposes them into three channel components, including: R channel component R(x), G channel component G(x), and B channel component B(x);

[0014] The first complement-to-original code module converts the three-channel components into their original codes. Then, it multiplies these original codes by the corresponding three-channel components of the blurred image stored in the second image RAM module to obtain the initial three-channel components. These initial components are then added to the corresponding R-channel component R(x), G-channel component G(x), and B-channel component B(x) to obtain the R-channel component K. R (x), G channel component K G (x), B channel component K B (x);

[0015] The three ASM buffers respectively target K R (x), K G (x), K B (x) After being cached, it is subtracted from the corresponding biases stored in the three ASM bias RAMs. The three results are then transmitted to the three sets of ASM shift registers for right shifting. The shifted components of the three channels are then transmitted to the first original code to two's complement module for conversion. After obtaining the two's complement of the three channels, they are transmitted to the ASM merging module.

[0016] The ASM merging module merges the two's complement of the three channels into pixel P(x) in the initial dehazed image P and sends it to the second sampling module for luminance component sampling. The luminance component of pixel P(x) in the initial dehazed image P is then transmitted to the BI module.

[0017] The BI module includes: two BI separation modules, a BI weight storage module, a BI bias storage module, a BI threshold storage module, a brightness factor module, a BI buffer module, and two sets of BI shift registers;

[0018] The first BI separation module decomposes the luminance component of P(x) into luminance components of the R channel, G channel, and B channel;

[0019] The second BI separation module decomposes pixel I2(x) into blur components of the R channel, G channel, and B channel;

[0020] The BI weight storage module stores the weights corresponding to the R, G, and B channels respectively. The luminance components of P(x) in the R, G, and B channels are multiplied by their respective weights to obtain the first weight R component W for each of the three channels. R First weighted component G W G First weighted component B W B ;

[0021] The BI cache module takes W respectively R W G W BThe middle and high 24 bits are registered, and then the high 24 bits of the three channels are accumulated with the bias in the bias storage module to obtain the first bias component B1, which is then transmitted to the BI threshold storage module.

[0022] The BI threshold storage module includes: two sets of shift registers, a minimum threshold storage module, and two sets of maximum threshold storage modules;

[0023] The first group of BI shift registers receives the first bias component B1 and performs a right shift operation to obtain the shifted first bias component, which is then transmitted to the maximum threshold module and the minimum threshold module respectively.

[0024] The maximum threshold module compares the shifted first bias component with the maximum threshold L and retains the larger value;

[0025] The minimum threshold module compares the shifted first bias component with the minimum threshold M and retains the larger value;

[0026] The higher eight bits of the larger of the two larger values ​​are added to the brightness factor in the brightness factor module to obtain a new brightness factor, which is then stored in the brightness factor module for the next accumulation calculation. This process continues until the image sampling is completed, at which point the finally stored brightness factor Light1 is sent to the divider.

[0027] The BI weight storage module multiplies the blur components of the R, G, and B channels of pixel I2(x) with the corresponding weights of the R, G, and B channels in the BI weight storage module to obtain the second weight R component W of the three channels. R2 The second weighted component G, W G2 The second weighted component B, W B2 ;

[0028] The BI cache module takes W respectively R W G W B The middle and high 24 bits are registered and then accumulated with the bias to obtain the second bias component B2. This component is then transmitted to the BI threshold module for comparison and processing to obtain the fuzzy brightness factor Light2, which is then sent to the divider.

[0029] The divider divides the luminance factor Light1 and the blur luminance factor Light2, and then multiplies the resulting combined luminance factor by the luminance components of the R, G, and B channels of P(x) in the initial dehazed image P, respectively, to obtain the three-channel components F of the final dehazed image. R F G F B The image is then sent to the BI merging module for merging processing to obtain the final dehazed image F.

[0030] The lightweight network hardware accelerator for image dehazing of this invention is characterized by the following: the BLD-Net module includes: eight convolutional layer modules, three combination layer modules, and four pipeline data RAMs.

[0031] Each convolutional layer module includes: pixel RAM1, pixel RAM2, three BLD weight RAMs, BLD bias RAM, read / write control module, BLD shift register, pipeline data module, BLD register, BLD separation module, second complement to original code module, second original code to complement module, comparator, adder, and BLD threshold module.

[0032] The read / write control module in the first convolutional layer module writes pixel I1(x) into its own RAM1 for storage, and writes the original data in pixel RAM1 into pixel RAM2. At the same time, it stores I1(x), the original data in pixel RAM1 and the original data in pixel RAM2 in its own pipeline data module, thereby obtaining a 3×3 convolution kernel and then passing it into its own BLD separation module.

[0033] The BLD separation module in the first convolutional layer module decomposes the 3×3 convolutional kernel into an R-channel convolutional kernel, a G-channel convolutional kernel, and a B-channel convolutional kernel. Then, through its own second original code to two's complement module, it performs two's complement conversion to obtain the R-channel convolutional kernel, G-channel convolutional kernel, and B-channel convolutional kernel in two's complement form, and then sends them to the BLD register for storage.

[0034] The three BLD weight RAMs in the first convolutional layer module multiply their own weights by the corresponding R-channel convolutional kernel, G-channel convolutional kernel, and B-channel convolutional kernel in two's complement form in the BLD register, respectively, to obtain the R-channel convolutional kernel weight, G-channel convolutional kernel weight, and B-channel convolutional kernel weight;

[0035] The BLD shift register in the first convolutional layer module shifts the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights respectively. The resulting three shifted weights are then sent to its own second complement to original code module for original code conversion to obtain the original code form of the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights.

[0036] The adder in the first convolutional layer module accumulates the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights in their original form with the bias in the bias RAM to obtain the first layer parameter K1(x) and transmits it to the first layer comparator.

[0037] The comparator in the first convolutional layer module compares K1(x) with the threshold of the BLD threshold module. If K1(x) is greater than the threshold, K1(x) is output; otherwise, 0 is output. K1(x) is then input into the second convolutional layer module and the first pipeline data RAM, respectively.

[0038] After the second convolutional layer module processes K1(x) according to the process of the first convolutional layer module, it obtains the second layer parameter K2(x), which is then input into the third convolutional layer module and the second pipeline data RAM respectively. After the third convolutional layer module performs the operation and processing, it obtains the third layer parameter K3(x), which is then input into the first combination layer module.

[0039] The first pipeline data RAM transmits the stored K1(x) to the first combination layer module;

[0040] The first combined layer module merges K1(x) and K3(x) to obtain the first combined layer parameter K. a (x) is transmitted to the fourth convolutional layer module for processing to obtain the fourth layer parameter K4(x). Then, it is input into the fifth convolutional layer module and the third pipeline data RAM respectively. After processing by the fifth convolutional layer module, the fifth layer parameter K5(x) is obtained. Then, it is input into the sixth convolutional layer module and the fourth pipeline data RAM respectively. After processing by the sixth convolutional layer module, the sixth layer parameter result K6(x) is obtained and input into the second combination layer module.

[0041] The third pipeline data RAM transmits the stored K4(x) to the second combination layer module;

[0042] The second combined layer module merges K4(x) and K6(x) to obtain the second combined layer parameter K. b (x) is then transmitted to the seventh convolutional layer module for processing, and the sixth layer parameter result K7(x) is then input into the third combination layer module;

[0043] The second pipeline data RAM transmits the stored K2(x) to the third combination layer module;

[0044] The fourth pipeline data RAM transfers the stored K5(x) to the third combination layer module;

[0045] After merging K2(x), K5(x), and K7(x) in the third combination layer module, the parameter K of the third combination layer is obtained. c (x) is then transmitted to the eighth convolutional layer module for processing to obtain the image reconstruction parameters K(x).

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] 1. This invention proposes a lightweight and easy-to-deploy dehazing neural network for low-resource platforms. It solves the artifact problem in the dehazed images generated by the original network and introduces a combination layer to retain the semantics learned by the early convolutional layers to compensate for the information loss during the convolution operation. It reconstructs the blurred image by jointly computing the transmission map and atmospheric light of the given blurred image, which greatly reduces the complexity of the architecture model, making the time complexity 5.8% of the original network, and is easy to deploy on resource-constrained platforms.

[0048] 2. To recover the brightness loss that occurs during the reconstruction of blurred images, this invention introduces a BI module. The BI module samples the original blurred image and the initial dehazed image processed by BLD-Net using the brightness component to obtain an adaptive brightness factor, which is then multiplied with the initial dehazed image to obtain the final dehazed image that recovers the brightness loss that occurred during image reconstruction, thereby mitigating the brightness loss experienced during image reconstruction.

[0049] 3. This invention designs a lightweight network hardware accelerator for image dehazing. The accelerator employs a multi-stage pipeline technique, significantly improving the efficiency of generating haze-free images. The proposed accelerator achieves an impressive maximum frame rate of 106 frames per second, exceeding software-based methods by 56 times. Furthermore, in this multi-stage pipeline architecture, the storage requirement for each convolutional layer involving the combination layers is reduced to storing only a few rows per image. This effectively reduces on-chip storage requirements, resulting in significant storage savings. Attached Figure Description

[0050] Figure 1 This is a diagram showing the overall architecture of the lightweight network hardware accelerator for image dehazing used in this invention.

[0051] Figure 2 This is a hardware implementation diagram of the convolutional layer used in this invention;

[0052] Figure 3 This is the pipeline timing diagram used in this invention;

[0053] Figure 4 This is a hardware implementation diagram of ASM used in this invention;

[0054] Figure 5 This is a hardware implementation diagram of the BI used in this invention. Detailed Implementation

[0055] In this example, such as Figure 1 As shown, the overall architecture of the lightweight network hardware accelerator for image dehazing includes: a blurred image storage module, two sampling modules, a BLD-Net module, an ASM module, a BI module, and an address control module.

[0056] The blurred image storage module contains two image RAM modules, each used to store an identical blurred image. In this example, the size of the stored blurred image is 640×480 pixels. The first image RAM module transmits a blurred image to the first sampling module.

[0057] After receiving the blurred image, the first sampling module converts it into a blurred digital image I and detects whether any pixel I(x) in the blurred digital image I belongs to a preset sampling point. If it does, the corresponding pixel I2(x) is transmitted to the BI module; otherwise, the corresponding pixel I1(x) is transmitted to the BLD-Net module. In this example, pixel sampling is performed on 225 points in a 15×15 grid.

[0058] The BLD-Net module uses equation (1) to convert the received pixel I1(x) into image reconstruction parameters K(x) and passes them to the ASM module;

[0059]

[0060] Where t(x) represents the transfer function, A represents the atmospheric light value, and b represents the bias;

[0061] In this example, the BLD-Net module includes: eight convolutional layer modules, three combination layer modules, and four pipeline data RAMs;

[0062] like Figure 2 As shown, each convolutional layer module includes: pixel RAM1, pixel RAM2, three BLD weight RAMs, BLD bias RAM, read / write control module, BLD shift register, pipeline data module, BLD register, BLD separation module, second complement to original code module, second original code to complement module, comparator, adder, and BLD threshold module.

[0063] The read / write control module in the first convolutional layer module writes pixel I1(x) into its own RAM1 for storage, and writes the original data in pixel RAM1 into pixel RAM2. At the same time, it stores I1(x), the original data in pixel RAM1 and the original data in pixel RAM2 in its own pipeline data module, thereby obtaining a 3×3 convolution kernel and then passing it into its own BLD separation module.

[0064] In practice, a complete read / write operation of a data storage RAM group requires three cycles. When the CLK count is 0, a read operation is performed, that is, the read / write control module assigns pixel I1(x) to Data3, reads the corresponding address data from RAM2 to Data2, and reads the corresponding address data from RAM1 to Data1. When the CLK count is 1, a write operation is performed, that is, the value of Data3 is written to RAM2, and the value of Data2 is written to RAM1. When the CLK count is 2, no read / write operation is performed. Every three cycles, a 3×3 convolution kernel replacement is performed.

[0065] The BLD decomposition module in the first convolutional layer module decomposes the 3×3 convolutional kernel into R-channel, G-channel, and B-channel convolutional kernels. Then, it performs two-complementary code conversion on each kernel using its own second original code to two-complementary code module to obtain two-complementary R-channel, G-channel, and B-channel convolutional kernels. These kernels are then sent to the BLD register for storage. The weights in the three BLD weight RAMs are then multiplied by the corresponding two-complementary R-channel, G-channel, and B-channel convolutional kernels in the BLD register to obtain the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights. After shifting, the three shifted weights are sent to the second two-complementary code to original code module for original code conversion. The original code R-channel, G-channel, and B-channel convolutional kernel weights are then accumulated with the bias in the BLD bias RAM to obtain the first layer parameter K1(x), which is then transmitted to the comparator.

[0066] The comparator compares K1(x) with the threshold of the BLD thresholding module. In this example, the threshold is set to 0. If the result is greater than the threshold, the sum of K1(x) is output. Otherwise, 0 is output, and K1(x) is input into the second convolutional layer module and the first pipeline data RAM respectively.

[0067] The second convolutional layer module processes K1(x) according to the process of the first convolutional layer module, and obtains the second layer parameter K2(x). K2(x) is then input into the third convolutional layer module and the second pipeline data RAM respectively. After processing by the third convolutional layer module, the third layer parameter K3(x) is obtained and input into the first combination layer module. The first pipeline data RAM transmits the stored K1(x) to the first combination layer module.

[0068] The first combined layer module merges K1(x) and K3(x) to obtain the first combined layer parameter K. a(x) is transmitted to the fourth convolutional layer module for processing to obtain the fourth layer parameter K4(x). Then, it is input into the fifth convolutional layer module and the third pipeline data RAM respectively. After processing by the fifth convolutional layer module, the fifth layer parameter K5(x) is obtained. Then, it is input into the sixth convolutional layer module and the fourth pipeline data RAM respectively. After processing by the sixth convolutional layer module, the sixth layer parameter result K6(x) is obtained and input into the second combination layer module.

[0069] The third pipeline data RAM transmits the stored K4(x) to the second combination layer module; the second combination layer module merges K4(x) and K6(x) to obtain the second combination layer parameter K. b (x) is then transmitted to the seventh convolutional layer module for processing, and the sixth layer parameter result K7(x) is then input into the third combination layer module;

[0070] The second pipeline data RAM transmits the stored K2(x) to the third combination layer module; the fourth pipeline data RAM transmits the stored K5(x) to the third combination layer module; the third combination layer module merges K2(x), K5(x), and K7(x) to obtain the third combination layer parameter K. c (x) is then transmitted to the eighth convolutional layer module for processing to obtain the image reconstruction parameters K(x).

[0071] In this example, such as Figure 3 As shown, the pipeline timing diagram illustrates the cyclic time series of the pipeline data module in the BLD-Net module;

[0072] Where N is the square of the kernel side length of each convolutional layer, L is the width of the input image, H is the height of the input image, and r is the period. S1, S2, and S3 represent (L+1), (2L+2), and (3L+3) periods, respectively, which are the number of periods the previous layer runs when building the next pipeline. In the pre-storage stage, the weights and biases required by each convolutional layer are stored in the weight RAM and bias RAM, respectively. In the pipeline data waiting stage, each convolutional layer will be paused until the previous layer runs a specific number of periods. In the pipeline loop stage, each convolutional layer will be in a pipelined state.

[0073] Every three cycles, a 3*3 convolution kernel is replaced. A row of L columns of images requires 3(L+1) cycles to complete the convolution operation of a row of image data. Here, 3L is the convolution cycle required for image convolution, and the extra three cycles are the cycles required for convolution line wrapping. In this example, the value of L is 640.

[0074] In this example, such as Figure 4As shown, the hardware implementation of ASM includes: an ASM separation module, a first original code to two's complement module, a first two's complement to original code module, three sets of ASM shift registers, three ASM bias RAMs, an ASM merging module, and three ASM buffers;

[0075] In the ASM module, the ASM decomposition module decomposes the image reconstruction parameters K(x) obtained from the BLD-Net module to obtain R-channel components R(x), G-channel components G(x), and B-channel components B(x). The three-channel components of K(x) are then converted to their original codes using the first two's complement to original code module. These original codes are then multiplied by the corresponding three-channel components of the blurred image stored in the second image RAM module to obtain the initial components of the three channels. Finally, these initial components are added to the corresponding R-channel components R(x), G-channel components G(x), and B-channel components B(x) to obtain the R-channel component K(x). R (x), G channel component K G (x), B channel component K B (x), K R (x), K G (x), K B After (x) is cached, it is subtracted from the corresponding biases stored in the three ASM bias RAMs, and then transmitted to the three sets of ASM shift registers for right shift operation. The resulting three-channel initial dehazed image components are then transmitted to the first original code to two's complement module for conversion. After obtaining the two's complement of the three channels, they are transmitted to the merging module. The ASM merging module merges the two's complement of the three channels into pixel P(x) in the initial dehazed image P and sends it to the second sampling module for luminance component sampling. The luminance component of pixel P(x) in the initial dehazed image P is obtained and transmitted to the BI module.

[0076] In this example, such as Figure 5 As shown, the hardware implementation of BI includes: two BI separation modules, a BI weight storage module, a BI bias storage module, a BI threshold storage module, a brightness factor module, a BI buffer module, and two sets of BI shift registers.

[0077] The first BI separation module decomposes the luminance component of P(x) into luminance components of R channel, G channel and B channel, and the second BI separation module decomposes the pixel I2(x) into blur components of R channel, G channel and B channel.

[0078] The BI weight storage module stores the weights corresponding to the R, G, and B channels respectively. The luminance components of P(x) in the R, G, and B channels are multiplied by their respective weights to obtain the first weight R component W for each of the three channels. R First weighted component G W GFirst weighted component B W B Take W respectively R W G W B The middle and high 24 bits are registered, and then the high 24 bits of the three channels are accumulated with the bias in the BI bias storage module to obtain the first bias component B1, which is then transmitted to the BI threshold storage module.

[0079] The BI threshold storage module includes: two sets of shift registers, a minimum threshold storage module, and two sets of maximum threshold storage modules;

[0080] The first group of BI shift registers receives the first bias component B1 and performs a right shift operation to obtain the shifted first bias component, which is then transmitted to the maximum threshold module and the minimum threshold module respectively. The maximum threshold module compares the shifted first bias component with the maximum threshold L and retains the smaller value. The minimum threshold module compares the shifted first bias component with the minimum threshold M and retains the larger value. The higher eight bits of the larger value are added to the brightness factor in the brightness factor module to obtain a new brightness factor, which is then stored in the brightness factor module for the next accumulation calculation. This process continues until the image sampling is completed, at which point the finally stored brightness factor Light1 is sent to the divider.

[0081] Similarly, the blur components of the R, G, and B channels of pixel I2(x) are multiplied by the corresponding weights of the R, G, and B channels in the BI weight storage module to obtain the second weight R component W of the three channels. R2 The second weighted component G, W G2 The second weighted component B, W B2 Take W R W G W B The middle and high 24 bits are registered and then accumulated with the bias to obtain the second bias component B2, which is then transmitted to the BI threshold module for comparison to obtain the blur brightness factor Light2, which is then sent to the divider.

[0082] After dividing the initial image's luminance factor Light1 and blur luminance factor Light2, the resulting comprehensive luminance factor is multiplied by the luminance components of the R, G, and B channels of P(x) in the initial dehazed image P, respectively, to obtain the three-channel components F of the final dehazed image. R F G F B The image is then sent to the BI merging module for merging processing to obtain the final dehazed image F.

Claims

1. A lightweight network hardware accelerator for image dehazing, characterized in that, include: The system includes a blurred image storage module, two sampling modules, a BLD-Net module, an ASM module, a BI module, and an address control module. The blurred image storage module includes two image RAM modules, each used to store an identical blurred image. The first image RAM module transmits a blurred image to the first sampling module. The first sampling module converts the received blurred image into a blurred digital image I, and detects whether any pixel I(x) in the blurred digital image I belongs to a preset sampling point. If it does, I(x) is recorded as pixel I2(x) and transmitted to the BI module; otherwise, I(x) is recorded as pixel I1(x) and transmitted to the BLD-Net module. The BLD-Net module uses equation (1) to convert the received pixel I1(x) into image reconstruction parameters K(x) and passes them to the ASM module; In equation (1), t(x) represents the transfer function; A represents the atmospheric light value; and b represents the bias. The ASM module includes: an ASM separation module, a first original code to two's complement module, a first two's complement to original code module, three sets of ASM shift registers, three ASM bias RAMs, an ASM merging module, and three ASM buffers; The ASM separation module receives the image reconstruction parameters K(x) and decomposes them into three channel components, including: R channel component R(x), G channel component G(x), and B channel component B(x); The first complement-to-original code module converts the three-channel components into their original codes. Then, it multiplies these original codes by the corresponding three-channel components of the blurred image stored in the second image RAM module to obtain the initial three-channel components. These initial components are then added to the corresponding R-channel component R(x), G-channel component G(x), and B-channel component B(x) to obtain the R-channel component K. R (x), G channel component K G (x), B channel component K B (x); The three ASM buffers respectively target K R (x), K G (x), K B (x) After being cached, it is subtracted from the corresponding biases stored in the three ASM bias RAMs. The three results are then transmitted to the three sets of ASM shift registers for right shifting. The shifted components of the three channels are then transmitted to the first original code to two's complement module for conversion. After obtaining the two's complement of the three channels, they are transmitted to the ASM merging module. The ASM merging module merges the two's complement of the three channels into pixel P(x) in the initial dehazed image P and sends it to the second sampling module for luminance component sampling. The luminance component of pixel P(x) in the initial dehazed image P is then transmitted to the BI module. The BI module includes: two BI separation modules, a BI weight storage module, a BI bias storage module, a BI threshold storage module, a brightness factor module, a BI buffer module, and two sets of BI shift registers; The first BI separation module decomposes the luminance component of P(x) into luminance components of the R channel, G channel, and B channel; The second BI separation module decomposes pixel I2(x) into blur components of the R channel, G channel, and B channel; The BI weight storage module stores the weights corresponding to the R, G, and B channels respectively. The luminance components of P(x) in the R, G, and B channels are multiplied by their respective weights to obtain the first weight R component W for each of the three channels. R First weighted component G W G First weighted component B W B ; The BI cache module takes W respectively R W G W B The middle and high 24 bits are registered, and then the high 24 bits of the three channels are accumulated with the bias in the bias storage module to obtain the first bias component B1, which is then transmitted to the BI threshold storage module. The BI threshold storage module includes: two sets of shift registers, a minimum threshold storage module, and two sets of maximum threshold storage modules; The first group of BI shift registers receives the first bias component B1 and performs a right shift operation to obtain the shifted first bias component, which is then transmitted to the maximum threshold module and the minimum threshold module respectively. The maximum threshold module compares the shifted first bias component with the maximum threshold L and retains the larger value; The minimum threshold module compares the shifted first bias component with the minimum threshold M and retains the larger value; The higher eight bits of the larger of the two larger values ​​are added to the brightness factor in the brightness factor module to obtain a new brightness factor, which is then stored in the brightness factor module for the next accumulation calculation. This process continues until the image sampling is completed, at which point the finally stored brightness factor Light1 is sent to the divider. The BI weight storage module multiplies the blur components of the R, G, and B channels of pixel I2(x) with the corresponding weights of the R, G, and B channels in the BI weight storage module to obtain the second weight R component W of the three channels. R2 The second weighted component G, W G2 The second weighted component B, W B2 ; The BI cache module takes W respectively R W G W B The middle and high 24 bits are registered and then accumulated with the bias to obtain the second bias component B2. This component is then transmitted to the BI threshold module for comparison and processing to obtain the fuzzy brightness factor Light2, which is then sent to the divider. The divider divides the luminance factor Light1 and the blur luminance factor Light2, and then multiplies the resulting combined luminance factor by the luminance components of the R, G, and B channels of P(x) in the initial dehazed image P, respectively, to obtain the three-channel components F of the final dehazed image. R F G F B The image is then sent to the BI merging module for merging processing to obtain the final dehazed image F.

2. The lightweight network hardware accelerator for image dehazing according to claim 1, characterized in that, The BLD-Net module includes: eight convolutional layer modules, three combination layer modules, and four pipeline data RAMs; Each convolutional layer module includes: pixel RAM1, pixel RAM2, three BLD weight RAMs, BLD bias RAM, read / write control module, BLD shift register, pipeline data module, BLD register, BLD separation module, second complement to original code module, second original code to complement module, comparator, adder, and BLD threshold module. The read / write control module in the first convolutional layer module writes pixel I1(x) into its own RAM1 for storage, and writes the original data in pixel RAM1 into pixel RAM2. At the same time, it stores I1(x), the original data in pixel RAM1 and the original data in pixel RAM2 in its own pipeline data module, thereby obtaining a 3×3 convolution kernel and then passing it into its own BLD separation module. The BLD separation module in the first convolutional layer module decomposes the 3×3 convolutional kernel into an R-channel convolutional kernel, a G-channel convolutional kernel, and a B-channel convolutional kernel. Then, through its own second original code to two's complement module, it performs two's complement conversion to obtain the R-channel convolutional kernel, G-channel convolutional kernel, and B-channel convolutional kernel in two's complement form, and then sends them to the BLD register for storage. The three BLD weight RAMs in the first convolutional layer module multiply their own weights by the corresponding R-channel convolutional kernel, G-channel convolutional kernel, and B-channel convolutional kernel in two's complement form in the BLD register, respectively, to obtain the R-channel convolutional kernel weight, G-channel convolutional kernel weight, and B-channel convolutional kernel weight; The BLD shift register in the first convolutional layer module shifts the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights respectively. The resulting three shifted weights are then sent to its own second complement to original code module for original code conversion to obtain the original code form of the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights. The adder in the first convolutional layer module accumulates the R-channel convolutional kernel weights, G-channel convolutional kernel weights, and B-channel convolutional kernel weights in their original form with the bias in the bias RAM to obtain the first layer parameter K1(x) and transmits it to the first layer comparator. The comparator in the first convolutional layer module compares K1(x) with the threshold of the BLD threshold module. If K1(x) is greater than the threshold, K1(x) is output; otherwise, 0 is output. K1(x) is then input into the second convolutional layer module and the first pipeline data RAM, respectively. After the second convolutional layer module processes K1(x) according to the process of the first convolutional layer module, it obtains the second layer parameter K2(x), which is then input into the third convolutional layer module and the second pipeline data RAM respectively. After the third convolutional layer module performs the operation and processing, it obtains the third layer parameter K3(x), which is then input into the first combination layer module. The first pipeline data RAM transmits the stored K1(x) to the first combination layer module; The first combined layer module merges K1(x) and K3(x) to obtain the first combined layer parameter K. a (x) is transmitted to the fourth convolutional layer module for processing to obtain the fourth layer parameter K4(x). Then, it is input into the fifth convolutional layer module and the third pipeline data RAM respectively. After processing by the fifth convolutional layer module, the fifth layer parameter K5(x) is obtained. Then, it is input into the sixth convolutional layer module and the fourth pipeline data RAM respectively. After processing by the sixth convolutional layer module, the sixth layer parameter result K6(x) is obtained and input into the second combination layer module. The third pipeline data RAM transmits the stored K4(x) to the second combination layer module; The second combined layer module merges K4(x) and K6(x) to obtain the second combined layer parameter K. b (x) is then transmitted to the seventh convolutional layer module for processing, and the sixth layer parameter result K7(x) is then input into the third combination layer module; The second pipeline data RAM transmits the stored K2(x) to the third combination layer module; The fourth pipeline data RAM transfers the stored K5(x) to the third combination layer module; After merging K2(x), K5(x), and K7(x) in the third combination layer module, the parameter K of the third combination layer is obtained. c (x) is then transmitted to the eighth convolutional layer module for processing to obtain the image reconstruction parameters K(x).

Images