A lightweight neural network data processing method and system

Abstract

This invention discloses a lightweight neural network data processing method and system, belonging to the field of image reconstruction technology. The invention first performs multi-scale feature extraction on a blurred image to obtain highlight and low-brightness feature images at the largest, medium, and smallest scales. Next, texture coefficients are extracted from each pixel of the highlight and low-brightness feature images, and these texture coefficients are used to enhance the two types of feature images, resulting in enhanced highlight and low-brightness images. Then, a texture attention matrix is ​​obtained for each scale of the enhanced image. Finally, an image reconstruction neural network is used to process the multi-scale enhanced images, and attention is applied based on the corresponding scale's texture attention matrix, thereby obtaining the reconstructed image. This invention effectively improves the texture restoration accuracy in the image reconstruction process by combining multi-scale feature extraction, texture coefficient analysis, and an attention mechanism.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, specifically to a lightweight neural network data processing method and system. Background Technology

[0002] Image data is widely used in various fields, such as security monitoring, medical imaging, and autonomous driving. However, the images actually acquired are often affected by various factors, such as noise interference, changes in lighting, and limitations of the shooting equipment, resulting in blurred images and degraded quality, making it difficult to meet the needs of subsequent analysis and processing.

[0003] Traditional image processing methods have limitations when dealing with blurred images in complex scenes, failing to fully extract effective image features and resulting in poor enhancement effects. While deep learning technology has led to significant advancements in image enhancement through neural networks, large neural networks typically require substantial computational resources and storage space, making them difficult to run in real-time on resource-constrained devices.

[0004] Lightweight neural networks have broad application prospects in resource-constrained environments due to their fewer parameters and lower computational cost. However, limited by their limited computing power and model capacity, they are unable to fully explore the complex relationships between image features when processing blurred images, unlike large neural networks. This leads to the loss of details during image reconstruction and low accuracy in texture restoration. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a lightweight neural network data processing method and system that solves the problem of low texture restoration accuracy in existing image reconstruction techniques based on lightweight neural networks.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a lightweight neural network data processing method, comprising the following steps:

[0007] Multi-scale feature extraction is performed on the blurred image to obtain highlight feature images and low-brightness feature images at the maximum, medium, and minimum scales.

[0008] Extract texture coefficients for each pixel in the bright feature image and the low-brightness feature image respectively;

[0009] Based on the texture coefficient of each pixel, the highlight feature image and the low brightness feature image are enhanced separately to obtain the highlight enhanced image and the low brightness enhanced image.

[0010] Obtain the texture attention matrix for the highlight-enhanced image and the low-brightness-enhanced image at each scale;

[0011] An image reconstruction neural network is used to process multi-scale highlight enhancement images and low-brightness enhancement images. Attention is applied based on the texture attention matrix of the corresponding scale to obtain the reconstructed image.

[0012] Furthermore, the process of obtaining the highlight feature images and low-brightness feature images at the maximum, medium, and minimum scales includes:

[0013] Divide the blurred image into multiple 2×2 regions, multiple 3×3 regions, and multiple 4×4 regions respectively;

[0014] Extract the maximum pixel value in a 2×2 region, and arrange the maximum pixel values ​​according to their corresponding region positions to obtain the maximum scale highlight feature image;

[0015] Extract the minimum pixel value in a 2×2 region, and arrange the minimum pixel values ​​according to their corresponding region positions to obtain the low-brightness feature image at the largest scale;

[0016] Extract the maximum pixel value from a 3×3 area, and arrange the maximum pixel values ​​according to their corresponding area positions to obtain a medium-scale highlight feature image.

[0017] Extract the minimum pixel value from a 3×3 area, and arrange the minimum pixel values ​​according to their corresponding area positions to obtain a medium-scale low-brightness feature image.

[0018] Extract the maximum pixel value from a 4×4 area, and arrange the maximum pixel values ​​according to their corresponding area positions to obtain the minimum scale highlight feature image.

[0019] Extract the minimum pixel value from a 4×4 area, and arrange the minimum pixel values ​​according to their corresponding area positions to obtain the minimum scale low-brightness feature image.

[0020] Furthermore, the process of extracting texture coefficients includes:

[0021] Calculate the horizontal and vertical gradients of each pixel in the highlight and low-brightness feature images, respectively.

[0022] The horizontal and vertical gradients of each pixel are summed and normalized to obtain the texture coefficient of each pixel.

[0023] Furthermore, the process of obtaining the highlight-enhanced image and the low-brightness-enhanced image includes:

[0024] The pixel value of each pixel is enhanced based on the texture coefficient of each pixel to obtain the enhanced pixel value of each pixel;

[0025] Arrange the enhanced pixel values ​​corresponding to the highlighted feature image according to the pixel position to obtain the highlighted enhanced image;

[0026] The enhanced pixel values ​​corresponding to the low-brightness feature image are arranged according to the pixel position to obtain the low-brightness enhanced image.

[0027] Furthermore, the enhancement process includes: adding 1 to the texture coefficient of each pixel to obtain the enhancement coefficient of each pixel, and multiplying the enhancement coefficient of each pixel by the pixel value of the corresponding pixel to obtain the enhanced pixel value of each pixel.

[0028] Furthermore, the process of obtaining the texture attention matrix includes;

[0029] In the highlighted image, with each pixel as the center, calculate the difference in pixel value between the central pixel and other pixels within a 3×3 neighborhood. Take the absolute value of the 8 differences and sum them to obtain the first pixel difference.

[0030] In the low-brightness enhancement image, with each pixel as the center, calculate the difference in pixel value between the central pixel and other pixels within a 3×3 neighborhood. Take the absolute value of the 8 differences and sum them to obtain the second pixel difference.

[0031] Calculate the texture attention of a pixel based on the first pixel difference and the second pixel difference at the same pixel location;

[0032] Arrange the texture attention according to the pixel position to obtain the texture attention matrix.

[0033] Furthermore, the formula for calculating the texture attention of a pixel is: , where γ i For texture attention of the i-th pixel, E 1,i E is the first pixel difference of the i-th pixel. 2,i Let be the second pixel difference of the i-th pixel, where i is a positive integer.

[0034] Furthermore, the image reconstruction neural network includes: a first feature fusion reconstruction unit, a second feature fusion reconstruction unit, a third feature fusion reconstruction unit, adder A1, adder A2, a Concat layer, a first residual unit, a second residual unit, and a convolutional output unit;

[0035] The first feature fusion and reconstruction unit is used as input for the maximum scale highlight enhancement image, low brightness enhancement image, and texture attention matrix; the second feature fusion and reconstruction unit is used as input for the medium scale highlight enhancement image, low brightness enhancement image, and texture attention matrix; the third feature fusion and reconstruction unit is used as input for the minimum scale highlight enhancement image, low brightness enhancement image, and texture attention matrix.

[0036] The input terminals of adder A1 are connected to the first output terminals of the first feature fusion and reconstruction unit, the second feature fusion and reconstruction unit, and the third feature fusion and reconstruction unit, respectively; the input terminals of adder A2 are connected to the second output terminals of the first feature fusion and reconstruction unit, the second feature fusion and reconstruction unit, and the third feature fusion and reconstruction unit, respectively.

[0037] The input of the Concat layer is connected to the output of adder A1 and adder A2, respectively, and its output is connected to the input of the first residual unit. The input of the second residual unit is connected to the output of the first residual unit, and its output is connected to the input of the convolution output unit. The output of the convolution output unit serves as the output of the image reconstruction neural network.

[0038] Furthermore, the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, and the third feature fusion reconstruction unit each include: a first convolutional layer, a second convolutional layer, a multiplier M1, a multiplier M2, a first-scale reconstruction module, and a second-scale reconstruction module;

[0039] The input of the first convolutional layer is used to input the highlight enhancement image; the input of the second convolutional layer is used to input the low brightness enhancement image; the first input of multiplier M1 is connected to the output of the first convolutional layer, its second input is used to input the texture attention matrix, and its output is connected to the input of the first scale reconstruction module; the first input of multiplier M2 is connected to the output of the second convolutional layer, its second input is used to input the texture attention matrix, and its output is connected to the input of the second scale reconstruction module.

[0040] The output of the first-scale reconstruction module serves as the first output of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, or the third feature fusion reconstruction unit; the output of the second-scale reconstruction module serves as the second output of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, or the third feature fusion reconstruction unit.

[0041] A lightweight neural network data processing system includes: a multi-feature extraction subsystem, a texture extraction subsystem, a texture enhancement subsystem, a matrix acquisition subsystem, and a reconstruction subsystem;

[0042] The multi-feature extraction subsystem is used to extract features from blurred images at multiple scales, resulting in high-brightness and low-brightness feature images at the maximum, medium, and minimum scales.

[0043] The texture extraction subsystem is used to extract texture coefficients for each pixel in the bright feature image and the low-brightness feature image, respectively.

[0044] The texture enhancement subsystem is used to enhance the highlight feature image and the low-brightness feature image respectively according to the texture coefficient of each pixel, so as to obtain the highlight enhanced image and the low-brightness enhanced image;

[0045] The matrix acquisition subsystem is used to acquire the texture attention matrix for the highlight-enhanced image and the low-brightness-enhanced image at each scale;

[0046] The reconstruction subsystem is used to process multi-scale highlight enhancement images and low brightness enhancement images using an image reconstruction neural network, and obtains the reconstructed image based on the attention applied by the texture attention matrix of the corresponding scale.

[0047] The beneficial effects of this invention are as follows: This invention performs multi-scale feature extraction on blurred images to obtain multi-scale bright and low-brightness feature images. This facilitates image reconstruction by referencing image features at different scales and then enhancing the images based on pixel texture coefficients, resulting in richer detail in both the bright and low-brightness enhanced images. Furthermore, a texture attention matrix is ​​obtained for each scale of the bright and low-brightness enhanced images, enabling the network to intelligently focus on high-value texture regions, suppress irrelevant information, and improve reconstruction accuracy.

[0048] This invention forms multi-scale texture information through the synergistic effect of multi-scale features and attention matrix, and also achieves adaptive feature fusion through attention guidance, avoiding the loss of texture details during image reconstruction and improving the accuracy of texture restoration. Attached Figure Description

[0049] Figure 1 A flowchart of a lightweight neural network data processing method;

[0050] Figure 2 A schematic diagram of the structure of a neural network for image reconstruction;

[0051] Figure 3 This is a schematic diagram of the structure of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, and the third feature fusion reconstruction unit;

[0052] Figure 4 This is a schematic diagram of the structure of the first-scale reconstruction module and the second-scale reconstruction module;

[0053] Figure 5 This is a schematic diagram of the structure of the first residual unit and the second residual unit;

[0054] Figure 6 This is a schematic diagram of a blurred image;

[0055] Figure 7 The image is the reconstructed image obtained after reconstruction using the scheme of the present invention. Detailed Implementation

[0056] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0057] Example 1, such as Figure 1 As shown, a lightweight neural network data processing method includes the following steps:

[0058] Multi-scale feature extraction is performed on the blurred image to obtain highlight feature images and low-brightness feature images at the maximum, medium, and minimum scales.

[0059] Extract texture coefficients for each pixel in the bright feature image and the low-brightness feature image respectively;

[0060] Based on the texture coefficient of each pixel, the highlight feature image and the low brightness feature image are enhanced separately to obtain the highlight enhanced image and the low brightness enhanced image.

[0061] Obtain the texture attention matrix for the highlight-enhanced image and the low-brightness-enhanced image at each scale;

[0062] An image reconstruction neural network is used to process multi-scale highlight enhancement images and low-brightness enhancement images. Attention is applied based on the texture attention matrix of the corresponding scale to obtain the reconstructed image.

[0063] In this embodiment, the process of obtaining the highlight feature images and low-brightness feature images at the maximum, medium, and minimum scales includes:

[0064] Divide the blurred image into multiple 2×2 regions, multiple 3×3 regions, and multiple 4×4 regions respectively;

[0065] Extract the maximum pixel value from a 2×2 region, arrange the maximum pixel values ​​according to their corresponding region positions to obtain the maximum scale highlight feature image; extract the minimum pixel value from a 2×2 region, arrange the minimum pixel values ​​according to their corresponding region positions to obtain the maximum scale low brightness feature image.

[0066] Extract the maximum pixel value from a 3×3 region, arrange the maximum pixel values ​​according to their corresponding region positions to obtain a medium-scale highlight feature image; extract the minimum pixel value from a 3×3 region, arrange the minimum pixel values ​​according to their corresponding region positions to obtain a medium-scale low-brightness feature image.

[0067] Extract the maximum pixel value from a 4×4 region, arrange the maximum pixel values ​​according to their corresponding region positions to obtain the minimum scale highlight feature image; extract the minimum pixel value from a 4×4 region, arrange the minimum pixel values ​​according to their corresponding region positions to obtain the minimum scale low-brightness feature image.

[0068] The size of the blurred image is 1920×1080, the size of the largest-scale highlight feature image and the size of the low-scale feature image are 960×540, the size of the medium-scale highlight feature image and the medium-scale low-scale feature image are 640×360, and the size of the smallest-scale highlight feature image and the smallest-scale low-scale feature image are 480×270.

[0069] This invention divides a blurred image into multiple 2×2 regions, multiple 3×3 regions, and multiple 4×4 regions, which facilitates the extraction of the most significant and least significant pixel values ​​from different scales. The most significant pixel values ​​reflect the most significant structure of the image, while the least significant pixel values ​​reflect the shadow structure of the image.

[0070] Extracting pixel values ​​at different scales allows for better recovery of texture information in images. Small-scale regions help capture subtle texture changes, while medium- and large-scale regions ensure structural integrity and better reflect macroscopic texture information. The ability to preserve texture information is significantly improved throughout the process, helping to eliminate blurring effects and increase the accuracy of texture reconstruction.

[0071] In this embodiment, the process of extracting texture coefficients includes:

[0072] Calculate the horizontal and vertical gradients of each pixel in the highlight and low-brightness feature images, respectively.

[0073] The horizontal and vertical gradients of each pixel are summed and normalized to obtain the texture coefficient of each pixel.

[0074] In this embodiment, the process of obtaining the highlight-enhanced image and the low-brightness-enhanced image includes:

[0075] The pixel value of each pixel is enhanced based on the texture coefficient of each pixel to obtain the enhanced pixel value of each pixel;

[0076] Arrange the enhanced pixel values ​​corresponding to the highlighted feature image according to the pixel position to obtain the highlighted enhanced image;

[0077] The enhanced pixel values ​​corresponding to the low-brightness feature image are arranged according to the pixel position to obtain the low-brightness enhanced image.

[0078] The formula for calculating the vertical gradient is: Among them, G H(n,m) represents the vertical gradient at pixel (n,m) in the image, P(n,m+1) represents the pixel value at pixel (n,m+1) in the image, and P(n,m) represents the pixel value at pixel (n,m) in the image.

[0079] The formula for calculating the horizontal gradient is: Among them, G L (n,m) represents the horizontal gradient at pixel (n,m) in the image, P(n+1,m) represents the pixel value at pixel (n+1,m) in the image, and n and m are the coordinate numbers of the pixel.

[0080] In this embodiment, the texture coefficient of a pixel is equal to , || represents the absolute value operation, and R is a constant used for normalization.

[0081] In this embodiment, the enhancement process includes: adding 1 to the texture coefficient of each pixel to obtain the enhancement coefficient of each pixel, and multiplying the enhancement coefficient of each pixel by the pixel value of the corresponding pixel to obtain the enhanced pixel value of each pixel.

[0082] The enhancement formula is: ,in, The enhancement factor for each pixel. The pixel value of a pixel. The enhanced pixel value for each pixel, where i is the pixel number.

[0083] This invention dynamically adjusts pixel values ​​based on texture coefficients, specifically enhancing texture-rich areas (high coefficients) and less enhancing smooth areas (low coefficients), thereby enhancing texture details while avoiding noise in solid-color backgrounds due to blind enhancement.

[0084] In this embodiment, after enhancement, if the pixel value exceeds 255, the maximum pixel value of the enhanced image is extracted, and the ratio of 255 to the maximum pixel value is taken, that is, 255 divided by the maximum pixel value, to obtain the scaling factor. The pixel value of each pixel in the enhanced image is multiplied by the scaling factor so that the pixel value of the enhanced image is between 0 and 255.

[0085] This invention enhances both the high-brightness and low-brightness feature images, improving the texture details of both features and making it easier for the image reconstruction neural network to reference more prominent texture features during reconstruction.

[0086] In this embodiment, the process of obtaining the texture attention matrix includes:

[0087] In the highlighted image, with each pixel as the center, calculate the difference in pixel value between the central pixel and other pixels within a 3×3 neighborhood. Take the absolute value of the 8 differences and sum them to obtain the first pixel difference.

[0088] In the low-brightness enhancement image, with each pixel as the center, calculate the difference in pixel value between the central pixel and other pixels within a 3×3 neighborhood. Take the absolute value of the 8 differences and sum them to obtain the second pixel difference.

[0089] Calculate the texture attention of a pixel based on the first pixel difference and the second pixel difference at the same pixel location;

[0090] Arrange the texture attention according to the pixel position to obtain the texture attention matrix.

[0091] In this embodiment, the formula for calculating the texture attention of a pixel is: , where γ i For texture attention of the i-th pixel, E 1,i E is the first pixel difference of the i-th pixel. 2,i Let be the second pixel difference of the i-th pixel, where i is a positive integer.

[0092] This invention calculates texture attention on the enhanced image to reflect the texture intensity at that location, which facilitates the application of attention to the corresponding features in the image reconstruction neural network, increases the attention to areas with rich texture, and enables the image reconstruction neural network to reconstruct the image with higher accuracy.

[0093] like Figure 2 As shown, the image reconstruction neural network includes: a first feature fusion reconstruction unit, a second feature fusion reconstruction unit, a third feature fusion reconstruction unit, adder A1, adder A2, a Concat layer, a first residual unit, a second residual unit, and a convolutional output unit;

[0094] The first feature fusion and reconstruction unit is used as input for the maximum scale highlight enhancement image, low brightness enhancement image, and texture attention matrix; the second feature fusion and reconstruction unit is used as input for the medium scale highlight enhancement image, low brightness enhancement image, and texture attention matrix; the third feature fusion and reconstruction unit is used as input for the minimum scale highlight enhancement image, low brightness enhancement image, and texture attention matrix.

[0095] The input terminals of adder A1 are connected to the first output terminals of the first feature fusion and reconstruction unit, the second feature fusion and reconstruction unit, and the third feature fusion and reconstruction unit, respectively; the input terminals of adder A2 are connected to the second output terminals of the first feature fusion and reconstruction unit, the second feature fusion and reconstruction unit, and the third feature fusion and reconstruction unit, respectively.

[0096] The input of the Concat layer is connected to the output of adder A1 and adder A2, respectively, and its output is connected to the input of the first residual unit. The input of the second residual unit is connected to the output of the first residual unit, and its output is connected to the input of the convolution output unit. The output of the convolution output unit serves as the output of the image reconstruction neural network.

[0097] This invention employs three feature fusion and reconstruction units to perform feature fusion of the highlight enhancement image, low brightness enhancement image, and texture attention matrix at each scale, as well as image scale reconstruction. Adder A1 adds the corresponding features of the highlight enhancement image at the largest scale, the highlight enhancement image at the medium scale, and the highlight enhancement image at the smallest scale. Adder A2 adds the corresponding features of the low brightness enhancement image at the largest scale, the low brightness enhancement image at the medium scale, and the low brightness enhancement image at the smallest scale. This allows for fusion of image features reconstructed at different scales, improving the accuracy of texture restoration. The outputs of adder A1 and adder A2 are then concatenated. The gradient vanishing problem is avoided by using the first residual unit and the second residual unit. Finally, the output is generated through a convolutional output unit.

[0098] like Figure 3 As shown, the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, and the third feature fusion reconstruction unit all include: a first convolutional layer, a second convolutional layer, a multiplier M1, a multiplier M2, a first-scale reconstruction module, and a second-scale reconstruction module;

[0099] The input of the first convolutional layer is used to input the highlight enhancement image; the input of the second convolutional layer is used to input the low brightness enhancement image; the first input of multiplier M1 is connected to the output of the first convolutional layer, its second input is used to input the texture attention matrix, and its output is connected to the input of the first scale reconstruction module; the first input of multiplier M2 is connected to the output of the second convolutional layer, its second input is used to input the texture attention matrix, and its output is connected to the input of the second scale reconstruction module.

[0100] The output of the first-scale reconstruction module serves as the first output of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, or the third feature fusion reconstruction unit; the output of the second-scale reconstruction module serves as the second output of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, or the third feature fusion reconstruction unit.

[0101] This invention incorporates a "texture attention matrix" at each scale, allowing the reconstruction process to focus on real texture areas (rather than noise / smooth areas). For example, in highlight features, high attention values ​​at edges enhance their details, while in low-brightness features, shadow textures are specifically repaired.

[0102] Since the same pixel location in a highlight-enhanced image and a low-brightness-enhanced image of the same scale originates from the same region in a blurred image, this invention uses a texture attention matrix to characterize the "texture complexity and importance of the region". The texture complexity (such as whether there are wrinkles or patterns) of the same physical region is fixed, and a matrix is ​​used to describe the common attention requirement of the region for "highlight and low-brightness enhancement".

[0103] like Figure 4 As shown, the first-scale reconstruction module and the second-scale reconstruction module include, in sequence, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, and an upsampling layer.

[0104] The kernel size of the first, second, and third convolutional layers is 1×1;

[0105] The kernel size of both the fourth and fifth convolutional layers is 3×3.

[0106] The upsampling layer in the first feature fusion and reconstruction unit, the upsampling layer in the second feature fusion and reconstruction unit, and the upsampling layer in the third feature fusion and reconstruction unit all upsample the feature map to the size of the original blurred image.

[0107] In this embodiment, both the first residual unit and the second residual unit include: a sixth convolutional layer, a seventh convolutional layer, an eighth convolutional layer, and an adder A3. The kernel size of the sixth and seventh convolutional layers is 3×3, and the kernel size of the eighth convolutional layer is 1×1. Figure 5 As shown.

[0108] The convolutional output unit consists of a 3×3 convolutional layer and a 1×1 convolutional layer connected in sequence.

[0109] Figure 6 For a blurred image of the eye region, the technical solution of this invention is used to obtain the following: Figure 7 The reconstructed image shown demonstrates how, after processing, details such as the eyes become clear and identifiable, thus optimizing the visual effect.

[0110] Example 2: A lightweight neural network data processing system, comprising: a multi-feature extraction subsystem, a texture extraction subsystem, a texture enhancement subsystem, a matrix acquisition subsystem, and a reconstruction subsystem;

[0111] The multi-feature extraction subsystem is used to extract features from blurred images at multiple scales, resulting in high-brightness and low-brightness feature images at the maximum, medium, and minimum scales.

[0112] The texture extraction subsystem is used to extract texture coefficients for each pixel in the bright feature image and the low-brightness feature image, respectively.

[0113] The texture enhancement subsystem is used to enhance the highlight feature image and the low-brightness feature image respectively according to the texture coefficient of each pixel, so as to obtain the highlight enhanced image and the low-brightness enhanced image;

[0114] The matrix acquisition subsystem is used to acquire the texture attention matrix for the highlight-enhanced image and the low-brightness-enhanced image at each scale;

[0115] The reconstruction subsystem is used to process multi-scale highlight enhancement images and low brightness enhancement images using an image reconstruction neural network, and obtains the reconstructed image based on the attention applied by the texture attention matrix of the corresponding scale.

[0116] The specific implementation process of Example 2 is the same as that of Example 1.

[0117] In this invention, a lightweight neural network is achieved by using only a small number of convolutional layer images for clear reconstruction.

[0118] This invention performs multi-scale feature extraction on blurred images to obtain multi-scale highlight and low-brightness feature images. This facilitates image reconstruction by referencing image features at different scales and then enhancing the images based on pixel texture coefficients, resulting in richer detail in both the enhanced highlight and low-brightness images. Furthermore, a texture attention matrix is ​​obtained for each scale of the enhanced highlight and low-brightness images, enabling the network to intelligently focus on high-value texture regions, suppress irrelevant information, and improve reconstruction accuracy.

[0119] This invention forms multi-scale texture information through the synergistic effect of multi-scale features and attention matrix, and also achieves adaptive feature fusion through attention guidance, avoiding the loss of texture details during image reconstruction and improving the accuracy of texture restoration.

[0120] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight neural network data processing method, characterized in that, Includes the following steps: Multi-scale feature extraction is performed on the blurred image to obtain highlight feature images and low-brightness feature images at the maximum, medium, and minimum scales. Extract texture coefficients for each pixel in the bright feature image and the low-brightness feature image respectively; Based on the texture coefficient of each pixel, the highlight feature image and the low brightness feature image are enhanced separately to obtain the highlight enhanced image and the low brightness enhanced image. Obtain the texture attention matrix for the highlight-enhanced image and the low-brightness-enhanced image at each scale; An image reconstruction neural network is used to process multi-scale highlight enhancement images and low brightness enhancement images. Attention is applied based on the texture attention matrix of the corresponding scale to obtain the reconstructed image. The image reconstruction neural network includes: a first feature fusion reconstruction unit, a second feature fusion reconstruction unit, a third feature fusion reconstruction unit, adder A1, adder A2, a Concat layer, a first residual unit, a second residual unit, and a convolutional output unit; The first feature fusion and reconstruction unit is used as input for the maximum scale highlight enhancement image, low brightness enhancement image, and texture attention matrix; the second feature fusion and reconstruction unit is used as input for the medium scale highlight enhancement image, low brightness enhancement image, and texture attention matrix; the third feature fusion and reconstruction unit is used as input for the minimum scale highlight enhancement image, low brightness enhancement image, and texture attention matrix. The input terminals of adder A1 are connected to the first output terminals of the first feature fusion and reconstruction unit, the second feature fusion and reconstruction unit, and the third feature fusion and reconstruction unit, respectively; the input terminals of adder A2 are connected to the second output terminals of the first feature fusion and reconstruction unit, the second feature fusion and reconstruction unit, and the third feature fusion and reconstruction unit, respectively. The first output of the first feature fusion and reconstruction unit is used to output the feature corresponding to the maximum scale of the high-brightness enhanced image, and the second output of the first feature fusion and reconstruction unit is used to output the feature corresponding to the maximum scale of the low-brightness enhanced image. The first output of the second feature fusion and reconstruction unit is used to output the corresponding features of the high-brightness enhanced image at a medium scale, and the second output of the second feature fusion and reconstruction unit is used to output the corresponding features of the low-brightness enhanced image at a medium scale. The first output of the third feature fusion and reconstruction unit is used to output the feature corresponding to the highlight enhancement image at the smallest scale, and the second output of the third feature fusion and reconstruction unit is used to output the feature corresponding to the low brightness enhancement image at the smallest scale. The input of the Concat layer is connected to the output of adder A1 and adder A2, respectively, and its output is connected to the input of the first residual unit. The input of the second residual unit is connected to the output of the first residual unit, and its output is connected to the input of the convolution output unit. The output of the convolution output unit serves as the output of the image reconstruction neural network.

2. The lightweight neural network data processing method according to claim 1, characterized in that, The process of obtaining highlight and low-brightness feature images at the maximum, intermediate, and minimum scales includes: Divide the blurred image into multiple 2×2 regions, multiple 3×3 regions, and multiple 4×4 regions respectively; Extract the maximum pixel value from a 2×2 region, arrange the maximum pixel values ​​according to their corresponding region positions to obtain the maximum scale highlight feature image; extract the minimum pixel value from a 2×2 region, arrange the minimum pixel values ​​according to their corresponding region positions to obtain the maximum scale low brightness feature image. Extract the maximum pixel value from a 3×3 region, arrange the maximum pixel values ​​according to their corresponding region positions to obtain a medium-scale highlight feature image; extract the minimum pixel value from a 3×3 region, arrange the minimum pixel values ​​according to their corresponding region positions to obtain a medium-scale low-brightness feature image. Extract the maximum pixel value from a 4×4 region, arrange the maximum pixel values ​​according to their corresponding region positions to obtain the minimum scale highlight feature image; extract the minimum pixel value from a 4×4 region, arrange the minimum pixel values ​​according to their corresponding region positions to obtain the minimum scale low-brightness feature image.

3. The lightweight neural network data processing method according to claim 1, characterized in that, The process of extracting texture coefficients includes: Calculate the horizontal and vertical gradients of each pixel in the highlight and low-brightness feature images, respectively. The horizontal and vertical gradients of each pixel are summed and normalized to obtain the texture coefficient of each pixel.

4. The lightweight neural network data processing method according to claim 1, characterized in that, The process of obtaining highlight-enhanced and low-brightness-enhanced images includes: The pixel value of each pixel is enhanced based on the texture coefficient of each pixel to obtain the enhanced pixel value of each pixel; Arrange the enhanced pixel values ​​corresponding to the highlighted feature image according to the pixel position to obtain the highlighted enhanced image; The enhanced pixel values ​​corresponding to the low-brightness feature image are arranged according to the pixel position to obtain the low-brightness enhanced image.

5. The lightweight neural network data processing method according to claim 4, characterized in that, The enhancement process includes: adding 1 to the texture coefficient of each pixel to obtain the enhancement coefficient of each pixel, and multiplying the enhancement coefficient of each pixel by the pixel value of the corresponding pixel to obtain the enhanced pixel value of each pixel.

6. The lightweight neural network data processing method according to claim 1, characterized in that, The process of obtaining the texture attention matrix includes: In the highlighted image, with each pixel as the center, calculate the difference in pixel value between the central pixel and other pixels within a 3×3 neighborhood. Take the absolute value of the 8 differences and sum them to obtain the first pixel difference. In the low-brightness enhancement image, with each pixel as the center, calculate the difference in pixel value between the central pixel and other pixels within a 3×3 neighborhood. Take the absolute value of the 8 differences and sum them to obtain the second pixel difference. Calculate the texture attention of a pixel based on the first pixel difference and the second pixel difference at the same pixel location; Arrange the texture attention according to the pixel position to obtain the texture attention matrix.

7. The lightweight neural network data processing method according to claim 6, characterized in that, The formula for calculating the texture attention of a pixel is: , where γ i For texture attention of the i-th pixel, E 1,i E is the first pixel difference of the i-th pixel. 2,i Let be the second pixel difference of the i-th pixel, where i is a positive integer.

8. The data processing method for a lightweight neural network according to claim 1, characterized in that, The first feature fusion reconstruction unit, the second feature fusion reconstruction unit, and the third feature fusion reconstruction unit each include: a first convolutional layer, a second convolutional layer, a multiplier M1, a multiplier M2, a first-scale reconstruction module, and a second-scale reconstruction module; The input of the first convolutional layer is used to input the highlight enhancement image; the input of the second convolutional layer is used to input the low brightness enhancement image; the first input of multiplier M1 is connected to the output of the first convolutional layer, its second input is used to input the texture attention matrix, and its output is connected to the input of the first scale reconstruction module; the first input of multiplier M2 is connected to the output of the second convolutional layer, its second input is used to input the texture attention matrix, and its output is connected to the input of the second scale reconstruction module. The output of the first-scale reconstruction module serves as the first output of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, or the third feature fusion reconstruction unit; the output of the second-scale reconstruction module serves as the second output of the first feature fusion reconstruction unit, the second feature fusion reconstruction unit, or the third feature fusion reconstruction unit.

9. A lightweight neural network data processing system, implemented based on the lightweight neural network data processing method according to any one of claims 1 to 8, characterized in that, include: Multi-feature extraction subsystem, texture extraction subsystem, texture enhancement subsystem, matrix acquisition subsystem, and reconstruction subsystem; The multi-feature extraction subsystem is used to extract features from blurred images at multiple scales, resulting in high-brightness and low-brightness feature images at the maximum, medium, and minimum scales. The texture extraction subsystem is used to extract texture coefficients for each pixel in the bright feature image and the low-brightness feature image, respectively. The texture enhancement subsystem is used to enhance the highlight feature image and the low-brightness feature image respectively according to the texture coefficient of each pixel, so as to obtain the highlight enhanced image and the low-brightness enhanced image; The matrix acquisition subsystem is used to acquire the texture attention matrix for the highlight-enhanced image and the low-brightness-enhanced image at each scale; The reconstruction subsystem is used to process multi-scale highlight enhancement images and low brightness enhancement images using an image reconstruction neural network, and obtains the reconstructed image based on the attention applied by the texture attention matrix of the corresponding scale.

Images