RAW domain image edge enhancement method and device based on lightweight neural network

Abstract

The present invention discloses a method and device for edge enhancement of RAW domain images based on a lightweight neural network, the method comprising: inputting a RAW format image to be sharpened into a trained convolutional neural network model, classifying the input RAW format image with a classification module, and outputting group information corresponding to the image; then, according to the group information, calling a corresponding set of convolutional neural network weight parameters in an external memory, combining with the sharpening module structure in the convolutional neural network model, sharpening the RAW format image, and outputting an RGB format image. The present invention replaces the traditional image sharpening algorithm with a convolutional neural network, which can combine the advantages of four traditional image sharpening algorithms and is suitable for different types of scenes. In addition, the present invention lightweights the convolutional neural network algorithm, reduces the amount of calculation, and provides the possibility of realizing edge computing.

Description

Technical Field

[0001] The present invention relates to an image processing method and device, and in particular to a RAW domain image edge enhancement method and device. Background Art

[0002] Due to the limited computing power of edge devices, in order to solve the problem of blurred image edges, traditional image sharpening algorithms are generally used instead of convolutional neural network algorithms to process images. There are mainly the following traditional image sharpening algorithms:

[0003] 1. Robert Cross: This algorithm calculates the sharpened pixel value by weighted averaging the four neighborhoods around the pixel. Its advantages are simple calculation, fast speed, and suitable for real-time processing. The disadvantage is that it is sensitive to noise and may produce some artifacts, such as noise, aliasing, and artifacts, and is not suitable for images with complex textures.

[0004] Sobel sharpening: Sobel sharpening algorithm is a gradient-based sharpening algorithm that enhances edges by calculating the horizontal and vertical gradients of the image. Its advantage is that it has a strong ability to detect edges and is applicable to edges in various directions. Its disadvantage is that compared with other algorithms, it is also more sensitive to noise and may produce some false edges.

[0005] Laplacian sharpening: Laplacian sharpening algorithm is a sharpening algorithm based on second-order derivatives. It enhances the details of the image by calculating the grayscale difference around the pixel. Its advantage is that the generated sharpening effect is obvious and can enhance the details of the image. The disadvantage is that it will introduce noise, causing obvious burrs and artifacts in the image.

[0006] 4. Unsharp Masking: Unsharp Masking is a hybrid algorithm based on blurring and sharpening. It obtains a sharpened result by subtracting the original image from the Gaussian blurred image. Its advantage is that it can reduce the influence of noise. Its disadvantage is that for some images, the sharpening effect may be too strong or too weak, and the parameters need to be adjusted to obtain the desired result.

[0007] These algorithms each have their own application scenarios: Robert cross sharpening algorithm is suitable for fast processing, Sobel sharpening algorithm is suitable for enhancing edges, Laplace sharpening algorithm is suitable for retaining details, and unsharp mask algorithm is suitable for reducing the impact of noise.

[0008] However, there is currently no algorithm that can adapt to the limited computing power of edge devices and combine the advantages of these four traditional image sharpening algorithms. Summary of the invention

[0009] Purpose of the invention: In view of the above-mentioned prior art, a RAW domain image edge enhancement method based on a lightweight neural network is proposed to solve the shortcomings of the limited application scenarios of the traditional image sharpening algorithm, and at the same time, the convolutional neural network algorithm is lightweight to reduce the amount of calculation, which makes it possible to realize edge computing. Another purpose of the present invention is to propose a RAW domain image edge enhancement device based on a lightweight neural network.

[0010] Technical solution: A RAW domain image edge enhancement method based on a lightweight neural network, comprising: inputting a RAW format image to be sharpened into a trained convolutional neural network model, the first stage processing process of the model comprising: using a classification module to classify the input RAW format image, and outputting group information corresponding to the image; the second stage processing process of the model comprising: according to the group information, calling a corresponding set of convolutional neural network weight parameters in an external memory, combining the set of weight parameters with the sharpening module structure in the convolutional neural network model, sharpening the RAW format image, and outputting the processed RGB format image.

[0011] Furthermore, the group information data is a four-digit array, each digit represents the probability that the image belongs to the corresponding group, and it is defined that: the image with group information [0,0,0,1] is most suitable for processing by Robert's cross sharpening algorithm, the image with group information [0,0,1,0] is most suitable for processing by Sobel sharpening algorithm, the image with group information [0,1,0,0] is most suitable for processing by Laplace sharpening algorithm, and the image with group information [1,0,0,0] is most suitable for processing by unsharp masking algorithm.

[0012] Furthermore, in the convolutional neural network model, the classification module includes a 5-layer structure: the 1st to 4th layers are all convolutional layers, each convolutional layer is composed of a Conv layer and a ReLu layer connected, the size of the convolution kernel in the Conv layer is 3*3, and the step size is 2; the 5th layer includes two fully connected layers, which flattens the feature map into a one-dimensional vector, performs weighted summation on each element in the vector, and then uses a softmax activation function to convert the output into a one-dimensional array as the group information.

[0013] Furthermore, in the convolutional neural network model, the sharpening module has a 13-layer structure: the first layer is a preprocessing layer, which uses a convolution kernel with a size of 1*1 and a step size of 1 to extract R, G, and B in the RAW format image as separate channels;

[0014] The 2nd to 9th layers are all convolutional layers, where every 2 convolutional layers form a convolutional operation module, that is, the module includes a feature extraction convolutional layer and a feature fusion convolutional layer; the feature extraction convolutional layer is composed of a Conv layer and a ReLu layer connected, the size of the convolution kernel in the Conv layer is 3*3 and the step size is 2, which is used to perform a convolution operation on each input channel using a separate convolution kernel to obtain an intermediate feature map with the same number of input channels; the feature fusion convolutional layer uses a convolution kernel with a size of 1*1 and a step size of 1, which is used to multiply and add each position in the intermediate feature map with the corresponding convolution kernel to generate a final output feature map;

[0015] The 10th to 13th layers are all deconvolution layers. Each layer is composed of a Conv layer with a deconvolution kernel of size 3*3 and a stride of 2 connected to a ReLu layer, which is used to upsample and restore the input feature map.

[0016] Furthermore, the training process of the convolutional neural network model includes:

[0017] Establish a training set: collect several RAW format images, classify each collected image into groups and attach corresponding group information, then process each image using the corresponding sharpening algorithm according to the group information to obtain the processed image;

[0018] Training the classification module: input the unsharpened images in the training set, output the group information data in each round of training, and compare it with the group information data attached to the image, and use the cross entropy loss function to calculate the loss value Loss 1. According to the loss value Loss 1. Perform back propagation to optimize the weight of the classification module;

[0019] Training the sharpening module: establishing 4 sets of convolutional neural network weight parameters for the sharpening module, each corresponding to a group information; inputting the unsharpened image in the training set and the accompanying group information data, calling the corresponding set of weight parameters for training according to the group information of the image; after each round of training, using the gradient difference between each pixel and its eight neighboring pixels in the sharpened image in the training set and the corresponding image output by the sharpening module to calculate the loss value Loss 2. According to the loss value Loss 2. Back propagation is performed to optimize the four sets of weight parameters of the sharpening module; after the sharpening module completes the training, the four sets of weight parameters of the convolutional neural network are stored in the external memory.

[0020] Furthermore, in the step of establishing a training set, the specific method for grouping each collected image includes: processing the image once with Robert cross sharpening algorithm, Sobel sharpening algorithm, Laplace sharpening algorithm and unsharp masking algorithm respectively to obtain 4 sharpened images, and then calculating the signal-to-noise ratio of the 4 images respectively, and according to the sharpening algorithm used for the image with the highest signal-to-noise ratio, the original image is divided into the group corresponding to the algorithm.

[0021] Furthermore, the Loss 1 is represented by:

[0022] , where N Indicates the total number of RAW images collected. P n1 , P n2 , P n3 , P n4 They represent the first, second, third, and fourth elements in the array of the group information data attached to the nth image in the training set; Q n1 , Q n2 , Q n3 , Q n4 They respectively represent the 1st, 2nd, 3rd, and 4th elements in the array of the group information data output after the nth image in the training set passes through the classification module.

[0023] Furthermore, of the two images with the same image content in the training set, the original image without sharpening processing is named T1, the image after sharpening processing is named T2, and the image output after T1 is processed by the sharpening module is named T3; Loss 2 is expressed as: Loss 2=1 / 3( LossR + LossG + LossB ),in LossR , LossG , LossB Respectively represent the loss values ​​of RGB format images T2 and T3 in the three channels of R, G, and B.

[0024] Furthermore, for an image with an input size of H*H, LossR It is expressed as:

[0025] , where T 2 R ijis the gradient difference between the pixel in the i-th row and j-th column in the R channel of the RGB format image T2 and its eight neighboring pixels; T 3 R ij It is the gradient difference between the pixel in the i-th row and j-th column in the R channel of the RGB format image T3 and its eight neighboring pixels.

[0026] A RAW domain image edge enhancement device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the RAW domain image edge enhancement method based on a lightweight neural network.

[0027] Beneficial effects: 1. The present invention uses a convolutional neural network to replace the traditional image sharpening algorithm, which can integrate the advantages of four traditional image sharpening algorithms (Robert cross sharpening algorithm, Laplace sharpening algorithm, Sobel sharpening algorithm, and unsharp mask algorithm), and is suitable for different application scenarios such as fast processing, edge enhancement, detail retention, and noise reduction. In addition, the present invention can improve the shortcomings of some traditional image sharpening algorithms, such as noise, jagged edges, artifacts, and false edges.

[0028] Lightweighting the convolutional neural network algorithm makes it possible to realize edge computing. The lightweighting can be achieved in the following two ways:

[0029] (1) Each convolutional layer of the traditional convolutional neural network is replaced by a feature extraction convolutional layer and a feature fusion convolutional layer. By combining these two layers, features can be effectively extracted and fused, replacing the function of a traditional convolutional layer, thereby reducing the amount of computation and the number of model parameters. It is suitable for scenarios that need to run in a resource-constrained environment, such as real-time image processing and recognition tasks on mobile devices.

[0030] The reason why the amount of calculation can be reduced is that a traditional convolution operation step is equivalently replaced by two steps with less calculation, feature fusion and feature extraction. For example, for an input feature map with a size of H * W, an input channel of Cin, an output channel of Cout, and a convolution kernel size of k*k, the amount of calculation for traditional convolution is Cin*Cout*k^2*H*W. After replacing traditional convolution with "feature extraction" + "feature fusion", in the feature extraction step, the convolution operation is performed channel by channel, that is, each input channel will independently apply a convolution kernel, and each convolution kernel is only responsible for the convolution operation of one channel. The amount of calculation is Cin*k^2*H*W, because each channel is independently convolved and there is no Cout product. Then in the feature fusion step, Cout 1x1 convolution kernels are used to convolve the outputs of all channels. The purpose of this step is to mix and combine the features of different channels to generate a new feature map. The amount of calculation is Cin*Cout*1^2*H*W. Since the convolution kernel size is 1x1, k^2 becomes 1.

[0031] Therefore, after replacing the traditional convolution with "feature extraction" + "feature fusion", the total computational complexity is Cin*H*W(k^2+Cout), which is much smaller than the computational complexity of the traditional convolution Cin*H*W*k^2*Cout.

[0032] At the same time, this method uses Cin different k*k convolution kernels to extract features from different input channels, and uses Cout different 1x1 convolution kernels to fuse the extracted features according to their weights and distribute them to different output channels, thus realizing most of the functions of the traditional convolution layer.

[0033] (2) While the overall structure of the convolutional neural network remains unchanged, four sets of optional parameters are provided. These parameters are stored in external memory and do not occupy cache resources at runtime. Therefore, it is possible to meet more diverse computing needs under limited computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the structure of the classification module in the convolutional neural network model of the present invention;

[0035] Figure 2 Schematic diagram of the structure of the sharpening module in the convolutional neural network model of the present invention;

[0036] Figure 3 Flow chart of the method. DETAILED DESCRIPTION

[0037] The present invention will be further explained below in conjunction with the accompanying drawings.

[0038] A RAW domain image edge enhancement method based on a lightweight neural network, the specific steps are as follows:

[0039] Step 1: Create a training set.

[0040] Four groups of RAW images with rich scenes were collected, 1000 images in each group, and the pixel size of each image was 448*448. Each group of images corresponds to a group information data, and the format of the group information data is a 4-bit array.

[0041] For the images in group A, the group information data are all [0,0,0,1], indicating that they are most suitable for processing using the Robert cross sharpening algorithm; for the images in group B, the group information data are all [0,0,1,0], indicating that they are most suitable for processing using the Sobel sharpening algorithm; for the images in group C, the group information data are all [0,1,0,0], indicating that they are most suitable for processing using the Laplace sharpening algorithm; for the images in group D, the group information data are all [1,0,0,0], indicating that they are most suitable for processing using the unsharp masking algorithm.

[0042] These 4000 images were processed with the most suitable sharpening algorithm respectively, and the defects such as noise, aliasing, artifacts, false edges, etc. remaining after the sharpening algorithm processing were repaired with PS software to obtain 4000 processed images.

[0043] The established training set has a total of 8,000 images, that is, there are two images for the same scene content, one is the original image collected and the other is the image processed by the sharpening algorithm. Both images contain group information data.

[0044] In this embodiment, a sufficient number (greater than 4000) of RAW format images with rich scenes and a size of 448*448 are first collected, and each image is sharpened once using four algorithms (Robert cross sharpening algorithm, Sobel sharpening algorithm, Laplace sharpening algorithm and unsharp masking algorithm) to obtain four sharpened images, and then their signal-to-noise ratios are calculated respectively. According to the sharpening algorithm used for the image with the highest signal-to-noise ratio, the original image is divided into the group corresponding to the algorithm.

[0045] Step 2: Build a convolutional neural network model.

[0046] The convolutional neural network consists of two modules: a classification module and a sharpening module.

[0047] 1. Classification module: includes 5 layers of structure, such as Figure 1 shown.

[0048] The 1st to 4th layers are all convolutional layers, each of which is composed of a Conv layer and a ReLu layer. The size of the convolution kernel in the Conv layer is 3*3 and the step size is 2. The ReLU layer can increase the nonlinear ability of the network, allowing the network to learn more complex features.

[0049] The output sizes of the 1st to 4th layers are: 224*224*2, 112*112*4, 56*56*6, 28*28*8 respectively.

[0050] The fifth layer includes two fully connected layers. This layer flattens the feature map into a one-dimensional vector and then performs weighted summation on each element in the vector. The softmax activation function is then used to convert the output into a one-dimensional array, which contains four probability values ​​corresponding to four categories. This one-dimensional array is output as group information data.

[0051] The input size of the 5th layer is 28*28*8, the output size of the first fully connected layer in the 5th layer is 1*1*400, and the output size of the second fully connected layer in the 5th layer is a 4-bit array.

[0052] 2. Sharpening module: includes 13 layers of structure, such as Figure 2 shown.

[0053] The first layer is the preprocessing layer, which extracts R, G, and B in the RAW format image into separate channels by using a convolution kernel with a size of 1*1 and a step size of 1. Its input is a 448*448 size image and its output is a 448*448*3 size image.

[0054] The 2nd to 9th layers are all convolutional layers, where every two convolutional layers can form a convolution operation module, which includes a feature extraction convolution layer and a feature fusion convolution layer.

[0055] The feature extraction convolution layer uses a separate convolution kernel to perform convolution operations on each input channel to obtain an intermediate feature map with the same number of input channels. The feature extraction convolution layer is composed of a Conv layer and a ReLu layer. The size of the convolution kernel in the Conv layer is 3*3 and the step size is 2. This step performs a convolution operation on the input feature map to extract feature information.

[0056] The feature fusion convolution layer uses a convolution kernel with a size of 1*1 and a step size of 1 on the intermediate feature map of the previous step, multiplies each position in the intermediate feature map with the corresponding convolution kernel and adds them together to generate the final output feature map. This step can effectively reduce the number of channels in the feature map and reduce the number of parameters in the model.

[0057] The purpose of this operation is to replace each convolutional layer of the traditional convolutional neural network with a feature extraction convolutional layer and a feature fusion convolutional layer. By combining these two layers, features can be effectively extracted and fused, reducing the amount of computation while maintaining high performance.

[0058] The sizes of the output images from the 2nd to the 9th layers are: 224*224*6, 224*224*6, 112*112*12, 112*112*12, 56*56*24, 56*56*24, 28*28*48, 28*28*48.

[0059] The 10th to 13th layers are all deconvolution layers. Each layer consists of a Conv layer with a deconvolution kernel of size 3*3 and a stride of 2 connected to a ReLu layer to upsample and restore the input feature map. The output image sizes of the 10th to 13th layers are: 56*56*24, 112*112*12, 224*224*6, 448*448*3.

[0060] Step 3: Input the training set into the convolutional neural network model for training.

[0061] 1. Training of classification module.

[0062] Input 4000 unsharpened images from the training set, output group information data for each round of training, and compare it with the group information data attached to the image, and use the cross entropy loss function to calculate the loss value Loss 1:

[0063]

[0064] In the formula, P n1 , P n2 , P n3 , P n4 They represent the first, second, third, and fourth elements in the array of the group information data attached to the nth image in the training set; Q n1 , Q n2 , Q n3 , Q n4 They respectively represent the 1st, 2nd, 3rd, and 4th elements in the array of the group information data output after the nth image in the training set passes through the classification module.

[0065] According to the loss value Loss1. Back propagation is performed at a learning rate of 0.5% to optimize the weights of the classification module, and training is performed for 100 rounds in total.

[0066] 2. Training of sharpening module.

[0067] Four sets of convolutional neural network weight parameters are established for the sharpening module, named C1, C2, C3, and C4 respectively.

[0068] Input 4000 unsharpened images in the training set and the accompanying group information data. For images with group information [0,0,0,1], since they are most suitable for processing with Robert's cross sharpening algorithm, the weight parameters C1 are used for training; for images with group information [0,0,1,0], since they are most suitable for processing with Sobel sharpening algorithm, the weight parameters C2 are used for training; for images with group information [0,1,0,0], since they are most suitable for processing with Laplace sharpening algorithm, the weight parameters C3 are used for training; for images with group information [1,0,0,0], since they are most suitable for processing with unsharp masking algorithm, the weight parameters C4 are used for training.

[0069] Among the two images with the same image content in the training set, the original image without sharpening processing is named T1, the image after sharpening processing is named T2, and the image output after T1 is processed by the sharpening module is named T3.

[0070] After each round of training, the loss value is calculated by the gradient difference between each pixel in T2 and T3 and its eight neighboring pixels. Loss 2.

[0071] Among them, the loss value of RGB format images T2 and T3 in the R channel LossR for:

[0072]

[0073] In the formula, T 2 R ij is the gradient difference between the pixel in the i-th row and j-th column in the R channel of the RGB format image T2 and its eight neighboring pixels; T 3 R ij is the gradient difference between the pixel in the i-th row and j-th column in the R channel of the RGB format image T3 and its eight neighboring pixels. Similarly, we can get LossG and LossB . Then we get: Loss 2=1 / 3( LossR + LossG + LossB ).

[0074] According to the loss valueLoss 2. Back propagation is performed at a learning rate of 0.5% to optimize the four sets of weight parameters of the sharpening module, and training is performed for 100 rounds in total.

[0075] After the sharpening module completes the training, the weight parameters (C1, C2, C3, and C4) of the four sets of convolutional neural networks are stored in the external memory.

[0076] The purpose of this operation is to provide 4 sets of selectable weight parameters while keeping the overall structure of the convolutional neural network unchanged. Since these weight parameters are stored in external memory, they do not occupy cache resources at runtime. Therefore, it is possible to meet more diverse computing needs under limited computing power.

[0077] Step 4: Use the trained convolutional neural network.

[0078] like Figure 3 As shown, when used, the RAW format image that needs to be sharpened is input into the trained convolutional neural network model and then goes through a two-stage processing process of the model.

[0079] The first stage: Use the classification module in the convolutional neural network model to classify the input 448*448 RAW format image and output the group information data corresponding to the image. The group information data is a four-bit array, and each digit represents the probability that the image belongs to the corresponding group;

[0080] The second stage: According to the group information, determine the group to which the image belongs with the highest probability. Call the weight parameters corresponding to this group among the four groups of convolutional neural network weight parameters stored in the external memory, combine this group of weight parameters with the convolutional neural network structure of the sharpening module, sharpen the image, and output the processed 448*448*3 RGB format image.

[0081] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A RAW domain image edge enhancement method based on a lightweight neural network, characterized in that: include: The RAW format image to be sharpened is input into the trained convolutional neural network model. The first stage processing process of the model includes: using the classification module to classify the input RAW format image and outputting the group information corresponding to the image; the second stage processing process of the model includes: according to the group information, calling a corresponding set of convolutional neural network weight parameters in the external memory, combining the set of weight parameters with the sharpening module structure in the convolutional neural network model, sharpening the RAW format image, and outputting the processed RGB format image; The group information data is a four-digit array, each digit represents the probability that the image belongs to the corresponding group, and is defined as follows: images with group information [0,0,0,1] are most suitable for processing by the Robert cross sharpening algorithm, images with group information [0,0,1,0] are most suitable for processing by the Sobel sharpening algorithm, images with group information [0,1,0,0] are most suitable for processing by the Laplace sharpening algorithm, and images with group information [1,0,0,0] are most suitable for processing by the unsharp masking algorithm; based on the group information, determine the group to which the image has the highest probability of belonging.

2. The RAW domain image edge enhancement method based on lightweight neural network according to claim 1 is characterized in that: In the convolutional neural network model, the classification module includes a 5-layer structure: the 1st to 4th layers are all convolution layers, each convolution layer is composed of a Conv layer and a ReLu layer connected, the size of the convolution kernel in the Conv layer is 3*3, and the step size is 2; the 5th layer includes two fully connected layers, which flattens the feature map into a one-dimensional vector, performs weighted summation on each element in the vector, and then uses a softmax activation function to convert the output into a one-dimensional array as the group information.

3. The RAW domain image edge enhancement method based on lightweight neural network according to claim 2 is characterized in that: In the convolutional neural network model, the sharpening module has a 13-layer structure: the first layer is a preprocessing layer, which uses a convolution kernel with a size of 1*1 and a step size of 1 to extract R, G, and B in the RAW format image as separate channels; The 2nd to 9th layers are all convolutional layers, where every two convolutional layers form a convolutional operation module, that is, the module includes a feature extraction convolutional layer and a feature fusion convolutional layer; The feature extraction convolution layer is composed of a Conv layer and a ReLu layer connected together. The size of the convolution kernel in the Conv layer is 3*3 and the step size is 2, which is used to use a separate convolution kernel for each input channel to perform a convolution operation to obtain an intermediate feature map with the same number of input channels; the feature fusion convolution layer uses a convolution kernel with a size of 1*1 and a step size of 1, which is used to multiply and add each position in the intermediate feature map with the corresponding convolution kernel to generate a final output feature map; The 10th to 13th layers are all deconvolution layers. Each layer is composed of a Conv layer with a deconvolution kernel of size 3*3 and a stride of 2 connected to a ReLu layer, which is used to upsample and restore the input feature map.

4. The RAW domain image edge enhancement method based on lightweight neural network according to claim 3 is characterized in that: The training process of the convolutional neural network model includes: Establish a training set: collect several RAW format images, classify each collected image into groups and attach corresponding group information, then process each image using the corresponding sharpening algorithm according to the group information to obtain the processed image; Training the classification module: input the unsharpened images in the training set, output the group information data in each round of training, and compare it with the group information data attached to the image, and use the cross entropy loss function to calculate the loss value Loss 1. According to the loss value Loss 1. Perform back propagation to optimize the weight of the classification module; Training the sharpening module: establishing 4 sets of convolutional neural network weight parameters for the sharpening module, each corresponding to a group information; inputting the unsharpened image in the training set and the accompanying group information data, calling the corresponding set of weight parameters for training according to the group information of the image; after each round of training, using the gradient difference between each pixel and its eight neighboring pixels in the sharpened image in the training set and the corresponding image output by the sharpening module to calculate the loss value Loss 2. According to the loss value Loss 2. Back propagation is performed to optimize the four sets of weight parameters of the sharpening module; after the sharpening module completes the training, the four sets of weight parameters of the convolutional neural network are stored in the external memory.

5. The RAW domain image edge enhancement method based on lightweight neural network according to claim 4 is characterized in that: In the step of establishing a training set, the specific method for grouping each collected image includes: processing the image once with Robert cross sharpening algorithm, Sobel sharpening algorithm, Laplace sharpening algorithm and unsharp masking algorithm respectively to obtain 4 sharpened images, and then calculating the signal-to-noise ratio of the 4 images respectively, and according to the sharpening algorithm used for the image with the highest signal-to-noise ratio, the original image is divided into the group corresponding to the algorithm.

6. The RAW domain image edge enhancement method based on lightweight neural network according to claim 4 is characterized in that: Said Loss 1 is represented by: , where N Indicates the total number of RAW images collected. P n1 , P n2 , P n3 , P n4 They represent the first, second, third, and fourth elements in the array of the group information data attached to the nth image in the training set; Q n1 , Q n2 , Q n3 , Q n4 They respectively represent the 1st, 2nd, 3rd, and 4th elements in the array of the group information data output after the nth image in the training set passes through the classification module.

7. The RAW domain image edge enhancement method based on lightweight neural network according to claim 4, characterized in that: Of the two images with the same image content in the training set, the original image without sharpening processing is named T1, the image after sharpening processing is named T2, and the image output after T1 is processed by the sharpening module is named T3; Loss 2 is expressed as: Loss 2=1 / 3( LossR + LossG + LossB ),in LossR , LossG , LossB Respectively represent the loss values ​​of RGB format images T2 and T3 in the three channels of R, G, and B.

8. The RAW domain image edge enhancement method based on lightweight neural network according to claim 6, characterized in that: For an image with an input size of H*H, the LossR It is expressed as: , where T 2 R ij is the gradient difference between the pixel in the i-th row and j-th column in the R channel of the RGB format image T2 and its eight neighboring pixels; T 3 R ij It is the gradient difference between the pixel in the i-th row and j-th column in the R channel of the RGB format image T3 and its eight neighboring pixels.

9. A RAW domain image edge enhancement device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the RAW domain image edge enhancement method based on a lightweight neural network as described in any one of claims 1 to 8.

Images