Image processing method and device and storage medium

Abstract

The invention relates to an image processing method and device and a storage medium, and the method comprises the steps: obtaining a to-be-processed image which contains impurities; and background color removal processing is carried out on the to-be-processed image to obtain a target image, and the target image does not contain the impurities. According to the method, the to-be-processed image is subjected to background color removal processing, impurities in the to-be-processed image can be removed, the target image is obtained, the mode can be suitable for purification of different impurities, the complexity of an algorithm is reduced, the impurity purification cost is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] Currently, when taking images of documents, impurities such as shadows, moiré patterns, and noise may be generated in the captured document images. The impurities in the document images need to be purified to meet user needs.

[0003] There are currently different purification algorithms for different impurities. For example, for moiré patterns in document images, there are algorithms such as mrnet, RDNet, and MRGAN to solve them. For shadows in document images, there are algorithms such as BGSNet, DocShadow, and GCDRNet to solve them.

[0004] Thus, there are currently different purification algorithms for different impurities in document images, which undoubtedly greatly increases the complexity of the algorithm, increases the cost of impurity purification in document images, and reduces user experience. Summary of the invention

[0005] In order to solve the above-mentioned different impurities in document images, there are currently different purification algorithms, which undoubtedly greatly increases the complexity of the algorithm, increases the cost of impurity purification in document images, and reduces the technical problem of user experience. The embodiment of the present application provides an image processing method, device, electronic device and storage medium. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, an image processing method is first provided, the method comprising:

[0007] Acquire an image to be processed, wherein the image to be processed contains impurities;

[0008] Background color removal is performed on the image to be processed to obtain a target image, wherein the target image does not contain the impurities.

[0009] In an optional implementation, performing background color removal processing on the image to be processed to obtain a target image includes:

[0010] The pre-trained universal purification network is used to perform background color removal processing on the image to be processed to obtain a target image.

[0011] In an optional embodiment, the pre-trained universal purification network includes at least one pre-trained first background color removal network, a pre-trained second background color removal network, and a pre-trained third background color removal network;

[0012] The method of using a pre-trained universal purification network to remove background color from the image to be processed to obtain a target image includes:

[0013] In the case where the pre-trained universal purification network includes a pre-trained first background color removal network, the image to be processed is input into the pre-trained first background color removal network to obtain a first purified image feature; or, in the case where the pre-trained universal purification network includes two or more pre-trained first background color removal networks, the image to be processed is input into the pre-trained first background color removal networks in sequence to obtain a first purified image feature;

[0014] Inputting the first purified image feature into the pre-trained second background color removal network to obtain a second purified image feature;

[0015] The second cleaned image features are input into the pre-trained third background color removal network to obtain a target image.

[0016] In an optional embodiment, the pre-trained universal purification network includes at least one pre-trained first background color removal network and a pre-trained third background color removal network;

[0017] The method of using a pre-trained universal purification network to remove background color from the image to be processed to obtain a target image includes:

[0018] Inputting the image to be processed into the pre-trained first background color removal network in sequence to obtain a first purified image feature;

[0019] The first cleaned image feature is input into the pre-trained third background color removal network to obtain a target image.

[0020] In an optional embodiment, the pre-trained universal purification network further includes a pre-trained downsampling network and a pre-trained convolutional network;

[0021] The step of inputting the image to be processed into the pre-trained first background color removal network to obtain a first purified image feature includes:

[0022] Inputting the image to be processed into the pre-trained downsampling network to obtain the down-sampled image to be processed;

[0023] Inputting the downsampled image to be processed into the pre-trained convolutional network to obtain a first convolution result;

[0024] The first convolution result is input into the pre-trained first background color removal network to obtain a first purified image feature.

[0025] In an optional embodiment, the pre-trained first background color removal network includes M pre-trained first background color removal sub-networks, where M is a positive integer;

[0026] The step of inputting the first convolution result into the pre-trained first background color removal network to obtain a first purified image feature includes:

[0027] Iteratively perform the following operations until the Mth first impurity purification feature is obtained and then stop the following operations;

[0028] Inputting the i-th first impurity feature to be purified into the i-th pre-trained first background color removal sub-network to obtain the i-th first impurity purification feature;

[0029] Among them, the i-th first impurity purification feature is the i+1-th first impurity feature to be purified, the 1st first impurity feature to be purified is the first convolution result, the M-th first impurity purification feature is the first purified image feature, and i is 1, 2, 3, ..., M respectively.

[0030] In an optional embodiment, the i-th pre-trained first background color removal sub-network includes a pre-trained first residual dense block, N pre-trained first background color removal blocks, a pre-trained first convolutional layer, and a pre-trained first activation function layer, where N is a positive integer;

[0031] The step of inputting the i-th first impurity feature to be purified into the i-th pre-trained first background color removal sub-network to obtain the i-th first impurity purification feature comprises:

[0032] Inputting the i-th first impurity feature to be purified into the pre-trained first residual dense block to obtain a first enhanced image feature;

[0033] Iterate the following steps until the Nth first background color removal feature is obtained and then stop the following operations:

[0034] Inputting the jth first background color feature to be removed into the jth pre-trained first background color removal block to obtain the jth first background color removal feature;

[0035] The jth first background color removal feature is the j+1th first background color feature to be removed, the first first background color feature to be removed is the first enhanced image feature, and j is 1, 2, 3, ..., N in sequence;

[0036] Inputting the Nth first background color removal feature into the pre-trained first convolution layer to obtain a second convolution result;

[0037] The second convolution result is input into the pre-trained first activation function layer to obtain the i-th first impurity purification feature.

[0038] In an optional embodiment, the pre-trained first residual dense block includes O pre-trained first feature processing layers and a pre-trained second convolutional layer, where O is a positive integer;

[0039] The step of inputting the i-th first impurity feature to be purified into the pre-trained first residual dense block to obtain a first enhanced image feature includes:

[0040] Iterate and perform the following operations until the Oth residual feature is obtained and stop the following operations:

[0041] Inputting the kth feature to be enhanced into the kth pre-trained first feature processing layer to obtain the kth first initial feature;

[0042] Add the kth feature to be enhanced to the kth first initial feature to obtain the kth residual feature;

[0043] The kth residual feature is the k+1th feature to be enhanced, the 1st feature to be enhanced is the i-th first impurity feature to be purified, and k is 1, 2, 3, ..., 0 in sequence;

[0044] The Oth residual feature is input into the second convolutional layer of the pre-training to obtain the first enhanced image feature.

[0045] In an optional embodiment, the kth pre-trained first feature processing layer includes a pre-trained third convolution layer and a pre-trained second activation function layer;

[0046] The step of inputting the kth feature to be enhanced into the kth pre-trained first feature processing layer to obtain the kth first initial feature comprises:

[0047] Inputting the kth feature to be enhanced into the pre-trained third convolutional layer to obtain a third convolution result;

[0048] The third convolution result is input into the pre-trained second activation function layer to obtain the kth first initial feature.

[0049] In an optional embodiment, the jth pre-trained first background color removal block includes a pre-trained analog input layer of P scales and a pre-trained multi-scale fusion layer, where P is a positive integer;

[0050] The step of inputting the jth first background color feature to be removed into the jth pre-trained first background color removal block to obtain the jth first background color removal feature comprises:

[0051] Input the jth first background color feature to be removed into the vth scale pre-trained analog input layer to obtain the vth scale analog input feature, wherein v is 1, 2, 3, ..., P in sequence;

[0052] Inputting the simulated input features of P scales into the pre-trained multi-scale fusion layer to obtain fused features;

[0053] The j-th first background color feature to be removed is added to the fusion feature to obtain the j-th first background color removal feature.

[0054] In an optional embodiment, the pre-trained simulated input layer of the vth scale includes a pre-trained second residual dense block;

[0055] The step of inputting the jth first background color feature to be removed into the pre-trained analog input layer of the vth scale to obtain the analog input feature of the vth scale includes:

[0056] Determine the size reduction factor and size increase factor corresponding to the pre-trained simulated input layer of the vth scale;

[0057] According to the size reduction ratio, the j-th first background color feature to be removed is reduced in size to obtain the reduced j-th first background color feature to be removed;

[0058] Inputting the reduced j-th first background color feature to be removed into the pre-trained second residual dense block to obtain a second enhanced feature;

[0059] The second enhanced feature is scaled up according to the scale-up factor to obtain a simulated input feature of the vth scale.

[0060] In an optional embodiment, the pre-trained multi-scale fusion layer includes P pre-trained average pooling layers, L pre-trained second feature processing layers, a pre-trained fourth convolution layer, and a pre-trained third activation function layer:

[0061] The step of inputting the simulated input features of P scales into the pre-trained multi-scale fusion layer to obtain fused features includes:

[0062] Inputting the simulated input features of the vth scale into the vth pre-trained average pooling layer to obtain the vth average pooling result;

[0063] Add the P average pooling results to obtain an initial fusion feature;

[0064] Iterate and perform the following operations until the Lth second initial feature is obtained and then stop the following operations;

[0065] Input the tth feature to be processed into the tth pre-trained second processing feature layer to obtain the tth second initial feature;

[0066] Among them, the t-th second initial feature is the t+1-th feature to be processed, the 1st feature to be processed is the initial fusion feature, and t is 1, 2, 3, ..., L in sequence;

[0067] Input the Lth second initial feature into the pre-trained fourth convolution layer to obtain the fourth convolution result;

[0068] Inputting the fourth convolution result into the pre-trained third activation function layer to obtain weights corresponding to the simulated input features of P scales;

[0069] According to the weights, weighted summation is performed on the simulated input features of P scales to obtain fused features.

[0070] In an optional embodiment, the pre-trained second background color removal network includes a pre-trained third residual dense block and N pre-trained second background color removal blocks;

[0071] The step of inputting the first purified image feature into the pre-trained second background color removal network to obtain a second purified image feature comprises:

[0072] Inputting the first purified image feature into the pre-trained third residual dense block to obtain a third enhanced feature;

[0073] Iterate the following operations until the Nth second background color removal feature is obtained and then stop the following operations:

[0074] Inputting the jth second background color feature to be removed into the jth pre-trained second background color removal block to obtain the jth second background color removal feature;

[0075] Among them, the jth second background color removal feature is the j+1th second background color feature to be removed, the 1st first background color feature to be removed is the third enhancement feature, the Nth second background color removal feature is the second purified image feature, and j is 1, 2, 3,..., N respectively.

[0076] In an optional embodiment, the pre-trained third background color removal network includes M+1 pre-trained second background color removal sub-networks;

[0077] The step of inputting the second purified image feature into the pre-trained third background color removal network to obtain a target image comprises:

[0078] Iterate the following operations until the first second impurity purification feature is obtained and then stop the following operations:

[0079] Inputting the i-th second impurity feature to be purified into the i-th pre-trained second background color removal sub-network to obtain the i-th second impurity purification feature;

[0080] Among them, the i-th second impurity purification feature is the i-1-th second impurity feature to be purified, the M+1-th second impurity feature to be purified is the second purified image feature, the 1st second impurity purification feature is the target image, and i is M+1,...,3,2,1 respectively.

[0081] In an optional embodiment, the i-th pre-trained second background color removal sub-network includes a pre-trained fourth residual dense block, N pre-trained third background color removal blocks, a pre-trained fifth convolutional layer, a pre-trained sixth convolutional layer and a pre-trained upsampling layer;

[0082] The step of inputting the i-th second impurity feature to be purified into the i-th pre-trained second background color removal sub-network to obtain the i-th second impurity purification feature comprises:

[0083] Inputting the i-th second impurity feature to be purified into the pre-trained fifth convolution layer to obtain a fifth convolution result;

[0084] Inputting the fifth convolution result into the pre-trained fourth residual dense block to obtain a fourth enhanced feature;

[0085] Iterate the following operations until the Nth third background color removal feature is obtained and then stop the following operations:

[0086] Inputting the jth third background feature to be removed into the jth pre-trained third background color removal block to obtain the jth third background removal feature;

[0087] The jth third background removal feature is the j+1th third background feature to be removed, the first third background feature to be removed is the fourth enhanced feature, and j is 1, 2, 3, ..., N in sequence;

[0088] Inputting the Nth third background color removal feature into the pre-trained sixth convolution layer to obtain a sixth convolution result;

[0089] The sixth convolution result is input into the pre-trained upsampling layer to obtain the i-th second impurity purification feature.

[0090] In an optional implementation, the step of inputting the i-th second impurity feature to be purified into the pre-trained fifth convolution layer to obtain a fifth convolution result includes:

[0091] Determine an i-th pre-trained first background color removal sub-network at the same level as the i-th pre-trained second background color removal sub-network;

[0092] Add the i-th first impurity purification feature output by the i-th pre-trained first background color removal subnetwork to the i-th second impurity feature to be purified to obtain a short connection feature;

[0093] The short connection feature is input into the pre-trained fifth convolutional layer to obtain a fifth convolution result.

[0094] In an optional implementation, before executing the image processing method, the method further includes:

[0095] Acquire an image pair set, wherein the image pairs in the image pair set include a sample image and a standard image, and the impurities contained in the sample image cover different impurities;

[0096] Acquire the image pair from the image pair set, input the sample image in the image pair into a universal purification network, and obtain a predicted image;

[0097] Determine a loss value between the predicted image and the standard image in the image pair, and perform network training on the universal purification network according to the loss value;

[0098] When the loss value converges, the network training is stopped to obtain a pre-trained universal purification network.

[0099] In an optional embodiment, the step of inputting the sample image in the image pair into a universal purification network to obtain a predicted image includes:

[0100] Cropping the sample image in the image pair to obtain at least one image sub-region;

[0101] For any of the image sub-regions, compress the image sub-region to obtain a compressed image sub-region;

[0102] Inputting the compressed image sub-region into a universal purification network to obtain a predicted image sub-region;

[0103] The determining a loss value between the predicted image and the standard image in the image pair comprises:

[0104] searching, from the standard image of the image pair, a standard image sub-region corresponding to the image sub-region;

[0105] A loss value between the standard image sub-region and the predicted image sub-region is determined.

[0106] In an optional embodiment, the universal purification network includes a first background color removal network, a second background color removal network, and a third background color removal network;

[0107] The step of inputting the compressed image sub-region into a universal purification network to obtain a predicted image sub-region comprises:

[0108] Inputting the compressed image sub-region into the first background color removal network to obtain a first predicted purified image feature;

[0109] Inputting the first predicted purified image feature into the second background color removal network to obtain a second predicted purified image feature;

[0110] The second predicted cleaned image feature is input into the third background color removal network to obtain a predicted image sub-region.

[0111] In an optional embodiment, the universal purification network further includes a downsampling network and a convolutional network;

[0112] The step of inputting the compressed image sub-region into the first background color removal network to obtain a first predicted purified image feature includes:

[0113] Inputting the compressed image sub-region into the down-sampling network to obtain the compressed image sub-region after down-sampling;

[0114] Inputting the compressed image sub-region after downsampling into the convolution network to obtain a first predicted convolution result;

[0115] The first predicted convolution result is input into the first background color removal network to obtain a first predicted purified image feature.

[0116] In an optional embodiment, the first background color removal network includes M first background color removal sub-networks, where M is a positive integer;

[0117] The step of inputting the first predicted convolution result into the first background color removal network to obtain a first predicted purified image feature includes:

[0118] Iteratively perform the following operations until the Mth first predicted impurity purification feature is obtained and then stop the following operations;

[0119] Inputting the i-th first predicted impurity feature to be purified into the i-th first background color removal sub-network to obtain the i-th first predicted impurity purification feature;

[0120] Among them, the i-th first predicted impurity purification feature is the i+1-th first predicted impurity feature to be purified, the 1st first predicted impurity feature to be purified is the first predicted convolution result, the M-th first predicted impurity purification feature is the first predicted purified image feature, and i is 1, 2, 3,..., M respectively.

[0121] In an optional embodiment, the i-th first background color removal sub-network includes a first residual dense block, N first background color removal blocks, a first convolutional layer, and a first activation function layer, where N is a positive integer;

[0122] The step of inputting the i-th first predicted impurity feature to be purified into the i-th first background color removal subnetwork to obtain the i-th first predicted impurity purification feature comprises:

[0123] Inputting the i-th first predicted impurity feature to be purified into the first residual dense block to obtain a first predicted enhanced image feature;

[0124] Iterate the following steps until the Nth first predicted background color removal feature is obtained and then stop the following operations:

[0125] Inputting the jth first predicted background color feature to be removed into the jth first background color removal block to obtain the jth first predicted background color removal feature;

[0126] The jth first predicted background color removal feature is the j+1th first predicted background color feature to be removed, the first first predicted background color feature to be removed is the first predicted enhanced image feature, and j is 1, 2, 3, ..., N in sequence;

[0127] Inputting the Nth first predicted background color removal feature into the first convolution layer to obtain a second predicted convolution result;

[0128] The second predicted convolution result is input into the first activation function layer to obtain the i-th first predicted impurity purification feature.

[0129] In an optional embodiment, the first residual dense block includes O first feature processing layers and a second convolution layer, where O is a positive integer;

[0130] The step of inputting the i-th first predicted impurity feature to be purified into the first residual dense block to obtain a first predicted enhanced image feature includes:

[0131] Iterate and perform the following operations until the Oth prediction residual feature is obtained and stop the following operations:

[0132] Inputting the kth predicted feature to be enhanced into the kth first feature processing layer to obtain the kth first predicted initial feature;

[0133] Add the kth predicted feature to be enhanced to the kth first predicted initial feature to obtain the kth predicted residual feature;

[0134] The kth predicted residual feature is the k+1th predicted feature to be enhanced, the 1st predicted feature to be enhanced is the i-th first predicted impurity feature to be purified, and k is 1, 2, 3, ..., 0 in sequence;

[0135] The Oth predicted residual feature is input into the second convolutional layer to obtain the first predicted enhanced image feature.

[0136] In an optional embodiment, the kth first feature processing layer includes a third convolution layer and a second activation function layer;

[0137] The step of inputting the kth predicted feature to be enhanced into the kth first feature processing layer to obtain the kth first predicted initial feature comprises:

[0138] Inputting the kth predicted feature to be enhanced into the third convolution layer to obtain a third predicted convolution result;

[0139] The third prediction convolution result is input into the second activation function layer to obtain the kth first prediction initial feature.

[0140] In an optional implementation, the j-th first background color removal block includes P-scale simulation input layers and multi-scale fusion layers, where P is a positive integer;

[0141] The step of inputting the jth first predicted background color feature to be removed into the jth first background color removal block to obtain the jth first predicted background color removal feature comprises:

[0142] Inputting the jth first predicted background color feature to be removed into the vth scale simulation input layer to obtain the vth scale predicted simulation input feature, wherein v is 1, 2, 3, ..., P in sequence;

[0143] Inputting the predicted simulation input features of P scales into the multi-scale fusion layer to obtain predicted fusion features;

[0144] The j-th first predicted background color feature to be removed is added to the predicted fusion feature to obtain the j-th first predicted background color removal feature.

[0145] In an optional embodiment, the simulated input layer of the vth scale includes a second residual dense block;

[0146] The step of inputting the jth first predicted background color feature to be removed into the vth scale simulation input layer to obtain the vth scale predicted simulation input feature includes:

[0147] Determine the size reduction factor and size increase factor corresponding to the pre-trained simulated input layer of the vth scale;

[0148] According to the size reduction ratio, the j-th first predicted background color feature to be removed is reduced in size to obtain the j-th first predicted background color feature to be removed after reduction;

[0149] Inputting the reduced j-th first predicted background color feature to be removed into the second residual dense block to obtain a second predicted enhanced feature;

[0150] The second prediction enhancement feature is scaled up according to the scale-up factor to obtain a prediction simulation input feature of the vth scale.

[0151] In an optional embodiment, the multi-scale fusion layer includes P average pooling layers, L second feature processing layers, a fourth convolution layer, and a third activation function layer:

[0152] The step of inputting the predicted simulation input features of P scales into the multi-scale fusion layer to obtain predicted fusion features includes:

[0153] Inputting the predicted simulated input features of the vth scale into the vth average pooling layer to obtain the vth predicted average pooling result;

[0154] Add the P predicted average pooling results to obtain the predicted initial fusion feature;

[0155] Iterate and perform the following operations until the Lth second prediction initial feature is obtained and then stop the following operations;

[0156] Input the tth predicted feature to be processed into the tth second processing feature layer to obtain the tth second predicted initial feature;

[0157] Among them, the t-th second prediction initial feature is the t+1-th prediction feature to be processed, the 1st prediction feature to be processed is the prediction initial fusion feature, and t is 1, 2, 3, ..., L in sequence;

[0158] Input the Lth second prediction initial feature into the fourth convolution layer to obtain the fourth prediction convolution result;

[0159] Inputting the fourth prediction convolution result into the third activation function layer to obtain prediction weights corresponding to the prediction simulation input features of P scales;

[0160] According to the prediction weights, the prediction simulation input features of P scales are weighted and summed to obtain the prediction fusion features.

[0161] In an optional embodiment, the second background color removal network includes a third residual dense block and N second background color removal blocks;

[0162] The step of inputting the first predicted purified image feature into the second background color removal network to obtain a second predicted purified image feature comprises:

[0163] Inputting the first predicted purified image feature into the third residual dense block to obtain a third predicted enhanced feature;

[0164] Iterate the following operations until the Nth second predicted background color removal feature is obtained and then stop the following operations:

[0165] Inputting the jth second predicted background color feature to be removed into the jth second background color removal block to obtain the jth second predicted background color removal feature;

[0166] Among them, the jth second predicted background color removal feature is the j+1th second predicted background color feature to be removed, the 1st first predicted background color feature to be removed is the third predicted enhancement feature, the Nth second predicted background color removal feature is the second predicted purified image feature, and j is 1, 2, 3,..., N respectively.

[0167] In an optional embodiment, the third background color removal network includes M+1 second background color removal sub-networks;

[0168] The step of inputting the second predicted purified image feature into the third background color removal network to obtain a predicted image sub-region comprises:

[0169] Iterate the following operations until the first second predicted impurity purification feature is obtained and then stop the following operations:

[0170] Inputting the i-th second predicted impurity feature to be purified into the i-th second background color removal sub-network to obtain the i-th second predicted impurity purification feature;

[0171] Among them, the i-th second predicted impurity purification feature is the i-1-th second predicted impurity feature to be purified, the M+1-th second predicted impurity feature to be purified is the second predicted purified image feature, the 1st second predicted impurity purification feature is the predicted image sub-region, and i is M+1,...,3,2,1 respectively.

[0172] In an optional embodiment, the i-th second background color removal sub-network includes a fourth residual dense block, N third background color removal blocks, a fifth convolutional layer, a sixth convolutional layer and an upsampling layer;

[0173] The step of inputting the i-th second predicted impurity feature to be purified into the i-th second background color removal sub-network to obtain the i-th second predicted impurity purification feature comprises:

[0174] Inputting the i-th second predicted impurity feature to be purified into the fifth convolution layer to obtain a fifth predicted convolution result;

[0175] Inputting the fifth prediction convolution result into the fourth residual dense block to obtain a fourth prediction enhancement feature;

[0176] Iterate the following operations until the Nth third predicted background color removal feature is obtained and then stop the following operations:

[0177] Inputting the jth third predicted background feature to be removed into the jth third background color removal block to obtain the jth third predicted background removal feature;

[0178] The jth third predicted background removal feature is the j+1th third predicted background feature to be removed, the first third predicted background feature to be removed is the fourth predicted enhancement feature, and j is 1, 2, 3, ..., N in sequence;

[0179] Inputting the Nth third predicted background color removal feature into the sixth convolution layer to obtain a sixth predicted convolution result;

[0180] The sixth predicted convolution result is input into the upsampling layer to obtain the i-th second predicted impurity purification feature.

[0181] In an optional implementation, the step of inputting the i-th second predicted impurity feature to be purified into the fifth convolution layer to obtain a fifth predicted convolution result includes:

[0182] Determine an i-th first background color removal sub-network at the same level as the i-th second background color removal sub-network;

[0183] Add the i-th first predicted impurity purification feature output by the i-th first background color removal subnetwork to the i-th second predicted impurity feature to be purified to obtain a predicted short connection feature;

[0184] The predicted short connection feature is input into the fifth convolution layer to obtain a fifth predicted convolution result.

[0185] In a second aspect of the embodiments of the present application, an image processing device is further provided, the device comprising:

[0186] An image acquisition module, used for acquiring an image to be processed, wherein the image to be processed contains impurities;

[0187] The image processing module is used to perform background color removal processing on the image to be processed to obtain a target image, wherein the target image does not contain the impurities.

[0188] In a third aspect of the embodiments of the present application, there is further provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0189] Memory, used to store computer programs;

[0190] The processor is used to implement any image processing method described in the first aspect when executing the program stored in the memory.

[0191] In a fourth aspect of the embodiments of the present application, a storage medium is further provided, wherein instructions are stored in the storage medium, and when the storage medium is run on a computer, the computer executes the image processing method described in any one of the first aspects above.

[0192] In a fifth aspect of the embodiments of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned image processing methods.

[0193] The technical solution provided by the embodiment of the present application obtains an image to be processed, wherein the image to be processed contains impurities, performs background color removal processing on the image to be processed, and obtains a target image, wherein the target image does not contain impurities.

[0194] By performing background color removal on the image to be processed, impurities in the image to be processed can be removed to obtain the target image. This method can be applied to the purification of different impurities, reduces the complexity of the algorithm, reduces the cost of impurity purification, and improves user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0195] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0196] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0197] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0198] Figure 1 A schematic diagram of an implementation flow of an image processing method shown in an embodiment of the present application;

[0199] Figure 2 A schematic diagram of a document image containing a shadow shown in an embodiment of the present application;

[0200] Figure 3 A schematic diagram of a document image with shadow removed shown in an embodiment of the present application;

[0201] Figure 4 It is a schematic diagram of an implementation flow of another image processing method shown in an embodiment of the present application;

[0202] Figure 5 It is a schematic diagram of an implementation flow of another image processing method shown in an embodiment of the present application;

[0203] Figure 6 A schematic diagram of the structure of a pre-trained general purification network shown in an embodiment of the present application;

[0204] Figure 7 It is a schematic diagram of the structure of another pre-trained universal purification network shown in the embodiment of the present application;

[0205] Figure 8 It is a schematic diagram of the structure of another pre-trained universal purification network shown in the embodiment of the present application;

[0206] Fig. 9 A schematic diagram of the structure of a pre-trained first residual dense block shown in an embodiment of the present application;

[0207] Fig.10 This is a schematic diagram of the structure of another pre-trained first residual dense block shown in an embodiment of the present application;

[0208] Fig.11 A schematic diagram of the structure of a pre-trained first background color removal block shown in an embodiment of the present application;

[0209] Fig.12 A schematic diagram of the structure of a pre-trained multi-scale fusion layer shown in an embodiment of the present application;

[0210] Fig.13 A schematic diagram of the implementation process of the network training method shown in the embodiment of the present application;

[0211] Fig.14 A schematic diagram of an implementation flow of a compressed image sub-region processing method shown in an embodiment of the present application;

[0212] Fig.15 A schematic diagram of the structure of an image processing device shown in an embodiment of the present application;

[0213] Fig.16 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0214] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0215] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0216] like Figure 1 FIG. 1 is a schematic diagram of an implementation flow of an image processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0217] S101, obtaining an image to be processed, wherein the image to be processed contains impurities.

[0218] In the embodiment of the present application, an image to be processed is obtained, wherein the image to be processed may be a document image taken for various documents, and impurities such as shadows, moiré patterns, and noise may be generated in the document image, and of course, it may also be other types of images.

[0219] It should be noted that the impurities contained in the image to be processed may be at least one of the impurities such as shadows, moiré patterns, and noise, which is not limited in the present embodiment. Figure 2 As shown, the impurities in the image to be processed, ie, the document image, are shadows.

[0220] S102, performing background color removal processing on the image to be processed to obtain a target image, wherein the target image does not contain impurities.

[0221] In the embodiment of the present application, for the image to be processed, the background color removal process can be performed on the image to be processed to obtain the target image. The target image does not contain impurities, which means that the impurities in the image to be processed can be removed by the background color removal process, and the foreground content of the image to be processed can be retained. At this time, the background of the target image becomes white. Figure 3 shown.

[0222] For example, for Figure 2 The document image shown in FIG. 1 is subjected to background color removal processing to obtain the following Figure 3 The document image shown in the figure can be removed by background color removal. Figure 2 The shadows in the document image shown here retain the Figure 2 The foreground content (ie, the document) in the document image is shown.

[0223] Through the above description of the technical solution provided in the embodiment of the present application, an image to be processed is obtained, wherein the image to be processed contains impurities, and background color removal processing is performed on the image to be processed to obtain a target image, wherein the target image does not contain impurities.

[0224] By performing background color removal on the image to be processed, impurities in the image to be processed can be removed to obtain the target image. This method can be applied to the purification of different impurities, reduces the complexity of the algorithm, reduces the cost of impurity purification, and improves user experience.

[0225] In addition, in the embodiments of the present application, a universal purification network is provided, which can realize universal purification of different impurities, for example, shadows, moiré patterns, noise, etc. can be removed, and different purification algorithms do not need to be designed for different impurities, which greatly reduces the complexity of the algorithm. Figure 4 FIG. 1 is a schematic diagram of an implementation flow of another image processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0226] S401, obtaining an image to be processed, wherein the image to be processed contains impurities.

[0227] In the embodiment of the present application, this step is similar to the above-mentioned step S101, and the embodiment of the present application will not be described one by one here.

[0228] S402, using a pre-trained universal purification network, performing background color removal processing on the image to be processed to obtain a target image, wherein the target image does not contain impurities.

[0229] In an embodiment of the present application, for an image to be processed, a pre-trained general purification network can be used to perform background color removal processing on the image to be processed to obtain a target image, wherein the target image does not contain impurities.

[0230] Specifically, the image to be processed can be input into a pre-trained universal purification network, and the pre-trained universal purification network performs background color removal on the image to be processed to obtain a target image.

[0231] The pre-trained general purification network can be roughly divided into three parts: a pre-trained first background color removal network, a pre-trained second background color removal network, and a pre-trained third background color removal network. For the process of using the pre-trained general purification network to remove the background color of the processed image, please refer to the following. Figure 5 As shown in the method. Figure 5 FIG. 1 is a schematic diagram of an implementation flow of another image processing method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0232] S501, inputting the image to be processed into a pre-trained first background color removal network to obtain a first purified image feature.

[0233] In an embodiment of the present application, for an image to be processed, the image to be processed can be input into a pre-trained first background color removal network to obtain a first purified image feature. The image to be processed needs to meet certain size requirements, otherwise the image to be processed needs to be pixel-filled and then input into the pre-trained first background color removal network.

[0234] In addition, the pre-trained general purification network also includes a pre-trained downsampling network and a pre-trained convolutional network, such as Figure 6 As shown, it means that for the image to be processed, it needs to be processed by downsampling, convolution, etc., and then input into the pre-trained first background color removal network. Among them, the pre-trained downsampling network can be, for example, pixelunshuffle, and the pre-trained convolution network can be, for example, Conv First, which is not limited in the embodiment of the present application.

[0235] Based on this, for the image to be processed, the image to be processed can be input into a pre-trained downsampling network to obtain the downsampled image to be processed, the downsampled image to be processed can be input into a pre-trained convolutional network to obtain a first convolution result, and the first convolution result can be input into a pre-trained first background color removal network to obtain a first purified image feature.

[0236] In one embodiment of the present application, the pre-trained first background color removal network is generally composed of M pre-trained first background color removal sub-networks, where M is a positive integer. For example, the pre-trained first background color removal network is composed of two pre-trained first background color removal sub-networks, such as pre-trained first background color removal sub-network 1 and pre-trained first background color removal sub-network 2. Figure 7 shown.

[0237] Based on this, the above-mentioned inputting the first convolution result into the pre-trained first background color removal network to obtain the first purified image feature can be specifically: iteratively performing the following operations until the Mth first impurity purification feature is obtained and stopping the following operations; inputting the i-th first impurity feature to be purified into the i-th pre-trained first background color removal sub-network to obtain the i-th first impurity purification feature; wherein the i-th first impurity purification feature is the i+1th first impurity feature to be purified, the 1st first impurity feature to be purified is the first convolution result, the Mth first impurity purification feature is the first purified image feature, and i is 1, 2, 3,..., M respectively.

[0238] Among them, in one embodiment of the present application, for the i-th pre-trained first background color removal sub-network, it specifically includes a pre-trained first residual dense block (such as RDB), N pre-trained first background color removal blocks, a pre-trained first convolutional layer (such as Conv), and a pre-trained first activation function layer (such as Relu), where N is a positive integer. For example, for the first pre-trained first background color removal sub-network and the second pre-trained first background color removal sub-network, they are both composed of 1 pre-trained first residual dense block, 2 pre-trained first background color removal blocks, 1 pre-trained first convolutional layer, and 1 pre-trained first activation function layer, such as Figure 8 shown.

[0239] Then the above-mentioned inputting the i-th first impurity feature to be purified into the i-th pre-trained first background color removal subnetwork to obtain the i-th first impurity purification feature can be specifically: inputting the i-th first impurity feature to be purified into the pre-trained first residual dense block to obtain the first enhanced image feature; iteratively performing the following steps until the N-th first background color removal feature is obtained and stopping the following operation: inputting the j-th first background color feature to be removed into the j-th pre-trained first background color removal block to obtain the j-th first background color removal feature; wherein the j-th first background color removal feature is the j+1-th first background color feature to be removed, the 1st first background color feature to be removed is the first enhanced image feature, and j is 1, 2, 3, ..., N respectively; inputting the N-th first background color removal feature into the pre-trained first convolutional layer to obtain the second convolution result; inputting the second convolution result into the pre-trained first activation function layer to obtain the i-th first impurity purification feature.

[0240] In one embodiment of the present application, the pre-trained first residual dense block generally includes O pre-trained first feature processing layers and pre-trained second convolutional layers, where O is a positive integer. Fig. 9 As shown, the pre-trained first residual dense block consists of three pre-trained first feature processing layers and one pre-trained second convolutional layer.

[0241] Based on this, the above-mentioned input of the i-th first impurity feature to be purified into the first pre-trained residual dense block to obtain the first enhanced image feature can be specifically: iteratively performing the following operations until the O-th residual feature is obtained and stopping the following operations: inputting the k-th feature to be enhanced into the k-th pre-trained first feature processing layer to obtain the k-th first initial feature; adding the k-th feature to be enhanced and the k-th first initial feature to obtain the k-th residual feature; wherein the k-th residual feature is the k+1-th feature to be enhanced, the 1st feature to be enhanced is the i-th first impurity feature to be purified, and k is 1, 2, 3, ..., O respectively; inputting the O-th residual feature into the second convolutional layer of pre-training to obtain the first enhanced image feature.

[0242] In one embodiment of the present application, the kth pre-trained first feature processing layer generally includes a pre-trained third convolution layer (e.g., Conv) and a pre-trained second activation function layer (e.g., Relu). Fig.10 As shown, the pre-trained first feature processing layer 1, the pre-trained first feature processing layer 2, and the pre-trained first feature processing layer 3 are all composed of one pre-trained third convolutional layer (e.g., Conv) and one pre-trained second activation function layer (e.g., Relu).

[0243] Based on this, the kth feature to be enhanced is input into the kth pre-trained first feature processing layer to obtain the kth first initial feature. Specifically, the kth feature to be enhanced can be input into the pre-trained third convolution layer to obtain the third convolution result; the third convolution result is input into the pre-trained second activation function layer to obtain the kth first initial feature.

[0244] In addition, the j-th pre-trained first background color removal block includes P scale pre-trained analog input layers and pre-trained multi-scale fusion layers, where P is a positive integer. For example, the j-th pre-trained first background color removal block includes 4 scale pre-trained analog input layers and 1 pre-trained multi-scale fusion layer.

[0245] Based on this, the above-mentioned inputting the jth first background color feature to be removed into the jth pre-trained first background color removal block to obtain the jth first background color removal feature can be specifically: inputting the jth first background color feature to be removed into the vth scale pre-trained simulation input layer to obtain the vth scale simulation input feature, wherein v is 1, 2, 3, ..., P in sequence; inputting the P scale simulation input features into the pre-trained multi-scale fusion layer to obtain the fusion feature; adding the jth first background color feature to be removed and the fusion feature to obtain the jth first background color removal feature.

[0246] Among them, in one embodiment of the present application, for the pre-trained analog input layer of the vth scale, it includes a pre-trained second residual dense block. The above-mentioned inputting the jth first background color feature to be removed into the pre-trained analog input layer of the vth scale to obtain the analog input feature of the vth scale can be specifically: determining the size reduction ratio and size increase ratio corresponding to the pre-trained analog input layer of the vth scale; reducing the size of the jth first background color feature to be removed according to the size reduction ratio to obtain the reduced jth first background color feature to be removed; inputting the reduced jth first background color feature to be removed into the pre-trained second residual dense block to obtain the second enhanced feature; increasing the size of the second enhanced feature according to the size increase ratio to obtain the analog input feature of the vth scale.

[0247] In one embodiment of the present application, the j-th pre-trained first background color removal block includes 4 scale pre-trained analog input layers and 1 pre-trained multi-scale fusion layer, and the size reduction ratios and size increase ratios corresponding to the 4 scale pre-trained analog input layers are as follows: Fig.11As shown, "0" represents the same size, "0.5" represents the size is reduced by half, "2" represents the size is increased by 2 times, and so on. In this way, the jth first background color feature to be removed is first reduced in size according to the size reduction factor corresponding to the pre-trained simulated input layer of the vth scale, and then input into the pre-trained second residual dense block to obtain the second enhanced feature, and then the second enhanced feature is increased in size according to the size increase factor corresponding to the pre-trained simulated input layer of the vth scale to obtain the simulated input feature of the vth scale.

[0248] It should be noted that the size reduction ratio and size increase ratio can be achieved through the F.interpolate function, and the size reduction ratio and size increase ratio corresponding to the pre-trained simulated input layer of each scale are different, such as Fig.11 As shown, the embodiments of the present application do not limit this. In addition, for the pre-trained second residual dense block, its internal structure is similar to the pre-trained first residual dense block, then the internal processing of the pre-trained second residual dense block is similar to the internal processing of the pre-trained first residual dense block, and the embodiments of the present application will not be repeated here.

[0249] In one embodiment of the present application, the pre-trained multi-scale fusion layer includes P pre-trained average pooling layers (e.g., AvgPooling), L pre-trained second feature processing layers, a pre-trained fourth convolution layer, and a pre-trained third activation function layer. For example, the pre-trained multi-scale fusion layer includes 4 pre-trained average pooling layers, 2 pre-trained second feature processing layers, 1 pre-trained fourth convolution layer, and 1 pre-trained third activation function layer (e.g., Sigmoid), such as Fig.12 shown.

[0250] Based on this, the above-mentioned inputting of the simulated input features of P scales into the pre-trained multi-scale fusion layer to obtain the fused features can be specifically: inputting the simulated input features of the v-th scale into the v-th pre-trained average pooling layer to obtain the v-th average pooling result; adding the P average pooling results to obtain the initial fused features; iteratively performing the following operations until the L-th second initial feature is obtained and stopping the following operations; inputting the t-th feature to be processed into the t-th pre-trained second processing feature layer to obtain the t-th second initial feature; wherein the t-th second initial feature is the t+1-th feature to be processed, the 1st feature to be processed is the initial fused feature, and t is 1, 2, 3, ..., L respectively; inputting the L-th second initial feature into the pre-trained fourth convolutional layer to obtain the fourth convolution result; inputting the fourth convolution result into the pre-trained third activation function layer to obtain the weights corresponding to the simulated input features of the P scales (for example, Fig.12w0, w1, w2, w3) as shown; according to the weights, the simulated input features of P scales are weighted summed to obtain the fused features.

[0251] It should be noted that the pre-trained second processing feature layer is similar to the pre-trained first feature processing layer mentioned above. For the structure of the pre-trained second processing feature layer and its internal processing flow, reference can be made to the structure of the pre-trained first feature processing layer and its internal processing flow, and the embodiments of the present application will not be described in detail here.

[0252] S502: Input the first purified image feature into a pre-trained second background color removal network to obtain a second purified image feature.

[0253] In an embodiment of the present application, for the first purified image feature obtained above, the first purified image feature can be input into a pre-trained second background color removal network to obtain a second purified image feature. The pre-trained second background color removal network includes a pre-trained third residual dense block and N pre-trained second background color removal blocks. For example, the pre-trained second background color removal network includes a pre-trained third residual dense block and two pre-trained second background color removal blocks, such as Figure 8 shown.

[0254] Based on this, the first purified image feature is input into the pre-trained third residual dense block to obtain the third enhanced feature; the following operations are iteratively performed until the Nth second background color removal feature is obtained, and the following operations are stopped: the jth second background color feature to be removed is input into the jth pre-trained second background color removal block to obtain the jth second background color removal feature; wherein the jth second background color removal feature is the j+1th second background color feature to be removed, the 1st first background color feature to be removed is the third enhanced feature, the Nth second background color removal feature is the second purified image feature, and j is 1, 2, 3, ..., N respectively.

[0255] It should be noted that, for the pre-trained third residual dense block, similar to the pre-trained first residual dense block, the structure of the pre-trained third residual dense block and its internal processing flow can refer to the structure of the pre-trained first residual dense block and its internal processing flow. Similarly, for the pre-trained second background color removal block, similar to the pre-trained first background color removal block, the structure of the pre-trained second background color removal block and its internal processing flow can refer to the structure of the pre-trained first background color removal block and its internal processing flow, and the embodiments of the present application will not be described one by one here.

[0256] S503: Input the second purified image features into a pre-trained third background color removal network to obtain a target image.

[0257] In the embodiment of the present application, for the second purified image feature, the second purified image feature can be input into the pre-trained third background color removal network to obtain the target image. The pre-trained third background color removal network usually includes M+1 pre-trained second background color removal sub-networks. For example, the pre-trained third background color removal network includes 3 pre-trained second background color removal sub-networks, such as Figure 7 shown.

[0258] Based on this, the above-mentioned input of the second purified image feature into the pre-trained third background color removal network to obtain the target image can be specifically: iteratively performing the following operations until the first second impurity purification feature is obtained and stopping the following operations: inputting the i-th second impurity feature to be purified into the i-th pre-trained second background color removal sub-network to obtain the i-th second impurity purification feature; wherein the i-th second impurity purification feature is the i-1-th second impurity feature to be purified, the M+1-th second impurity feature to be purified is the second purified image feature, the first second impurity purification feature is the target image, and i is M+1,...,3,2,1 respectively.

[0259] Among them, in one embodiment of the present application, the i-th pre-trained second background color removal sub-network includes a pre-trained fourth residual dense block, N pre-trained third background color removal blocks, a pre-trained fifth convolutional layer, a pre-trained sixth convolutional layer, and a pre-trained upsampling layer. For example, the i-th pre-trained second background color removal sub-network includes a pre-trained fourth residual dense block, 2 pre-trained third background color removal blocks, a pre-trained fifth convolutional layer, a pre-trained sixth convolutional layer, and a pre-trained upsampling layer (e.g., pixel shuffle), such as Figure 8 shown.

[0260] To this end, the above-mentioned input of the i-th second impurity feature to be purified into the i-th pre-trained second background color removal subnetwork to obtain the i-th second impurity purification feature can be specifically: inputting the i-th second impurity feature to be purified into the pre-trained fifth convolutional layer to obtain the fifth convolution result; inputting the fifth convolution result into the pre-trained fourth residual dense block to obtain the fourth enhanced feature; iteratively performing the following operations until the N-th third background color removal feature is obtained and stopping the following operations: inputting the j-th third background feature to be removed into the j-th pre-trained third background color removal block to obtain the j-th third background removal feature; wherein the j-th third background removal feature is the j+1-th third background feature to be removed, the 1st third background feature to be removed is the fourth enhanced feature, and j is 1, 2, 3, ..., N respectively; inputting the N-th third background color removal feature into the pre-trained sixth convolutional layer to obtain the sixth convolution result; inputting the sixth convolution result into the pre-trained upsampling layer to obtain the i-th second impurity purification feature.

[0261] In addition, in the embodiment of the present application, for the pre-trained second background color removal sub-network and the pre-trained first background color removal sub-network, there are short connections at the same level, such as Figure 7 As shown. To this end, the above-mentioned input of the i-th second impurity feature to be purified into the pre-trained fifth convolutional layer to obtain the fifth convolution result can be specifically: determining the i-th pre-trained first background color removal subnetwork at the same level as the i-th pre-trained second background color removal subnetwork; adding the i-th first impurity purification feature output by the i-th pre-trained first background color removal subnetwork to the i-th second impurity feature to be purified to obtain a short connection feature; inputting the short connection feature into the pre-trained fifth convolutional layer to obtain the fifth convolution result. Among them, i here does not include M+1, which means that the M+1-th pre-trained second background color removal subnetwork does not have a short connection.

[0262] It should be noted that, for the pre-trained fourth residual dense block, which is similar to the above-mentioned pre-trained first residual dense block, the structure of the pre-trained fourth residual dense block and its internal processing flow can refer to the structure of the above-mentioned pre-trained first residual dense block and its internal processing flow. Similarly, for the pre-trained third background color removal block, which is similar to the above-mentioned pre-trained first background color removal block, the structure of the pre-trained third background color removal block and its internal processing flow can refer to the structure of the above-mentioned pre-trained first background color removal block and its internal processing flow, and the embodiments of the present application will not be described one by one here.

[0263] In addition, in an embodiment of the present application, for a pre-trained general purification network, the above-mentioned residual dense block (i.e., the above-mentioned first, second, third, and fourth residual dense blocks) is introduced. For the residual dense block, there are multiple layers of dense connections, and each layer adds the input of the previous layer to the input of the current layer through the residual connection. This dense connection mechanism can effectively transfer and enhance features, improve feature extraction capabilities, and reduce the loss of features during network propagation. In addition, for the residual dense block, by continuously stacking feature processing layers, the depth of the network can be increased, the expression ability of the network can be further improved, and the residual dense block itself can learn richer feature representations, which can improve the quality of the target image and make the target image clearer and richer in details. For the pre-trained general purification network, a background color removal block (i.e., the above-mentioned first, second, and third background color removal blocks) is also introduced. For the background color removal block, by multi-scale downsampling of each level to simulate inputs of different resolutions, the background colors of images of different resolutions can be removed more accurately.

[0264] In addition, in one embodiment of the present application, the pre-trained universal purification network includes three parts, namely, at least one pre-trained first background color removal network, a pre-trained second background color removal network, and a pre-trained third background color removal network. The process of performing background color removal processing on the image to be processed using the pre-trained universal purification network can be specifically: when the pre-trained universal purification network includes a pre-trained first background color removal network, the image to be processed is input into the pre-trained first background color removal network to obtain a first purified image feature, or, when the pre-trained universal purification network includes two or more pre-trained first background color removal networks, the image to be processed is sequentially input into the pre-trained first background color removal networks to obtain a first purified image feature, wherein, after the image to be processed is sequentially input into the first pre-trained first background color removal network, the input of the pre-trained first background color removal network is the output of the previous pre-trained first background color removal network; the first purified image feature is input into the pre-trained second background color removal network to obtain a second purified image feature, and the second purified image feature is input into the pre-trained third background color removal network to obtain a target image.

[0265] For example, the pre-trained general purification network includes two pre-trained first background color removal networks, a pre-trained second background color removal network, a pre-trained third background color removal network, etc. The image to be processed is input into the first pre-trained first background color removal network to obtain a result, and then the result is input into the second pre-trained first background color removal network to obtain the first purified image feature, and the image to be processed is input into the pre-trained first background color removal network in sequence to obtain the first purified image feature, and then the first purified image feature is input into the pre-trained second background color removal network to obtain the second purified image feature, and the second purified image feature is input into the pre-trained third background color removal network to obtain the target image. The internal processing of the pre-trained first background color removal network, the pre-trained second background color removal network, and the pre-trained third background color removal network here can refer to the internal processing of the pre-trained first background color removal network, the pre-trained second background color removal network, and the pre-trained third background color removal network.

[0266] In addition, in one embodiment of the present application, the pre-trained general purification network includes at least two parts, namely, a pre-trained first background color removal network and a pre-trained third background color removal network. The process of performing background color removal processing on the image to be processed using the pre-trained general purification network can be specifically inputting the image to be processed into the pre-trained first background color removal network in sequence to obtain the first purified image feature, and inputting the first purified image feature into the pre-trained third background color removal network to obtain the target image. The internal processing process of the pre-trained first background color removal network and the pre-trained third background color removal network here can refer to the internal processing process of the pre-trained first background color removal network and the pre-trained third background color removal network mentioned above.

[0267] For the pre-trained general purification network, network training is required. Fig.13 FIG. 1 is a schematic diagram of an implementation flow of a network training method provided in an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0268] S1301, obtaining an image pair set, wherein the image pairs in the image pair set include a sample image and a standard image, and the impurities contained in the sample image cover different impurities.

[0269] In the embodiment of the present application, in order to perform universal purification for multiple impurities such as moiré, shadow, noise, etc., the image pairs for network training need to be evenly covered with multiple impurities such as moiré, shadow, noise, etc., so that the purification capabilities of multiple impurities such as moiré, shadow, noise, etc. can be learned in a balanced manner during the training process. For example, the image pairs with multiple impurities such as moiré, shadow, noise, etc. are not balanced, which may cause the network to learn the purification capabilities of, for example, moiré, while the purification capabilities of impurities such as shadow, noise, etc. are poor.

[0270] To this end, a set of image pairs is collected in advance. The image pairs in the set of image pairs include sample images and standard images. The sample images contain impurities, while the standard images do not contain impurities. The impurities contained in the sample images cover different impurities, and the proportions of each impurity are relatively balanced. In addition, the image pairs can be currently open source image pairs. Most of these open source image pairs are synthetic data, which means that they are not data of real scenes and have certain differences from the data of real scenes.

[0271] S1302, obtaining an image pair from an image pair set, inputting a sample image in the image pair into a universal purification network, and obtaining a predicted image.

[0272] S1303, determining a loss value between the predicted image and the standard image in the image pair, and performing network training on the universal purification network according to the loss value.

[0273] S1304, when the loss value converges, stop the network training to obtain a pre-trained universal purification network.

[0274] In an embodiment of the present application, for an image pair set, an image pair is obtained from the image pair set, a sample image in the image pair is input into a universal purification network to obtain a predicted image, a loss value between the predicted image and a standard image in the image pair is determined, and the universal purification network is trained based on the loss value. When the loss value converges, the network training is stopped to obtain a pre-trained universal purification network.

[0275] Among them, for an image pair, a sample image in the image pair is cropped to obtain at least one image subregion, for any image subregion, the image subregion is compressed to obtain a compressed image subregion, the compressed image subregion is input into a universal purification network to obtain a predicted image subregion, and from the standard image of the image pair, a standard image subregion corresponding to the image subregion is searched to determine the loss value between the standard image subregion and the predicted image subregion.

[0276] It should be noted that, for the image sub-region, its size can be, for example, 512*512, and the compression method used can be, for example, jpeg compression. The use of jpeg compression is considered to be a relatively common compression method for images during network transmission. Performing jpeg compression on the data input to the network can improve the network's generalization ability for images to be processed in real scenes. In addition, the loss value is determined by VGG Loss, which can supervise the colors in the image and increase the fidelity of the image.

[0277] In one embodiment of the present application, the general purification network may include a first background color removal network, a second background color removal network, and a third background color removal network. For the process of performing background color removal processing on a compressed image sub-region using the general purification network, reference may be made to the following. Fig.14 As shown in the method. Fig.14 FIG. 1 is a schematic diagram of an implementation flow of a compressed image sub-region processing method provided by an embodiment of the present application. The method is applied to an electronic device and may specifically include the following steps:

[0278] S1401, input the compressed image sub-region into a first background color removal network to obtain a first predicted purified image feature.

[0279] In an embodiment of the present application, for a compressed image sub-region, the compressed image sub-region can be input into a first background color removal network to obtain a first predicted purified image feature. Wherein, for the compressed image sub-region, a certain size requirement needs to be met, otherwise the compressed image sub-region needs to be pixel-filled and then input into the first background color removal network.

[0280] In addition, in one embodiment of the present application, the general purification network also includes a downsampling network and a convolution network, which means that the compressed image sub-region needs to be downsampled, convolved, and processed before being input into the first background color removal network. To this end, the compressed image sub-region is input into the downsampling network to obtain the downsampled compressed image sub-region; the downsampled compressed image sub-region is input into the convolution network to obtain the first predicted convolution result; the first predicted convolution result is input into the first background color removal network to obtain the first predicted purified image feature.

[0281] Among them, in one embodiment of the present application, for the first background color removal network, it includes M first background color removal sub-networks, and M is a positive integer. To this end, the above-mentioned inputting the first predicted convolution result into the first background color removal network to obtain the first predicted purified image feature can be specifically: iteratively performing the following operations until the Mth first predicted impurity purification feature is obtained and stopping the following operations; inputting the i-th first predicted impurity feature to be purified into the i-th first background color removal sub-network to obtain the i-th first predicted impurity purification feature; wherein the i-th first predicted impurity purification feature is the i+1th first predicted impurity feature to be purified, the 1st first predicted impurity feature to be purified is the first predicted convolution result, the Mth first predicted impurity purification feature is the first predicted purified image feature, and i is 1, 2, 3, ..., M in sequence.

[0282] In one embodiment of the present application, the i-th first background color removal sub-network includes a first residual dense block, N first background color removal blocks, a first convolutional layer, and a first activation function layer, where N is a positive integer. The above-mentioned inputting the i-th first predicted impurity feature to be purified into the i-th first background color removal subnetwork to obtain the i-th first predicted impurity purification feature can be specifically: inputting the i-th first predicted impurity feature to be purified into the first residual dense block to obtain the first predicted enhanced image feature; iteratively performing the following steps until the N-th first predicted background color removal feature is obtained and stopping the following operation: inputting the j-th first predicted background color feature to be removed into the j-th first background color removal block to obtain the j-th first predicted background color removal feature; wherein the j-th first predicted background color removal feature is the j+1-th first predicted background color feature to be removed, the 1st first predicted background color feature to be removed is the first predicted enhanced image feature, and j is 1, 2, 3, ..., N respectively; inputting the N-th first predicted background color removal feature into the first convolution layer to obtain the second predicted convolution result; inputting the second predicted convolution result into the first activation function layer to obtain the i-th first predicted impurity purification feature.

[0283] In one embodiment of the present application, the first residual dense block includes O first feature processing layers and a second convolution layer, where O is a positive integer. To this end, the above-mentioned inputting the i-th first predicted impurity feature to be purified into the first residual dense block to obtain the first predicted enhanced image feature can be specifically: iteratively performing the following operations until the O-th predicted residual feature is obtained and stopping the following operations: inputting the k-th predicted feature to be enhanced into the k-th first feature processing layer to obtain the k-th first predicted initial feature; adding the k-th predicted feature to be enhanced to the k-th first predicted initial feature to obtain the k-th predicted residual feature; wherein the k-th predicted residual feature is the k+1-th predicted feature to be enhanced, the 1st predicted feature to be enhanced is the i-th first predicted impurity feature to be purified, and k is 1, 2, 3, ..., O in sequence; inputting the O-th predicted residual feature into the second convolution layer to obtain the first predicted enhanced image feature.

[0284] Among them, in one embodiment of the present application, for the kth first feature processing layer, it usually includes a third convolution layer (such as Conv) and a second activation function layer (such as Relu). Based on this, the kth predicted feature to be enhanced is input to the kth first feature processing layer to obtain the kth first predicted initial feature, which can be specifically: the kth predicted feature to be enhanced is input to the third convolution layer to obtain the third predicted convolution result; the third predicted convolution result is input to the second activation function layer to obtain the kth first predicted initial feature.

[0285] In addition, in one embodiment of the present application, the j-th first background color removal block includes a simulation input layer of P scales and a multi-scale fusion layer, where P is a positive integer. Based on this, the j-th first predicted background color feature to be removed is input into the j-th first background color removal block to obtain the j-th first predicted background color removal feature, which can be specifically: the j-th first predicted background color feature to be removed is input into the simulation input layer of the v-th scale to obtain the predicted simulation input feature of the v-th scale, where v is 1, 2, 3, ..., P in sequence; the predicted simulation input features of P scales are input into the multi-scale fusion layer to obtain the predicted fusion feature; the j-th first predicted background color feature to be removed is added to the predicted fusion feature to obtain the j-th first predicted background color removal feature.

[0286] Among them, in one embodiment of the present application, the analog input layer of the vth scale includes a second residual dense block. To this end, the above-mentioned inputting the jth first predicted background color feature to be removed into the analog input layer of the vth scale to obtain the predicted analog input feature of the vth scale can be specifically: determining the size reduction ratio and size increase ratio corresponding to the pre-trained analog input layer of the vth scale; performing size reduction processing on the jth first predicted background color feature to be removed according to the size reduction ratio to obtain the reduced jth first predicted background color feature to be removed; inputting the reduced jth first predicted background color feature to be removed into the second residual dense block to obtain the second predicted enhanced feature; performing size increase processing on the second predicted enhanced feature according to the size increase ratio to obtain the predicted analog input feature of the vth scale.

[0287] It should be noted that the size reduction ratio and size increase ratio can be realized by the F.interpolate function, and the size reduction ratio and size increase ratio corresponding to the analog input layer of each scale are different, such as Fig.11 In addition, for the second residual dense block, its internal structure is similar to the first residual dense block, and the internal processing of the second residual dense block is similar to the internal processing of the first residual dense block, and the embodiment of the present application will not be repeated here.

[0288] In one embodiment of the present application, the multi-scale fusion layer includes P average pooling layers (eg, AvgPooling), L second feature processing layers, a fourth convolutional layer, and a third activation function layer. To this end, the above-mentioned input of the predicted simulation input features of P scales into the multi-scale fusion layer to obtain the predicted fusion features, specifically, it can be: inputting the predicted simulation input features of the v-th scale into the v-th average pooling layer to obtain the v-th predicted average pooling result; adding the P predicted average pooling results to obtain the predicted initial fusion features; iteratively performing the following operations until the L-th second predicted initial feature is obtained and stopping the following operations; inputting the t-th predicted feature to be processed into the t-th second processing feature layer to obtain the t-th second predicted initial feature; wherein the t-th second predicted initial feature is the t+1-th predicted feature to be processed, the 1st predicted feature to be processed is the predicted initial fusion feature, and t is 1, 2, 3, ..., L respectively; inputting the L-th second predicted initial feature into the fourth convolution layer to obtain the fourth predicted convolution result; inputting the fourth predicted convolution result into the third activation function layer to obtain the prediction weights corresponding to the predicted simulation input features of P scales; according to the prediction weights, weighted summing is performed on the predicted simulation input features of P scales to obtain the predicted fusion features.

[0289] It should be noted that the second processing feature layer is similar to the first feature processing layer. For the structure of the second processing feature layer and its internal processing flow, reference can be made to the structure of the first feature processing layer and its internal processing flow, and the embodiments of the present application will not be described in detail here.

[0290] S1402: Input the first predicted purified image feature into a second background color removal network to obtain a second predicted purified image feature.

[0291] In an embodiment of the present application, for the first predicted purified image feature obtained above, the first predicted purified image feature can be input into the second background color removal network to obtain the second predicted purified image feature. Wherein, for the second background color removal network, it includes a third residual dense block and N pre-trained second background color removal blocks.

[0292] Based on this, the first predicted purified image feature is input into the third residual dense block to obtain the third predicted enhanced feature; the following operations are iteratively performed until the Nth second predicted background color removal feature is obtained, and the following operations are stopped: the jth second predicted background color feature to be removed is input into the jth second background color removal block to obtain the jth second predicted background color removal feature; wherein the jth second predicted background color removal feature is the j+1th second predicted background color feature to be removed, the 1st first predicted background color feature to be removed is the third predicted enhanced feature, the Nth second predicted background color removal feature is the second predicted purified image feature, and j is 1, 2, 3,..., N respectively.

[0293] It should be noted that, for the third residual dense block, similar to the first residual dense block, the structure of the third residual dense block and its internal processing flow can refer to the structure of the first residual dense block and its internal processing flow. Similarly, for the second background color removal block, similar to the first background color removal block, the structure of the second background color removal block and its internal processing flow can refer to the structure of the first background color removal block and its internal processing flow, and the embodiments of the present application will not be repeated here one by one.

[0294] S1403: Input the second predicted purified image feature into a third background color removal network to obtain a predicted image sub-region.

[0295] In an embodiment of the present application, for the second predicted cleaned image feature, the second predicted cleaned image feature is input into the third background color removal network to obtain a predicted image sub-region, wherein the third background color removal network generally includes M+1 second background color removal sub-networks.

[0296] Based on this, the above-mentioned input of the second predicted purified image feature into the third background color removal network to obtain the predicted image sub-region can be specifically: iteratively perform the following operations until the first second predicted impurity purification feature is obtained, and stop the following operations: input the i-th second predicted impurity feature to be purified into the i-th second background color removal sub-network to obtain the i-th second predicted impurity purification feature; wherein the i-th second predicted impurity purification feature is the i-1-th second predicted impurity feature to be purified, the M+1-th second predicted impurity feature to be purified is the second predicted purified image feature, the first second predicted impurity purification feature is the predicted image sub-region, and i is M+1,...,3,2,1 respectively.

[0297] In one embodiment of the present application, the i-th second background color removal sub-network includes a fourth residual dense block, N third background color removal blocks, a fifth convolutional layer, a sixth convolutional layer and an upsampling layer. To this end, the above-mentioned input of the i-th second predicted impurity feature to be purified into the i-th second background color removal subnetwork to obtain the i-th second predicted impurity purification feature can be specifically: inputting the i-th second predicted impurity feature to be purified into the fifth convolutional layer to obtain the fifth predicted convolution result; inputting the fifth predicted convolution result into the fourth residual dense block to obtain the fourth predicted enhanced feature; iteratively performing the following operations until the N-th third predicted background color removal feature is obtained and stopping the following operations: inputting the j-th third predicted background feature to be removed into the j-th third background color removal block to obtain the j-th third predicted background removal feature; wherein the j-th third predicted background removal feature is the j+1-th third predicted background feature to be removed, the 1st third predicted background feature to be removed is the fourth predicted enhanced feature, and j is 1, 2, 3, ..., N respectively; inputting the N-th third predicted background color removal feature into the sixth convolutional layer to obtain the sixth predicted convolution result; inputting the sixth predicted convolution result into the upsampling layer to obtain the i-th second predicted impurity purification feature.

[0298] In addition, in an embodiment of the present application, for the second background color removal subnetwork and the first background color removal subnetwork, there are short connections at the same level. To this end, the above-mentioned input of the i-th second predicted impurity feature to be purified into the fifth convolutional layer to obtain the fifth predicted convolution result can be specifically: determining the i-th first background color removal subnetwork at the same level as the i-th second background color removal subnetwork; adding the i-th first predicted impurity purification feature output by the i-th first background color removal subnetwork to the i-th second predicted impurity feature to be purified to obtain a predicted short connection feature; inputting the predicted short connection feature into the fifth convolutional layer to obtain the fifth predicted convolution result.

[0299] It should be noted that, for the fourth residual dense block, it is similar to the first residual dense block, so the structure of the fourth residual dense block and its internal processing flow can refer to the structure of the first residual dense block and its internal processing flow. Similarly, for the third background color removal block, it is similar to the first background color removal block, so the structure of the third background color removal block and its internal processing flow can refer to the structure of the first background color removal block and its internal processing flow, and the embodiments of the present application will not be repeated here one by one.

[0300] After the above network training, a pre-trained general purification network can be obtained. Since the open-source image pairs used in network training are mostly synthetic data, which means they are not real scene data and have certain differences from real scene data, resulting in poor reasoning effect in real scenes.

[0301] Based on this, for the above target image, if its impurities are removed well, it can form an image pair with the image to be processed, add it to the image pair set, and participate in the subsequent network training of the pre-trained general purification network. If its impurities are removed poorly, the PS tool can be used to process the target image, and then form an image pair with the image to be processed, add it to the image pair set, and participate in the subsequent network training of the pre-trained general purification network. In this way, image pairs of real scenes can be obtained to participate in the subsequent network training of the pre-trained general purification network. After repeated many times, the trained network can correctly process the images to be processed in real scenes. This progressive, pseudo-label approach is used because it is very difficult to directly remove impurities such as moiré, shadows, and noise in real scenes. This step-by-step training, step-by-step reasoning, and step-by-step PS method can reduce the difficulty of impurity removal, and the network training is also gradually from simple to difficult, and the robustness of the network is getting better and better.

[0302] Corresponding to the above method embodiment, the present application embodiment also provides an image processing device, such as Fig.15 As shown, the device may include: an image acquisition module 1510 and an image processing module 1520.

[0303] An image acquisition module 1510 is used to acquire an image to be processed, wherein the image to be processed contains impurities;

[0304] The image processing module 1520 is used to perform background color removal processing on the image to be processed to obtain a target image, wherein the target image does not contain the impurities.

[0305] In an optional implementation, the image processing module is specifically used to:

[0306] The pre-trained universal purification network is used to perform background color removal processing on the image to be processed to obtain a target image.

[0307] In an optional embodiment, the pre-trained universal purification network includes at least one pre-trained first background color removal network, a pre-trained second background color removal network, and a pre-trained third background color removal network; the image processing module specifically includes:

[0308] A first background color removal submodule is used for, when the pre-trained universal purification network includes one pre-trained first background color removal network, inputting the image to be processed into the pre-trained first background color removal network to obtain a first purified image feature; or, when the pre-trained universal purification network includes two or more pre-trained first background color removal networks, inputting the image to be processed into the pre-trained first background color removal networks in sequence to obtain a first purified image feature;

[0309] A second background color removal submodule, configured to input the first purified image feature into the pre-trained second background color removal network to obtain a second purified image feature;

[0310] The third background color removal submodule is used to input the second cleaned image features into the pre-trained third background color removal network to obtain a target image.

[0311] In an optional embodiment, the pre-trained universal purification network includes at least one pre-trained first background color removal network and a pre-trained third background color removal network; the image processing module specifically includes:

[0312] A first background color removal submodule, used for sequentially inputting the image to be processed into the pre-trained first background color removal network to obtain a first purified image feature;

[0313] The third background color removal submodule is used to input the first cleaned image features into the pre-trained third background color removal network to obtain a target image.

[0314] In an optional implementation, the pre-trained universal purification network further includes a pre-trained downsampling network and a pre-trained convolutional network; and the first background color removal submodule specifically includes:

[0315] An image downsampling unit, used for inputting the image to be processed into the pre-trained downsampling network to obtain the image to be processed after downsampling;

[0316] An image convolution unit, used for inputting the downsampled image to be processed into the pre-trained convolution network to obtain a first convolution result;

[0317] The first background color removal unit is used to input the first convolution result into the pre-trained first background color removal network to obtain a first purified image feature.

[0318] In an optional embodiment, the pre-trained first background color removal network includes M pre-trained first background color removal sub-networks, where M is a positive integer; and the first background color removal unit specifically includes:

[0319] An iterative subunit is used to iteratively perform the following operations until the Mth first impurity purification feature is obtained and then stop the following operations;

[0320] A first background color removal subunit inputs the i-th first impurity feature to be purified into the i-th pre-trained first background color removal subnetwork to obtain the i-th first impurity purification feature;

[0321] Among them, the i-th first impurity purification feature is the i+1-th first impurity feature to be purified, the 1st first impurity feature to be purified is the first convolution result, the M-th first impurity purification feature is the first purified image feature, and i is 1, 2, 3, ..., M respectively.

[0322] In an optional embodiment, the i-th pre-trained first background color removal sub-network includes a pre-trained first residual dense block, N pre-trained first background color removal blocks, a pre-trained first convolutional layer, and a pre-trained first activation function layer, where N is a positive integer; the first background color removal sub-unit is specifically used for:

[0323] Inputting the i-th first impurity feature to be purified into the pre-trained first residual dense block to obtain a first enhanced image feature;

[0324] Iterate the following steps until the Nth first background color removal feature is obtained and then stop the following operations:

[0325] Inputting the jth first background color feature to be removed into the jth pre-trained first background color removal block to obtain the jth first background color removal feature;

[0326] The jth first background color removal feature is the j+1th first background color feature to be removed, the first first background color feature to be removed is the first enhanced image feature, and j is 1, 2, 3, ..., N in sequence;

[0327] Inputting the Nth first background color removal feature into the pre-trained first convolution layer to obtain a second convolution result;

[0328] The second convolution result is input into the pre-trained first activation function layer to obtain the i-th first impurity purification feature.

[0329] In an optional embodiment, the pre-trained first residual dense block includes O pre-trained first feature processing layers and a pre-trained second convolutional layer, where O is a positive integer; the first background color removal subunit is specifically used for:

[0330] Iterate and perform the following operations until the Oth residual feature is obtained and stop the following operations:

[0331] Inputting the kth feature to be enhanced into the kth pre-trained first feature processing layer to obtain the kth first initial feature;

[0332] Add the kth feature to be enhanced to the kth first initial feature to obtain the kth residual feature;

[0333] The kth residual feature is the k+1th feature to be enhanced, the 1st feature to be enhanced is the i-th first impurity feature to be purified, and k is 1, 2, 3, ..., 0 in sequence;

[0334] The Oth residual feature is input into the second convolutional layer of the pre-training to obtain the first enhanced image feature.

[0335] In an optional embodiment, the kth pre-trained first feature processing layer includes a pre-trained third convolution layer and a pre-trained second activation function layer; and the first background color removal subunit is specifically used for:

[0336] Inputting the kth feature to be enhanced into the pre-trained third convolutional layer to obtain a third convolution result;

[0337] The third convolution result is input into the pre-trained second activation function layer to obtain the kth first initial feature.

[0338] In an optional embodiment, the jth pre-trained first background color removal block includes a pre-trained analog input layer of P scales and a pre-trained multi-scale fusion layer, where P is a positive integer; the first background color removal subunit is specifically used for:

[0339] In an optional embodiment, the pre-trained simulated input layer of the vth scale includes a pre-trained second residual dense block; the first background color removal subunit is specifically used for:

[0340] Determine the size reduction factor and size increase factor corresponding to the pre-trained simulated input layer of the vth scale;

[0341] According to the size reduction ratio, the j-th first background color feature to be removed is reduced in size to obtain the reduced j-th first background color feature to be removed;

[0342] Inputting the reduced j-th first background color feature to be removed into the pre-trained second residual dense block to obtain a second enhanced feature;

[0343] The second enhanced feature is scaled up according to the scale-up factor to obtain a simulated input feature of the vth scale.

[0344] In an optional embodiment, the pre-trained multi-scale fusion layer includes P pre-trained average pooling layers, L pre-trained second feature processing layers, a pre-trained fourth convolution layer, and a pre-trained third activation function layer: the first background color removal subunit is specifically used for:

[0345] Inputting the simulated input features of the vth scale into the vth pre-trained average pooling layer to obtain the vth average pooling result;

[0346] Add the P average pooling results to obtain an initial fusion feature;

[0347] Iterate and perform the following operations until the Lth second initial feature is obtained and then stop the following operations;

[0348] Input the tth feature to be processed into the tth pre-trained second processing feature layer to obtain the tth second initial feature;

[0349] Among them, the t-th second initial feature is the t+1-th feature to be processed, the 1st feature to be processed is the initial fusion feature, and t is 1, 2, 3, ..., L in sequence;

[0350] Input the Lth second initial feature into the pre-trained fourth convolution layer to obtain the fourth convolution result;

[0351] Inputting the fourth convolution result into the pre-trained third activation function layer to obtain weights corresponding to the simulated input features of P scales;

[0352] According to the weights, weighted summation is performed on the simulated input features of P scales to obtain fused features.

[0353] In an optional embodiment, the pre-trained second background color removal network includes a pre-trained third residual dense block and N pre-trained second background color removal blocks; the second background color removal submodule is specifically used for:

[0354] Inputting the first purified image feature into the pre-trained third residual dense block to obtain a third enhanced feature;

[0355] Iterate the following operations until the Nth second background color removal feature is obtained and then stop the following operations:

[0356] Inputting the jth second background color feature to be removed into the jth pre-trained second background color removal block to obtain the jth second background color removal feature;

[0357] Among them, the jth second background color removal feature is the j+1th second background color feature to be removed, the 1st first background color feature to be removed is the third enhancement feature, the Nth second background color removal feature is the second purified image feature, and j is 1, 2, 3,..., N respectively.

[0358] In an optional embodiment, the pre-trained third background color removal network includes M+1 pre-trained second background color removal sub-networks; the third background color removal sub-module specifically includes:

[0359] The iteration unit is used to iteratively perform the following operations until the first second impurity purification feature is obtained and the following operations are stopped:

[0360] A third background color removal unit is used to input the i-th second impurity feature to be purified into the i-th pre-trained second background color removal sub-network to obtain the i-th second impurity purification feature;

[0361] Among them, the i-th second impurity purification feature is the i-1-th second impurity feature to be purified, the M+1-th second impurity feature to be purified is the second purified image feature, the 1st second impurity purification feature is the target image, and i is M+1,...,3,2,1 respectively.

[0362] In an optional embodiment, the i-th pre-trained second background color removal subnetwork includes a pre-trained fourth residual dense block, N pre-trained third background color removal blocks, a pre-trained fifth convolutional layer, a pre-trained sixth convolutional layer and a pre-trained upsampling layer; the third background color removal unit specifically includes:

[0363] A second impurity feature to be purified input subunit, used for inputting the i-th second impurity feature to be purified into the pre-trained fifth convolution layer to obtain a fifth convolution result;

[0364] A fifth convolution result input subunit, used for inputting the fifth convolution result into the pre-trained fourth residual dense block to obtain a fourth enhanced feature;

[0365] The iterative subunit is used to iteratively perform the following operations until the Nth third background color removal feature is obtained and the following operations are stopped:

[0366] A third background feature to be removed input subunit, used for inputting the jth third background feature to be removed into the jth pre-trained third background color removal block to obtain the jth third background removal feature;

[0367] The jth third background removal feature is the j+1th third background feature to be removed, the first third background feature to be removed is the fourth enhanced feature, and j is 1, 2, 3, ..., N in sequence;

[0368] A third background color removal feature input subunit is used to input the Nth third background color removal feature into the pre-trained sixth convolution layer to obtain a sixth convolution result;

[0369] The sixth convolution result input subunit is used to input the sixth convolution result into the pre-trained upsampling layer to obtain the i-th second impurity purification feature.

[0370] In an optional embodiment, the second impurity characteristic input subunit to be purified is specifically used for:

[0371] Determine an i-th pre-trained first background color removal sub-network at the same level as the i-th pre-trained second background color removal sub-network;

[0372] Add the i-th first impurity purification feature output by the i-th pre-trained first background color removal subnetwork to the i-th second impurity feature to be purified to obtain a short connection feature;

[0373] The short connection feature is input into the pre-trained fifth convolutional layer to obtain a fifth convolution result.

[0374] In an optional embodiment, the device further comprises:

[0375] An image pair set acquisition module, used for acquiring an image pair set, wherein the image pairs in the image pair set include a sample image and a standard image, and the impurities contained in the sample image cover different impurities;

[0376] A sample image input module, used for acquiring the image pair from the image pair set, inputting the sample image in the image pair into a universal purification network, and obtaining a predicted image;

[0377] a loss value determination module, configured to determine a loss value between the predicted image and the standard image in the image pair;

[0378] The network training module is used to perform network training on the universal purification network according to the loss value, and stop the network training when the loss value converges to obtain a pre-trained universal purification network.

[0379] In an optional implementation, the sample image input module specifically includes:

[0380] A sample image cropping submodule, used for cropping the sample image in the image pair to obtain at least one image sub-region;

[0381] An image sub-region compression sub-module, used for compressing any of the image sub-regions to obtain a compressed image sub-region;

[0382] A compressed image sub-region input sub-module, used for inputting the compressed image sub-region into a universal purification network to obtain a predicted image sub-region;

[0383] The loss value determination module is specifically used for:

[0384] searching, from the standard image of the image pair, a standard image sub-region corresponding to the image sub-region;

[0385] A loss value between the standard image sub-region and the predicted image sub-region is determined.

[0386] In an optional embodiment, the universal purification network includes a first background color removal network, a second background color removal network, and a third background color removal network; the compressed image sub-region input submodule specifically includes:

[0387] A compressed image sub-region input unit, used for inputting the compressed image sub-region into the first background color removal network to obtain a first predicted purified image feature;

[0388] a first predicted cleansed image feature input unit, configured to input the first predicted cleansed image feature into the second background color removal network to obtain a second predicted cleansed image feature;

[0389] The second predicted cleaned image feature input unit is used to input the second predicted cleaned image feature into the third background color removal network to obtain a predicted image sub-region.

[0390] In an optional implementation, the universal purification network further includes a downsampling network and a convolutional network; the compressed image sub-region input unit specifically includes:

[0391] A compressed image sub-region down-sampling sub-unit, used for inputting the compressed image sub-region into the down-sampling network to obtain the compressed image sub-region after down-sampling;

[0392] A compressed image sub-region convolution sub-unit, used for inputting the down-sampled compressed image sub-region into the convolution network to obtain a first predicted convolution result;

[0393] The first predicted convolution result input subunit is used to input the first predicted convolution result into the first background color removal network to obtain a first predicted purified image feature.

[0394] In an optional embodiment, the first background color removal network includes M first background color removal sub-networks, where M is a positive integer; and the first predicted convolution result input sub-unit is specifically used for:

[0395] Iteratively perform the following operations until the Mth first predicted impurity purification feature is obtained and then stop the following operations;

[0396] Inputting the i-th first predicted impurity feature to be purified into the i-th first background color removal sub-network to obtain the i-th first predicted impurity purification feature;

[0397] Among them, the i-th first predicted impurity purification feature is the i+1-th first predicted impurity feature to be purified, the 1st first predicted impurity feature to be purified is the first predicted convolution result, the M-th first predicted impurity purification feature is the first predicted purified image feature, and i is 1, 2, 3,..., M respectively.

[0398] In an optional embodiment, the i-th first background color removal sub-network includes a first residual dense block, N first background color removal blocks, a first convolution layer, and a first activation function layer, where N is a positive integer; the first predicted convolution result input subunit is specifically used for:

[0399] Inputting the i-th first predicted impurity feature to be purified into the first residual dense block to obtain a first predicted enhanced image feature;

[0400] Iterate the following steps until the Nth first predicted background color removal feature is obtained and then stop the following operations:

[0401] Inputting the jth first predicted background color feature to be removed into the jth first background color removal block to obtain the jth first predicted background color removal feature;

[0402] The jth first predicted background color removal feature is the j+1th first predicted background color feature to be removed, the first first predicted background color feature to be removed is the first predicted enhanced image feature, and j is 1, 2, 3, ..., N in sequence;

[0403] Inputting the Nth first predicted background color removal feature into the first convolution layer to obtain a second predicted convolution result;

[0404] The second predicted convolution result is input into the first activation function layer to obtain the i-th first predicted impurity purification feature.

[0405] In an optional implementation, the first residual dense block includes O first feature processing layers and a second convolution layer, where O is a positive integer; and the first prediction convolution result input subunit is specifically used for:

[0406] Iterate and perform the following operations until the Oth prediction residual feature is obtained and stop the following operations:

[0407] Inputting the kth predicted feature to be enhanced into the kth first feature processing layer to obtain the kth first predicted initial feature;

[0408] Add the kth predicted feature to be enhanced to the kth first predicted initial feature to obtain the kth predicted residual feature;

[0409] The kth predicted residual feature is the k+1th predicted feature to be enhanced, the 1st predicted feature to be enhanced is the i-th first predicted impurity feature to be purified, and k is 1, 2, 3, ..., 0 in sequence;

[0410] The Oth predicted residual feature is input into the second convolutional layer to obtain the first predicted enhanced image feature.

[0411] In an optional implementation, the kth first feature processing layer includes a third convolution layer and a second activation function layer; and the first prediction convolution result input subunit is specifically used for:

[0412] Inputting the kth predicted feature to be enhanced into the third convolution layer to obtain a third predicted convolution result;

[0413] The third prediction convolution result is input into the second activation function layer to obtain the kth first prediction initial feature.

[0414] In an optional implementation, the j-th first background color removal block includes P-scale simulation input layers and a multi-scale fusion layer, where P is a positive integer; and the first predicted convolution result input subunit is specifically used for:

[0415] Inputting the jth first predicted background color feature to be removed into the vth scale simulation input layer to obtain the vth scale predicted simulation input feature, wherein v is 1, 2, 3, ..., P in sequence;

[0416] Inputting the predicted simulation input features of P scales into the multi-scale fusion layer to obtain predicted fusion features;

[0417] The j-th first predicted background color feature to be removed is added to the predicted fusion feature to obtain the j-th first predicted background color removal feature.

[0418] In an optional implementation, the analog input layer of the vth scale includes a second residual dense block; the first prediction convolution result input subunit is specifically used for:

[0419] Determine the size reduction factor and size increase factor corresponding to the pre-trained simulated input layer of the vth scale;

[0420] According to the size reduction ratio, the j-th first predicted background color feature to be removed is reduced in size to obtain the j-th first predicted background color feature to be removed after reduction;

[0421] Inputting the reduced j-th first predicted background color feature to be removed into the second residual dense block to obtain a second predicted enhanced feature;

[0422] The second prediction enhancement feature is scaled up according to the scale-up factor to obtain a prediction simulation input feature of the vth scale.

[0423] In an optional embodiment, the multi-scale fusion layer includes P average pooling layers, L second feature processing layers, a fourth convolution layer, and a third activation function layer: the first prediction convolution result input subunit is specifically used for:

[0424] Inputting the predicted simulated input features of the vth scale into the vth average pooling layer to obtain the vth predicted average pooling result;

[0425] Add the P predicted average pooling results to obtain the predicted initial fusion feature;

[0426] Iterate and perform the following operations until the Lth second prediction initial feature is obtained and then stop the following operations;

[0427] Input the tth predicted feature to be processed into the tth second processing feature layer to obtain the tth second predicted initial feature;

[0428] Among them, the t-th second prediction initial feature is the t+1-th prediction feature to be processed, the 1st prediction feature to be processed is the prediction initial fusion feature, and t is 1, 2, 3, ..., L in sequence;

[0429] Input the Lth second prediction initial feature into the fourth convolution layer to obtain the fourth prediction convolution result;

[0430] Inputting the fourth prediction convolution result into the third activation function layer to obtain prediction weights corresponding to the prediction simulation input features of P scales;

[0431] According to the prediction weights, the prediction simulation input features of P scales are weighted and summed to obtain the prediction fusion features.

[0432] In an optional embodiment, the second background color removal network includes a third residual dense block and N second background color removal blocks; and the first predicted purified image feature input unit is specifically used for:

[0433] Inputting the first predicted purified image feature into the third residual dense block to obtain a third predicted enhanced feature;

[0434] Iterate the following operations until the Nth second predicted background color removal feature is obtained and then stop the following operations:

[0435] Inputting the jth second predicted background color feature to be removed into the jth second background color removal block to obtain the jth second predicted background color removal feature;

[0436] Among them, the jth second predicted background color removal feature is the j+1th second predicted background color feature to be removed, the 1st first predicted background color feature to be removed is the third predicted enhancement feature, the Nth second predicted background color removal feature is the second predicted purified image feature, and j is 1, 2, 3,..., N respectively.

[0437] In an optional embodiment, the third background color removal network includes M+1 second background color removal sub-networks; and the second predicted cleaned image feature input unit specifically includes:

[0438] The iterative subunit is used to iteratively perform the following operations until the first second predicted impurity purification feature is obtained and the following operations are stopped:

[0439] A second predicted impurity feature to be purified input subunit, used for inputting the i-th second predicted impurity feature to be purified into the i-th second background color removal subnetwork to obtain the i-th second predicted impurity purification feature;

[0440] Among them, the i-th second predicted impurity purification feature is the i-1-th second predicted impurity feature to be purified, the M+1-th second predicted impurity feature to be purified is the second predicted purified image feature, the 1st second predicted impurity purification feature is the predicted image sub-region, and i is M+1,...,3,2,1 respectively.

[0441] In an optional embodiment, the i-th second background color removal sub-network includes a fourth residual dense block, N third background color removal blocks, a fifth convolutional layer, a sixth convolutional layer and an upsampling layer; the second predicted impurity feature input sub-unit to be purified is specifically used for:

[0442] Inputting the i-th second predicted impurity feature to be purified into the fifth convolution layer to obtain a fifth predicted convolution result;

[0443] Inputting the fifth prediction convolution result into the fourth residual dense block to obtain a fourth prediction enhancement feature;

[0444] Iterate the following operations until the Nth third predicted background color removal feature is obtained and then stop the following operations:

[0445] Inputting the jth third predicted background feature to be removed into the jth third background color removal block to obtain the jth third predicted background removal feature;

[0446] The jth third predicted background removal feature is the j+1th third predicted background feature to be removed, the first third predicted background feature to be removed is the fourth predicted enhancement feature, and j is 1, 2, 3, ..., N in sequence;

[0447] Inputting the Nth third predicted background color removal feature into the sixth convolution layer to obtain a sixth predicted convolution result;

[0448] The sixth predicted convolution result is input into the upsampling layer to obtain the i-th second predicted impurity purification feature.

[0449] In an optional embodiment, the second predicted impurity feature input subunit to be purified is specifically used for:

[0450] Determine an i-th first background color removal sub-network at the same level as the i-th second background color removal sub-network;

[0451] Add the i-th first predicted impurity purification feature output by the i-th first background color removal subnetwork to the i-th second predicted impurity feature to be purified to obtain a predicted short connection feature;

[0452] The predicted short connection feature is input into the fifth convolution layer to obtain a fifth predicted convolution result.

[0453] The present application also provides an electronic device, such as Fig.16 As shown, it includes a processor 161, a communication interface 162, a memory 163 and a communication bus 164, wherein the processor 161, the communication interface 162, and the memory 163 communicate with each other through the communication bus 164.

[0454] Memory 163, for storing computer programs;

[0455] The processor 161 is used to execute the program stored in the memory 163 to implement the following steps:

[0456] An image to be processed is obtained, wherein the image to be processed contains impurities; a background color removal process is performed on the image to be processed to obtain a target image, wherein the target image does not contain impurities.

[0457] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0458] The communication interface is used for communication between the above electronic device and other devices.

[0459] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0460] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0461] In another embodiment provided in the present application, a storage medium is provided, in which instructions are stored. When the instructions are executed on a computer, the computer executes any one of the image processing methods in the above embodiments.

[0462] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the image processing methods in the above embodiments.

[0463] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium, or transmitted from one storage medium to another storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0464] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0465] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0466] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed, wherein the image to be processed contains impurities; Background color removal is performed on the image to be processed to obtain a target image, wherein the target image does not contain the impurities.

2. The method according to claim 1, characterized in that The step of performing background color removal processing on the image to be processed to obtain a target image includes: The pre-trained universal purification network is used to perform background color removal processing on the image to be processed to obtain a target image.

3. The method according to claim 2, characterized in that The pre-trained universal purification network includes at least one pre-trained first background color removal network, a pre-trained second background color removal network, and a pre-trained third background color removal network; The method of using a pre-trained universal purification network to remove background color from the image to be processed to obtain a target image includes: In the case where the pre-trained universal purification network includes a pre-trained first background color removal network, the image to be processed is input into the pre-trained first background color removal network to obtain a first purified image feature; or, in the case where the pre-trained universal purification network includes two or more pre-trained first background color removal networks, the image to be processed is input into the pre-trained first background color removal networks in sequence to obtain a first purified image feature; Inputting the first purified image feature into the pre-trained second background color removal network to obtain a second purified image feature; The second cleaned image features are input into the pre-trained third background color removal network to obtain a target image.

4. The method according to claim 2, characterized in that: The pre-trained universal purification network includes at least one pre-trained first background color removal network and a pre-trained third background color removal network; The method of using a pre-trained universal purification network to remove background color from the image to be processed to obtain a target image includes: Inputting the image to be processed into the pre-trained first background color removal network in sequence to obtain a first purified image feature; The first cleaned image feature is input into the pre-trained third background color removal network to obtain a target image.

5. The method according to claim 3, characterized in that: The pre-trained universal purification network also includes a pre-trained downsampling network and a pre-trained convolutional network; The step of inputting the image to be processed into the pre-trained first background color removal network to obtain a first purified image feature includes: Inputting the image to be processed into the pre-trained downsampling network to obtain the down-sampled image to be processed; Inputting the downsampled image to be processed into the pre-trained convolutional network to obtain a first convolution result; The first convolution result is input into the pre-trained first background color removal network to obtain a first purified image feature.

6. The method according to claim 5, characterized in that The pre-trained first background color removal network includes M pre-trained first background color removal sub-networks, where M is a positive integer; The step of inputting the first convolution result into the pre-trained first background color removal network to obtain a first purified image feature includes: Iteratively perform the following operations until the Mth first impurity purification feature is obtained and then stop the following operations; Inputting the i-th first impurity feature to be purified into the i-th pre-trained first background color removal sub-network to obtain the i-th first impurity purification feature; Among them, the i-th first impurity purification feature is the i+1-th first impurity feature to be purified, the 1st first impurity feature to be purified is the first convolution result, the M-th first impurity purification feature is the first purified image feature, and i is 1, 2, 3, ..., M respectively.

7. The method according to claim 6, characterized in that The i-th pre-trained first background color removal sub-network includes a pre-trained first residual dense block, N pre-trained first background color removal blocks, a pre-trained first convolutional layer, and a pre-trained first activation function layer, where N is a positive integer; The step of inputting the i-th first impurity feature to be purified into the i-th pre-trained first background color removal sub-network to obtain the i-th first impurity purification feature comprises: Inputting the i-th first impurity feature to be purified into the pre-trained first residual dense block to obtain a first enhanced image feature; Iterate the following steps until the Nth first background color removal feature is obtained and then stop the following operations: Inputting the jth first background color feature to be removed into the jth pre-trained first background color removal block to obtain the jth first background color removal feature; The jth first background color removal feature is the j+1th first background color feature to be removed, the first first background color feature to be removed is the first enhanced image feature, and j is 1, 2, 3, ..., N in sequence; Inputting the Nth first background color removal feature into the pre-trained first convolution layer to obtain a second convolution result; The second convolution result is input into the pre-trained first activation function layer to obtain the i-th first impurity purification feature.

8. The method according to claim 7, characterized in that The pre-trained first residual dense block includes O pre-trained first feature processing layers and a pre-trained second convolutional layer, where O is a positive integer; The step of inputting the i-th first impurity feature to be purified into the pre-trained first residual dense block to obtain a first enhanced image feature includes: Iterate and perform the following operations until the Oth residual feature is obtained and stop the following operations: Inputting the kth feature to be enhanced into the kth pre-trained first feature processing layer to obtain the kth first initial feature; Add the kth feature to be enhanced to the kth first initial feature to obtain the kth residual feature; The kth residual feature is the k+1th feature to be enhanced, the 1st feature to be enhanced is the i-th first impurity feature to be purified, and k is 1, 2, 3, ..., 0 in sequence; The Oth residual feature is input into the second convolutional layer of the pre-training to obtain the first enhanced image feature.

9. The method according to claim 8, characterized in that The kth pre-trained first feature processing layer includes a pre-trained third convolutional layer and a pre-trained second activation function layer; The step of inputting the kth feature to be enhanced into the kth pre-trained first feature processing layer to obtain the kth first initial feature comprises: Inputting the kth feature to be enhanced into the pre-trained third convolutional layer to obtain a third convolution result; The third convolution result is input into the pre-trained second activation function layer to obtain the kth first initial feature.

10. The method according to claim 7, characterized in that The j-th pre-trained first background color removal block includes a pre-trained analog input layer of P scales and a pre-trained multi-scale fusion layer, where P is a positive integer; The step of inputting the jth first background color feature to be removed into the jth pre-trained first background color removal block to obtain the jth first background color removal feature comprises: Input the jth first background color feature to be removed into the vth scale pre-trained analog input layer to obtain the vth scale analog input feature, wherein v is 1, 2, 3, ..., P in sequence; Inputting the simulated input features of P scales into the pre-trained multi-scale fusion layer to obtain fused features; The j-th first background color feature to be removed is added to the fusion feature to obtain the j-th first background color removal feature.

11. The method according to claim 10, characterized in that The pre-trained simulated input layer of the vth scale includes the pre-trained second residual dense block; The step of inputting the jth first background color feature to be removed into the pre-trained analog input layer of the vth scale to obtain the analog input feature of the vth scale includes: Determine the size reduction factor and size increase factor corresponding to the pre-trained simulated input layer of the vth scale; According to the size reduction ratio, the j-th first background color feature to be removed is reduced in size to obtain the reduced j-th first background color feature to be removed; Inputting the reduced j-th first background color feature to be removed into the pre-trained second residual dense block to obtain a second enhanced feature; The second enhanced feature is scaled up according to the scale-up factor to obtain a simulated input feature of the vth scale.

12. The method according to claim 10, characterized in that The pre-trained multi-scale fusion layer includes P pre-trained average pooling layers, L pre-trained second feature processing layers, a pre-trained fourth convolutional layer, and a pre-trained third activation function layer: The step of inputting the simulated input features of P scales into the pre-trained multi-scale fusion layer to obtain fused features includes: Inputting the simulated input features of the vth scale into the vth pre-trained average pooling layer to obtain the vth average pooling result; Add the P average pooling results to obtain an initial fusion feature; Iterate and perform the following operations until the Lth second initial feature is obtained and then stop the following operations; Input the tth feature to be processed into the tth pre-trained second processing feature layer to obtain the tth second initial feature; Among them, the t-th second initial feature is the t+1-th feature to be processed, the 1st feature to be processed is the initial fusion feature, and t is 1, 2, 3, ..., L in sequence; Input the Lth second initial feature into the pre-trained fourth convolution layer to obtain the fourth convolution result; Inputting the fourth convolution result into the pre-trained third activation function layer to obtain weights corresponding to the simulated input features of P scales; According to the weights, weighted summation is performed on the simulated input features of P scales to obtain fused features.

13. The method according to claim 3, characterized in that The pre-trained second background color removal network includes a pre-trained third residual dense block and N pre-trained second background color removal blocks; The step of inputting the first purified image feature into the pre-trained second background color removal network to obtain a second purified image feature comprises: Inputting the first purified image feature into the pre-trained third residual dense block to obtain a third enhanced feature; Iterate the following operations until the Nth second background color removal feature is obtained and then stop the following operations: Inputting the jth second background color feature to be removed into the jth pre-trained second background color removal block to obtain the jth second background color removal feature; Among them, the jth second background color removal feature is the j+1th second background color feature to be removed, the 1st first background color feature to be removed is the third enhancement feature, the Nth second background color removal feature is the second purified image feature, and j is 1, 2, 3,..., N respectively.

14. The method according to claim 3, characterized in that The pre-trained third background color removal network includes M+1 pre-trained second background color removal sub-networks; The step of inputting the second purified image feature into the pre-trained third background color removal network to obtain a target image comprises: Iterate the following operations until the first second impurity purification feature is obtained and then stop the following operations: Inputting the i-th second impurity feature to be purified into the i-th pre-trained second background color removal sub-network to obtain the i-th second impurity purification feature; Among them, the i-th second impurity purification feature is the i-1-th second impurity feature to be purified, the M+1-th second impurity feature to be purified is the second purified image feature, the 1st second impurity purification feature is the target image, and i is M+1,...,3,2,1 respectively.

15. The method according to claim 14, characterized in that The i-th pre-trained second background color removal sub-network includes a pre-trained fourth residual dense block, N pre-trained third background color removal blocks, a pre-trained fifth convolutional layer, a pre-trained sixth convolutional layer and a pre-trained upsampling layer; The step of inputting the i-th second impurity feature to be purified into the i-th pre-trained second background color removal sub-network to obtain the i-th second impurity purification feature comprises: Inputting the i-th second impurity feature to be purified into the pre-trained fifth convolution layer to obtain a fifth convolution result; Inputting the fifth convolution result into the pre-trained fourth residual dense block to obtain a fourth enhanced feature; Iterate the following operations until the Nth third background color removal feature is obtained and then stop the following operations: Inputting the jth third background feature to be removed into the jth pre-trained third background color removal block to obtain the jth third background removal feature; The jth third background removal feature is the j+1th third background feature to be removed, the first third background feature to be removed is the fourth enhanced feature, and j is 1, 2, 3, ..., N in sequence; Inputting the Nth third background color removal feature into the pre-trained sixth convolution layer to obtain a sixth convolution result; The sixth convolution result is input into the pre-trained upsampling layer to obtain the i-th second impurity purification feature.

16. The method according to claim 15, characterized in that The step of inputting the i-th second impurity feature to be purified into the pre-trained fifth convolution layer to obtain a fifth convolution result includes: Determine an i-th pre-trained first background color removal sub-network at the same level as the i-th pre-trained second background color removal sub-network; Add the i-th first impurity purification feature output by the i-th pre-trained first background color removal subnetwork to the i-th second impurity feature to be purified to obtain a short connection feature; The short connection feature is input into the pre-trained fifth convolutional layer to obtain a fifth convolution result.

17. The method according to claim 2, characterized in that Before executing the image processing method, the method further includes: Acquire an image pair set, wherein the image pairs in the image pair set include a sample image and a standard image, and the impurities contained in the sample image cover different impurities; Acquire the image pair from the image pair set, input the sample image in the image pair into a universal purification network, and obtain a predicted image; Determine a loss value between the predicted image and the standard image in the image pair, and perform network training on the universal purification network according to the loss value; When the loss value converges, the network training is stopped to obtain a pre-trained universal purification network.

18. The method according to claim 17, characterized in that The step of inputting the sample image in the image pair into a universal purification network to obtain a predicted image comprises: Cropping the sample image in the image pair to obtain at least one image sub-region; For any of the image sub-regions, compress the image sub-region to obtain a compressed image sub-region; Inputting the compressed image sub-region into a universal purification network to obtain a predicted image sub-region; The determining a loss value between the predicted image and the standard image in the image pair comprises: searching, from the standard image of the image pair, a standard image sub-region corresponding to the image sub-region; A loss value between the standard image sub-region and the predicted image sub-region is determined.

19. The method according to claim 18, characterized in that The universal purification network includes a first background color removal network, a second background color removal network, and a third background color removal network; The step of inputting the compressed image sub-region into a universal purification network to obtain a predicted image sub-region comprises: Inputting the compressed image sub-region into the first background color removal network to obtain a first predicted purified image feature; Inputting the first predicted purified image feature into the second background color removal network to obtain a second predicted purified image feature; The second predicted cleaned image feature is input into the third background color removal network to obtain a predicted image sub-region.

20. The method according to claim 19, characterized in that The universal purification network also includes a downsampling network and a convolutional network; The step of inputting the compressed image sub-region into the first background color removal network to obtain a first predicted purified image feature includes: Inputting the compressed image sub-region into the down-sampling network to obtain the compressed image sub-region after down-sampling; Inputting the compressed image sub-region after downsampling into the convolution network to obtain a first predicted convolution result; The first predicted convolution result is input into the first background color removal network to obtain a first predicted purified image feature.

21. An image processing device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be processed, wherein the image to be processed contains impurities; The image processing module is used to perform background color removal processing on the image to be processed to obtain a target image, wherein the target image does not contain the impurities.

22. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 20 is implemented.

Images