Image processing method and device, storage medium, electronic equipment and program product
Through the convolution processing, downsampling and upsampling technology of the image defuzzing model, the problem of network architecture and information loss in the image defuzzing process is solved, and efficient and accurate image defuzzing effect is achieved.
Patent Information
- Application Number
- CN202510422274.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-03
- Publication Date
- 2025-08-15
AI Technical Summary
In the image defuzzing process, the prior art there are problems of information loss caused by network architecture that ignores the blur scale, inefficient cross-scale information propagation and downsampling, which leads to unreliable and inaccurate results of the defuzzing process.
The image defuzzing model is used to pass the first convolution layer convolution processing, combining the downsampling of the first feature extraction layer and the upsampling of the second feature extraction layer to generate clear image features, avoid image feature loss, and improve the reliability and accuracy of the defuzzing process.
Effective defuzzing of blurred images of different sizes is realized, the reliability and accuracy of the defuzzing results are improved, and the robustness and applicability of the image defuzzing model are enhanced.
Smart Images
Figure CN120495126A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular, to an image processing method, device, storage medium, electronic device, and program product. Background Art
[0002] Images are an essential medium for receiving and transmitting information in our daily lives. However, various factors, such as jitter in the image capturing device or blur caused by motion, can degrade image quality and produce blurry images. In the field of Optical Character Recognition (OCR), image quality directly impacts OCR accuracy. Summary of the Invention
[0003] In order to address the deficiencies of the prior art, the present disclosure aims to provide an image processing method, apparatus, storage medium, electronic device, and program product.
[0004] In order to achieve the above objectives, in a first aspect, the present disclosure provides an image processing method, the method comprising: Acquire a target image by an image acquisition device; Inputting the target image into a trained image deblurring model to obtain a clear image output by the image deblurring model, wherein the clear image has a higher clarity than the target image; The image deblurring model outputs the clear image in the following manner: performing convolution processing on the target image through a first convolution layer to obtain a first image feature, down-sampling the first image feature through a first feature extraction layer to obtain a second image feature, up-sampling the second image feature through a second feature extraction layer to obtain a third image feature, and convolution processing on the third image feature through a second convolution layer to obtain a target image feature, which is used to generate a clear image.
[0005] In a second aspect, the present disclosure provides an image processing device, comprising: an acquisition module, configured to acquire a target image through an image acquisition device; a processing module configured to input the target image into a trained image deblurring model to obtain a clear image output by the image deblurring model, wherein the clear image has a higher clarity than the target image; The image deblurring model outputs the clear image in the following manner: performing convolution processing on the target image through a first convolution layer to obtain a first image feature, down-sampling the first image feature through a first feature extraction layer to obtain a second image feature, up-sampling the second image feature through a second feature extraction layer to obtain a third image feature, and convolution processing on the third image feature through a second convolution layer to obtain a target image feature, which is used to generate a clear image.
[0006] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when executed by a processor.
[0007] In a fourth aspect, the present disclosure provides an electronic device, comprising: a memory having a computer program stored thereon; A processor is configured to execute the computer program in the memory to implement the method described in the first aspect.
[0008] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.
[0009] Through the above technical solution, the target image captured by the image acquisition device is input into the trained image deblurring model. The image deblurring model performs convolution processing on the target image through a first convolutional layer to obtain a first image feature. The first image feature is downsampled through a first feature extraction layer to obtain a second image feature. The second image feature is upsampled through a second feature extraction layer to obtain a third image feature. The third image feature is convoluted through a second convolutional layer to obtain a target image feature for generating a clear image, and a clear image with higher clarity than the target image is output. The image deblurring model combines downsampling and upsampling to achieve multi-dimensional image feature extraction of the target image, avoid the loss of image features during the deblurring process, and improve the reliability and accuracy of the deblurring processing results.
[0010] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The figure is a flowchart showing an image processing method according to an exemplary embodiment of the present disclosure.
[0012] Figure 2 FIG. 4 is a schematic diagram of an image deblurring model according to an exemplary embodiment of the present disclosure.
[0013] Figure 3 FIG. 4 is another schematic diagram of an image demodulation model according to an exemplary embodiment of the present disclosure.
[0014] Figure 4a FIG. 4 is a schematic diagram showing a coding layer according to an exemplary embodiment of the present disclosure.
[0015] Figure 4b FIG. 4 is a schematic diagram showing a decoding layer according to an exemplary embodiment of the present disclosure.
[0016] Figure 5 FIG. 4 is a schematic diagram of a feature extraction layer according to an exemplary embodiment of the present disclosure.
[0017] Figure 6 FIG. 4 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure.
[0018] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0019] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.
[0020] As mentioned in the background art, in the field of OCR, the quality of image imaging will directly affect the recognition accuracy of OCR.
[0021] In related art, a small blur kernel and a clear image are estimated at a coarse scale for the blurred image, which are then used as the initial solution for the next scale. Based on the small image and blur size at the coarse scale, the clear image corresponding to the blurred image can be quickly estimated. Furthermore, the small blur size at the coarse scale allows for more accurate estimation of the blur kernel and latent image. This quickly provides an accurate initial solution for the next scale, improving deblurring quality and efficiency.
[0022] The inventors discovered that the related art has the following defects: 1. Blur-Scale-Agnostic Network Architecture. When deblurring an image, restoring the pixel value of a specific pixel requires a receptive field larger than the blur size. Therefore, larger blur sizes require a larger receptive field or a deeper network. Accordingly, the aforementioned related art requires a deeper sub-network for deblurring larger-scale images with higher precision. However, the related art uses the same network architecture across different scales, making it impossible to deblur larger-scale images with higher precision.
[0023] 2. Inefficient cross-scale information propagation. In related technologies, the pixel values of the deblurred image are transferred from the coarse scale to the next scale during the deblurring process, which leads to a significant loss of the rich information encoded in the feature vector of the coarse scale, ultimately reducing the deblurring performance.
[0024] 3. Information loss due to downsampling. When generating multi-scale blurred input images, related technologies construct an image pyramid by repeatedly downsampling the input blurred image. However, deblurring through downsampling results in significant information loss, making the resulting clear image unreliable.
[0025] In view of this, the present disclosure provides an image processing method, apparatus, storage medium, electronic device, and program product, which can perform deblurring on blurred images of different sizes and improve the reliability and accuracy of the deblurring results.
[0026] Figure 1 This is a flowchart of an image processing method proposed according to an exemplary embodiment of the present disclosure, which can be used in smart terminals such as laptops, smart phones, etc. Figure 1 As shown, the method may include the following steps: In step S11 , a target image is captured by an image capture device.
[0027] It should be understood that the image acquisition device may be a video camera, a camera, or an intelligent device with a camera function, etc. This disclosure does not limit this.
[0028] In step S12, the target image is input into the trained image deblurring model to obtain a clear image output by the image deblurring model, and the clarity of the clear image is higher than that of the target image; Among them, such as Figure 2 As shown, the image deblurring model outputs a clear image in the following manner: convolution processing is performed on the target image through the first convolution layer to obtain the first image feature, the first image feature is down-sampled through the first feature extraction layer to obtain the second image feature, the second image feature is up-sampled through the second feature extraction layer to obtain the third image feature, and the third image feature is convolution processing is performed on the second convolution layer to obtain the target image feature. The target image feature is used to generate a clear image.
[0029] It should be understood that the image deblurring model also includes an input layer and an output layer. The input layer is used to divide the input target image into multiple color channels, and the output layer is used to merge the image features into a clear image.
[0030] In the disclosed embodiment, a target image is captured by an image capture device, and the target image is input into a trained image deblurring model. The image deblurring model performs convolution processing on the target image through a first convolutional layer to obtain a first image feature, downsamples the first image feature through a first feature extraction layer to obtain a second image feature, upsamples the second image feature through a second feature extraction layer to obtain a third image feature, and convolutes the third image feature through a second convolutional layer to obtain a target image feature for generating a clear image, and outputs a clear image with a higher clarity than the target image. The image deblurring model combines downsampling and upsampling to achieve multi-dimensional image feature extraction of the target image, avoids the loss of image features during the deblurring process, and improves the reliability and accuracy of the deblurring processing results.
[0031] In order to facilitate those skilled in the art to better understand the image processing method provided by the present disclosure, the steps of the method are described in detail below.
[0032] In one possible implementation, Figure 3 As shown, the first feature extraction layer and the second feature extraction layer are connected through an intermediate layer, the first feature extraction layer includes at least one encoding layer, the second feature extraction layer includes at least one decoding layer, and the number of encoding layers and decoding layers is the same; Input the target image into the trained image deblurring model, which can include: Scale the target image to a preset size and input it into the trained image deblurring model; Downsampling the first image feature through the first feature extraction layer to obtain the second image feature, and upsampling the second image feature through the second feature extraction layer to obtain the third image feature may include: extracting a first intermediate image feature from the first image feature through at least one encoding layer, and performing a two-fold downsampling process on the first intermediate image feature to obtain a second image feature; The second image feature is subjected to a double upsampling process through at least one decoding layer to obtain a second intermediate image feature, and a third image feature is extracted from the second intermediate image feature.
[0033] It is worth noting that the preset size can be preset according to the processing capability of the image deblurring model, and this disclosure does not limit this.
[0034] It is worth noting that, based on the inference capabilities of the image deblurring model, the first feature extraction layer includes four encoding layers, each of which can be composed of an encoder block module; the second feature extraction layer includes four decoding layers, each of which can be composed of a decoder block module. The intermediate layer can be composed of either a moddle block module or a deblurring block module. The intermediate layer can connect the first and second feature extraction layers in series and transmit the output of the first feature extraction layer to the second feature extraction layer. The first and second convolutional layers can each be composed of a convblock module.
[0035] For example, Figure 3 As shown, the first feature extraction layer includes four encoder block modules and the second feature extraction layer includes four decoder block modules as an example. The target image is scaled to 512*512 size and input into the trained image deblurring model. The input layer input of the image deblurring model outputs a 1*3*512*512 image based on the 512*512 target image. Among them, the output of each layer in the image deblurring model is i*j*n*n, where i represents the number of images, j represents the color channel, and n*n represents the image size. For example, 1*3*512*512 represents 1 image, 3 color channels, and the size is 512*512 pixels; The conv block module performs convolution processing on 1*3*512*512 to obtain 1*16*512*512. The first encoder block module performs the first downsampling on 1*16*512*512 to obtain 1*32*256*256. The second encoder block module performs the second downsampling on 1*32*256*256 to obtain 1*64*128*128. The third encoder block module performs the third downsampling on 1*64*128*128 to obtain 1*128*64*64. The fourth encoder block module performs the fourth downsampling on 1*128*64*64 to obtain 1*256*32*32. The moddle block module transmits 1*256*32*32 to the first decoder block module. The first decoder block module performs the first upsampling of 1*256*32*32 to obtain 1*128*64*64; the second decoder block module performs the second upsampling of 1*128*64*64 to obtain 1*64*128*128; the third decoder block module performs the third upsampling of 1*64*128*128 to obtain 1*32*256*256; the fourth decoder block module performs the third upsampling of 1*32*256*256 to obtain 1*16*512*512; the conv block module performs convolution processing on 1*16*512*512 to obtain 1*3*512*512; the output layer outputs a clear image based on 1*3*512*512.
[0036] In the disclosed embodiments, a target image is scaled to a preset size and input into a trained image deblurring model, enabling the model to deblur target images of different sizes captured by different image acquisition devices, thereby improving the robustness of the model. Furthermore, the model downsamples first image features by a factor of two through at least one encoding layer to obtain second image features, and then upsamples the second image features by a factor of two through at least one decoding layer. This allows the model to have a certain degree of noise resistance and be applicable to a variety of complex and changing scenarios, thereby improving the applicability of the model and producing a high accuracy rate for the resulting clear image.
[0037] In one possible implementation, Figure 4a As shown in , the encoding layer includes the feature extraction layer and the downsampling layer in sequence, as shown in Figure 4b As shown, the decoding layer includes an upsampling layer and a feature extraction layer in sequence; Extracting a first intermediate image feature from the first image feature through the encoding layer and performing a two-fold downsampling process on the first intermediate image feature to obtain a second image feature may include: Extracting features of the first image through a feature extraction layer to obtain first intermediate image features; Performing a two-fold downsampling process on the first image feature through a downsampling layer to obtain a second image feature; Performing a two-fold upsampling process on the second image feature through the decoding layer to obtain a second intermediate image feature, and extracting a third image feature from the second intermediate image feature, including: Performing a double upsampling process on the second image feature through the upsampling layer to obtain a second intermediate image feature; The feature extraction layer performs feature extraction on the second image feature to obtain a third image feature.
[0038] It is worth noting that if Figure 4a and Figure 4b As shown in the figure, the feature extraction layer in the encoding layer and the decoding layer can be composed of a deblurring block module, the downsampling layer in the encoding layer can be composed of a down function, and the upsampling layer in the decoding layer can be composed of an up function.
[0039] For example, Figure 4a As shown in the figure, the encoder block module first passes through a deblurring block module for feature extraction, and then passes through a down function for double downsampling. The decoder block module first passes through an up function for double upsampling, and then passes through a deblurring block module for feature extraction.
[0040] In the embodiment of the present disclosure, each encoder block module in the image deblurring model first performs feature extraction through the deblurring block module, and then performs two-fold downsampling through the down function. Each decoder block module first performs two-fold upsampling through the up function, and then performs feature extraction through the deblurring block module, thereby realizing multi-channel feature extraction, avoiding image feature loss, and ensuring the accuracy and reliability of the output of the image deblurring processing.
[0041] In one possible implementation, Figure 5 As shown in the figure, the feature extraction layer includes the primary feature extraction layer, the secondary feature extraction layer and the spatial and channel reconstruction module (Spatial and Channel Reconstruction Convolution, SCC) in sequence. The feature extraction layer can perform feature extraction through the following steps: Performing a feature extraction on an initial tensor corresponding to an input image feature through a primary feature extraction layer to obtain a first intermediate tensor, and adding the first intermediate tensor to the initial tensor to obtain a first tensor; Perform secondary feature extraction on the first tensor through a secondary feature extraction layer to obtain a second intermediate tensor, and add the second intermediate tensor to the first tensor to obtain a target tensor; The spatial channel reconstruction module is used to determine the spatial features of the target tensor in the spatial dimension and the channel mixing features of the target tensor in the channel dimension respectively, and the spatial features and the channel mixing features are fused to obtain the target features.
[0042] In the embodiment of the present disclosure, a target tensor is obtained by sequentially performing a feature extraction layer and a secondary feature extraction layer, and then the spatial features of the target tensor in the spatial dimension and the channel mixed features in the channel dimension are determined by a spatial channel reconstruction module. The spatial features and the channel mixed features are fused into target features to realize feature extraction in the spatial dimension and the channel dimension. This reduces redundant calculations in the image deblurring model while avoiding the loss of image features, thereby improving the performance of the image deblurring model and further improving the accuracy and reliability of the obtained clear image.
[0043] In one possible implementation, Figure 5 As shown in the figure, a feature extraction layer includes the first normalization layer (Layer Norm), the third convolutional layer, the first activation function (Simplegate), the attention layer (Simple Channel Attention, SCA) and the fourth convolutional layer. Performing feature extraction on the initial tensor corresponding to the input image feature through a feature extraction layer to obtain a first intermediate tensor may include: The initial tensor of the input image feature is normalized by the first normalization layer to obtain a third intermediate tensor; Convolving the third intermediate tensor through a third convolutional layer to obtain a fourth intermediate tensor; The fourth intermediate tensor is equally divided into two fifth intermediate tensors by the first activation function, and the two fifth intermediate tensors are multiplied to obtain a sixth intermediate tensor; Adjust the channel weight of the sixth intermediate tensor to the preset channel weight through the attention layer to obtain the seventh intermediate tensor; The seventh intermediate tensor is convolved through the fourth convolutional layer to obtain a first intermediate tensor.
[0044] It is worth noting that the first intermediate tensor can be used as the target tensor, and the spatial and channel dimensions of the first intermediate tensor are determined by the spatial channel reconstruction module to obtain the target features. The third convolutional layer consists of a 1*1conv and a 3*3conv, and the fourth convolutional layer consists of a 1*1conv.
[0045] In one possible implementation, Figure 5 As shown, the secondary feature extraction layer includes the second normalization layer, the fifth convolution layer, the second activation function and the sixth convolution layer in sequence; Performing secondary feature extraction on the first tensor through a secondary feature extraction layer to obtain a second intermediate tensor may include: Normalizing the first tensor through the second normalization layer to obtain an eighth intermediate tensor; Convolving the eighth intermediate tensor through the fifth convolutional layer to obtain a ninth intermediate tensor; The ninth intermediate tensor is divided into two tenth intermediate tensors by the second activation function, and the two tenth intermediate tensors are multiplied to obtain the eleventh intermediate tensor; The eleventh intermediate tensor is convolved through the sixth convolutional layer to obtain a second intermediate tensor.
[0046] It is worth noting that both the fifth convolutional layer and the sixth convolutional layer can be composed of a 1*1conv.
[0047] For example, Figure 5 As shown, the complete processing of the feature extraction layer can include the following steps: The initial tensor corresponding to the input image feature is normalized by Layer Norm to obtain the third intermediate tensor, which is then convolved by 1*1conv and 3*3conv to obtain the fourth intermediate tensor; the fourth intermediate tensor is equally divided into two fifth intermediate tensors by Simplegate, and the two fifth intermediate tensors are multiplied to obtain the sixth intermediate tensor; the channel weight of the sixth intermediate tensor is adjusted to the preset channel weight by SCA to obtain the seventh intermediate tensor; the seventh intermediate tensor is convolved by 1*1conv to obtain the first intermediate tensor; the first intermediate tensor and the initial tensor are added to obtain the first tensor A1; The first tensor A1 is normalized by Layer Norm to obtain the eighth intermediate tensor; the eighth intermediate tensor is convolved by 1*1conv to obtain the ninth intermediate tensor; the ninth intermediate tensor is divided into two tenth intermediate tensors by Simplegate, and the two tenth intermediate tensors are multiplied to obtain the eleventh intermediate tensor; the eleventh intermediate tensor is convolved by 1*1conv to obtain the second intermediate tensor; the second intermediate tensor is added to the first tensor A1 to obtain the target tensor A2; The spatial features of the target tensor in the spatial dimension and the channel mixed features of the target tensor in the channel dimension are determined respectively through SCC, and the spatial features and the channel mixed features are fused to obtain the target features.
[0048] In the disclosed embodiment, Simplegte is used to replace the activation function to reduce the amount of computation involved in the feature extraction process; then, SCA is used to cancel the convolution and activation functions of the original two middle layers of Channel Attention and replace them with 1x1 convolution operations to enhance the features of specific channels and suppress unimportant channel features, so that the image deblurring processing model can pay more attention to the channel features that contribute more to the deblurring processing, thereby improving the overall performance of the model and the expression ability of features; finally, SCC is used to extract features in the spatial and channel dimensions, so that the image deblurring processing model can adaptively learn the importance of space and channels without relying on predefined rules, thereby improving the flexibility and adaptability of image deblurring processing.
[0049] Based on the same inventive concept, the present disclosure also provides an image processing device, such as Figure 6 As shown, the image processing device includes an acquisition module 601 and a processing module 602 .
[0050] The acquisition module 601 is configured to acquire a target image through an image acquisition device; The processing module 602 is configured to input the target image into the trained image deblurring model to obtain a clear image output by the image deblurring model, wherein the clear image has a higher clarity than the target image; Among them, the image deblurring model outputs a clear image in the following way: convolution processing is performed on the target image through the first convolution layer to obtain the first image feature, the first image feature is down-sampled through the first feature extraction layer to obtain the second image feature, the second image feature is up-sampled through the second feature extraction layer to obtain the third image feature, and the third image feature is convolution processing is performed on the second convolution layer to obtain the target image feature. The target image feature is used to generate a clear image.
[0051] In the disclosed embodiment, a target image is captured by an image capture device, and the target image is input into a trained image deblurring model. The image deblurring model performs convolution processing on the target image through a first convolutional layer to obtain a first image feature, downsamples the first image feature through a first feature extraction layer to obtain a second image feature, upsamples the second image feature through a second feature extraction layer to obtain a third image feature, and convolutes the third image feature through a second convolutional layer to obtain a target image feature for generating a clear image, and outputs a clear image with a higher clarity than the target image. The image deblurring model combines downsampling and upsampling to achieve multi-dimensional image feature extraction of the target image, avoids the loss of image features during the deblurring process, and improves the reliability and accuracy of the deblurring processing results.
[0052] Furthermore, the first feature extraction layer and the second feature extraction layer are connected via an intermediate layer, the first feature extraction layer includes at least one encoding layer, the second feature extraction layer includes at least one decoding layer, and the number of the encoding layers and the decoding layers is the same; The processing module 602 is configured to scale the target image to a preset size and input the scaled image into a trained image deblurring model; and extracting a first intermediate image feature from the first image feature through at least one encoding layer, and performing a two-fold downsampling process on the first intermediate image feature to obtain a second image feature; The second image feature is subjected to a double upsampling process through at least one decoding layer to obtain a second intermediate image feature, and a third image feature is extracted from the second intermediate image feature.
[0053] Furthermore, the encoding layer includes a feature extraction layer and a downsampling layer in sequence, and the decoding layer includes an upsampling layer and a feature extraction layer in sequence; The processing module 602 is configured to perform feature extraction on the first image feature through a feature extraction layer to obtain a first intermediate image feature; Performing a two-fold downsampling process on the first image feature through a downsampling layer to obtain a second image feature; Performing a double upsampling process on the second image feature through the upsampling layer to obtain a second intermediate image feature; The feature extraction layer performs feature extraction on the second image feature to obtain a third image feature.
[0054] Furthermore, the feature extraction layer includes a primary feature extraction layer, a secondary feature extraction layer, and a spatial channel reconstruction module in sequence. The processing module 602 is configured to perform a feature extraction on the initial tensor corresponding to the input image feature through the primary feature extraction layer to obtain a first intermediate tensor, and add the first intermediate tensor to the initial tensor to obtain a first tensor. Perform secondary feature extraction on the first tensor through a secondary feature extraction layer to obtain a second intermediate tensor, and add the second intermediate tensor to the first tensor to obtain a target tensor; The spatial channel reconstruction module is used to determine the spatial features of the target tensor in the spatial dimension and the channel mixing features of the target tensor in the channel dimension respectively, and the spatial features and the channel mixing features are fused to obtain the target features.
[0055] Furthermore, a feature extraction layer includes the first normalization layer, the third convolution layer, the first activation function, the attention layer, and the fourth convolution layer in sequence; The processing module 602 is configured to perform normalization processing on the initial tensor of the image feature to be input through the first normalization layer to obtain a third intermediate tensor; Convolving the third intermediate tensor through a third convolutional layer to obtain a fourth intermediate tensor; The fourth intermediate tensor is equally divided into two fifth intermediate tensors by the first activation function, and the two fifth intermediate tensors are multiplied to obtain a sixth intermediate tensor; Adjust the channel weight of the sixth intermediate tensor to the preset channel weight through the attention layer to obtain the seventh intermediate tensor; The seventh intermediate tensor is convolved through the fourth convolutional layer to obtain a first intermediate tensor.
[0056] Furthermore, the secondary feature extraction layer sequentially includes a second normalization layer, a fifth convolutional layer, a second activation function, and a sixth convolutional layer; The processing module 602 is configured to perform normalization processing on the first tensor through a second normalization layer to obtain an eighth intermediate tensor; Convolving the eighth intermediate tensor through the fifth convolutional layer to obtain a ninth intermediate tensor; The ninth intermediate tensor is divided into two tenth intermediate tensors by the second activation function, and the two tenth intermediate tensors are multiplied to obtain the eleventh intermediate tensor; The eleventh intermediate tensor is convolved through the sixth convolutional layer to obtain a second intermediate tensor.
[0057] Regarding the image processing apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.
[0058] Based on the same inventive concept, the present disclosure further provides an electronic device, including: a memory having a computer program stored thereon; The processor is used to execute the computer program in the memory to implement the above-mentioned image processing method.
[0059] In the disclosed embodiment, a target image is captured by an image capture device, and the target image is input into a trained image deblurring model. The image deblurring model performs convolution processing on the target image through a first convolutional layer to obtain a first image feature, downsamples the first image feature through a first feature extraction layer to obtain a second image feature, upsamples the second image feature through a second feature extraction layer to obtain a third image feature, and convolutes the third image feature through a second convolutional layer to obtain a target image feature for generating a clear image, and outputs a clear image with a higher clarity than the target image. The image deblurring model combines downsampling and upsampling to achieve multi-dimensional image feature extraction of the target image, avoids the loss of image features during the deblurring process, and improves the reliability and accuracy of the deblurring processing results.
[0060] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 , a memory 702 , and may further include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a communication component 705 .
[0061] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned image processing method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data such as target images, clear images, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and audio components. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module.
[0062] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned image processing method.
[0063] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the above-described image processing method. For example, the computer-readable storage medium may be the aforementioned memory 702 including the program instructions. The program instructions may be executed by the processor 701 of the electronic device 700 to perform the above-described image processing method.
[0064] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a processor, and the computer program implements the steps of the above-mentioned image processing method when executed by the processor.
[0065] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a processor, and the computer program implements the steps of the above-mentioned image processing method when executed by the processor.
[0066] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.
[0067] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0068] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. An image processing method, characterized in that: The method comprises: Acquire a target image by an image acquisition device; Inputting the target image into a trained image deblurring model to obtain a clear image output by the image deblurring model, wherein the clear image has a higher clarity than the target image; The image deblurring model outputs the clear image in the following manner: performing convolution processing on the target image through a first convolution layer to obtain a first image feature, down-sampling the first image feature through a first feature extraction layer to obtain a second image feature, up-sampling the second image feature through a second feature extraction layer to obtain a third image feature, and convolution processing on the third image feature through a second convolution layer to obtain a target image feature, which is used to generate a clear image.
2. The image processing method according to claim 1, wherein: The first feature extraction layer and the second feature extraction layer are connected via an intermediate layer, the first feature extraction layer includes at least one encoding layer, the second feature extraction layer includes at least one decoding layer, and the number of the encoding layers is the same as the number of the decoding layers; Inputting the target image into a trained image deblurring model comprises: Scaling the target image to a preset size and inputting it into a trained image deblurring model; The downsampling of the first image feature by the first feature extraction layer to obtain the second image feature, and upsampling of the second image feature by the second feature extraction layer to obtain the third image feature, includes: extracting a first intermediate image feature from the first image feature through at least one of the encoding layers, and performing a two-fold downsampling process on the first intermediate image feature to obtain a second image feature; The second image feature is subjected to a double upsampling process through at least one of the decoding layers to obtain a second intermediate image feature, and a third image feature is extracted from the second intermediate image feature.
3. The image processing method according to claim 2, wherein: The encoding layer includes a feature extraction layer and a downsampling layer in sequence, and the decoding layer includes an upsampling layer and a feature extraction layer in sequence; Extracting a first intermediate image feature from the first image feature through the encoding layer, and performing a two-fold downsampling process on the first intermediate image feature to obtain a second image feature, including: Extracting the first image feature through the feature extraction layer to obtain a first intermediate image feature; Performing a two-fold downsampling process on the first image feature through the downsampling layer to obtain a second image feature; Performing a double upsampling process on the second image feature through the decoding layer to obtain a second intermediate image feature, and extracting a third image feature from the second intermediate image feature, including: Performing a double upsampling process on the second image feature through the upsampling layer to obtain a second intermediate image feature; The feature extraction layer performs feature extraction on the second image feature to obtain a third image feature.
4. The image processing method according to claim 3, wherein: The feature extraction layer includes a primary feature extraction layer, a secondary feature extraction layer, and a spatial channel reconstruction module in sequence. The feature extraction layer performs feature extraction through the following steps: Performing a feature extraction on an initial tensor corresponding to an input image feature through the primary feature extraction layer to obtain a first intermediate tensor, and adding the first intermediate tensor to the initial tensor to obtain a first tensor; Performing secondary feature extraction on the first tensor by the secondary feature extraction layer to obtain a second intermediate tensor, and adding the second intermediate tensor to the first tensor to obtain a target tensor; The spatial channel reconstruction module is used to determine the spatial features of the target tensor in the spatial dimension and the channel mixing features of the target tensor in the channel dimension, and the spatial features and the channel mixing features are fused to obtain the target features.
5. The image processing method according to claim 4, characterized in that The primary feature extraction layer includes a first normalization layer, a third convolutional layer, a first activation function, an attention layer, and a fourth convolutional layer in sequence; Performing a feature extraction on the initial tensor corresponding to the input image feature through the primary feature extraction layer to obtain a first intermediate tensor, including: Normalizing the initial tensor of the input image features through the first normalization layer to obtain a third intermediate tensor; Performing convolution processing on the third intermediate tensor through the third convolution layer to obtain a fourth intermediate tensor; Splitting the fourth intermediate tensor into two fifth intermediate tensors by using the first activation function, and multiplying the two fifth intermediate tensors to obtain a sixth intermediate tensor; Adjusting the channel weight of the sixth intermediate tensor to a preset channel weight through the attention layer to obtain a seventh intermediate tensor; The seventh intermediate tensor is convolved by the fourth convolutional layer to obtain a first intermediate tensor.
6. The image processing method according to claim 4, wherein: The secondary feature extraction layer sequentially includes a second normalization layer, a fifth convolutional layer, a second activation function and a sixth convolutional layer; The performing secondary feature extraction on the first tensor by the secondary feature extraction layer to obtain a second intermediate tensor includes: Normalizing the first tensor by the second normalization layer to obtain an eighth intermediate tensor; performing convolution processing on the eighth intermediate tensor through the fifth convolutional layer to obtain a ninth intermediate tensor; Splitting the ninth intermediate tensor into two tenth intermediate tensors by the second activation function, and multiplying the two tenth intermediate tensors to obtain an eleventh intermediate tensor; The eleventh intermediate tensor is convolved by the sixth convolutional layer to obtain a second intermediate tensor.
7. An image processing device, characterized in that: include: an acquisition module, configured to acquire a target image through an image acquisition device; a processing module configured to input the target image into a trained image deblurring model to obtain a clear image output by the image deblurring model, wherein the clear image has a higher clarity than the target image; The image deblurring model outputs the clear image in the following manner: performing convolution processing on the target image through a first convolution layer to obtain a first image feature, down-sampling the first image feature through a first feature extraction layer to obtain a second image feature, up-sampling the second image feature through a second feature extraction layer to obtain a third image feature, and convolution processing on the third image feature through a second convolution layer to obtain a target image feature, which is used to generate a clear image.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.