Image blur degree determination method, dataset construction method, and deblurring method

Abstract

The disclosure provides an image blur degree determination method, a blurred image dataset construction method, an image deblurring method, a device, a storage medium and an electronic device, and relates to the technical field of image and video processing. The image blur degree determination method comprises: obtaining pixel values corresponding to a plurality of sub-exposure times of a pixel point within an exposure time of a target image; determining a blur degree value of the pixel point in the target image according to the degree of deviation between the pixel values corresponding to the plurality of sub-exposure times of the pixel point, and determining blur degree data of the target image according to the blur degree value of each pixel point in the target image; wherein the pixel value of the target image is obtained by merging the pixel values corresponding to the plurality of sub-exposure times of the pixel point. The disclosure can represent the blur degree of different pixel points in the image, and is beneficial to improving the image deblurring effect.

Description

Technical Field

[0001] This disclosure relates to the field of image and video processing technology, and in particular to a method for determining the degree of image blur, a method for constructing a blurred image dataset, a method for deblurring an image, an apparatus for determining the degree of image blur, an apparatus for constructing a blurred image dataset, an image deblurring apparatus, a computer-readable storage medium, and an electronic device. Background Technology

[0002] Blurry images are a common occurrence during image capture due to camera shake, defocusing, or other reasons. When deblurring images, it's usually necessary to first calculate the degree of blur in order to apply a matching standard for specific deblurring processing.

[0003] In related technologies, most blur degree calculation methods calculate the overall blur degree of the entire image, which cannot reflect the differences in blur degree in different regions of the image. Consequently, the same standard is used for the entire image during deblurring, resulting in poor deblurring effect. Summary of the Invention

[0004] This disclosure provides a method for determining the degree of image blur, a method for constructing a blurred image dataset, a method for deblurring an image, a device for determining the degree of image blur, a device for constructing a blurred image dataset, a device for deblurring an image, a computer-readable storage medium, and an electronic device, thereby at least partially solving the problems in related technologies such as the inability to determine the degree of blur in different regions of an image and the poor effect of image deblurring.

[0005] According to a first aspect of this disclosure, a method for determining the degree of image blur is provided, comprising: acquiring pixel values ​​corresponding to a plurality of sub-exposure times of a pixel within an exposure time of a target image; determining a blur degree value of the pixel in the target image based on the degree of deviation between the pixel values ​​corresponding to the plurality of sub-exposure times; and determining blur degree data of the target image based on the blur degree value of each pixel in the target image; wherein the pixel values ​​of the target image are obtained by merging the pixel values ​​corresponding to the pixel within the plurality of sub-exposure times.

[0006] According to a second aspect of this disclosure, a method for constructing a blurred image dataset is provided, comprising: acquiring a target image and blur degree data of the target image determined by the image blur degree determination method of the first aspect described above; using the target image as a sample image and the blur degree data of the target image as a first label to construct a blurred image dataset; the blurred image dataset is used to train a blur degree perception network, and the blur degree perception network is used to determine the blur degree data of an image input to the blur degree perception network.

[0007] According to a third aspect of this disclosure, an image deblurring method is provided, comprising: acquiring an image to be processed; determining blurring degree data of the image to be processed, using the image to be processed as a target image, according to the image blurring degree determination method of the first aspect described above; and performing deblurring processing on the image to be processed based on the blurring degree data of the image to be processed to obtain a deblurred image corresponding to the image to be processed.

[0008] According to a fourth aspect of this disclosure, an image deblurring method is provided, comprising: acquiring an image to be processed; processing the image to be processed using a blur level perception network to obtain blur level data of the image to be processed; and performing deblurring processing on the image to be processed based on the blur level data of the image to be processed to obtain a deblurred image corresponding to the image to be processed; wherein the blur level perception network is trained using a blur image dataset constructed using the blur image dataset construction method of the second aspect described above.

[0009] According to a fifth aspect of this disclosure, an image blur degree determination apparatus is provided, comprising: a data acquisition module configured to acquire pixel values ​​corresponding to a plurality of sub-exposure times of a pixel within an exposure time of a target image; and a blur degree data determination module configured to determine a blur degree value of the pixel in the target image based on the degree of deviation between the pixel values ​​corresponding to the plurality of sub-exposure times, and to determine blur degree data of the target image based on the blur degree value of each pixel in the target image; wherein the pixel values ​​of the target image are obtained by merging the pixel values ​​corresponding to the pixel within the plurality of sub-exposure times.

[0010] According to a sixth aspect of this disclosure, an apparatus for constructing a blurred image dataset is provided, comprising: a data acquisition module configured to acquire a target image and blur degree data of the target image determined according to the image blur degree determination method of the first aspect described above; and a dataset construction module configured to construct a blurred image dataset by using the target image as a sample image and the blur degree data of the target image as a first label; wherein the blurred image dataset is used to train a blur degree perception network, and the blur degree perception network is used to determine the blur degree data of an image input to the blur degree perception network.

[0011] According to a seventh aspect of this disclosure, an image deblurring apparatus is provided, comprising: a data acquisition module configured to acquire an image to be processed; a blur degree data determination module configured to determine blur degree data of the image to be processed, using the image to be processed as a target image, according to the image blur degree determination method of the first aspect described above; and a deblurring processing module configured to perform deblurring processing on the image to be processed based on the blur degree data of the image to be processed, to obtain a deblurred image corresponding to the image to be processed.

[0012] According to the eighth aspect of this disclosure, an image deblurring apparatus is provided, comprising: a data acquisition module configured to acquire an image to be processed; a blur level data determination module configured to process the image to be processed using a blur level perception network to obtain blur level data of the image to be processed; and a deblurring processing module configured to perform deblurring processing on the image to be processed based on the blur level data of the image to be processed to obtain a deblurred image corresponding to the image to be processed; wherein the blur level perception network is trained using a blur image dataset constructed using the blur image dataset construction method of the second aspect described above.

[0013] According to a ninth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image blur degree determination method of the first aspect, the blurred image dataset construction method of the second aspect, the image deblurring method of the third aspect, or the image deblurring method of the fourth aspect.

[0014] According to a tenth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the image blur degree determination method of the first aspect, the blurred image dataset construction method of the second aspect, the image deblurring method of the third aspect, or the image deblurring method of the fourth aspect via executing the executable instructions.

[0015] The technical solution disclosed herein has the following beneficial effects: This disclosure provides a technical solution for determining the blur level value of each pixel in an image. On one hand, compared with related technologies that calculate the overall blur level of an image, this solution can more precisely represent the blur level values ​​of different pixels in the image, reflecting the differences in blur level between different pixels. This facilitates targeted deblurring of different regions in the image; for example, different parameters can be used for deblurring different regions of the image, which helps improve the deblurring effect. On the other hand, this solution determines the blur level data based on the pixel value deviation within the exposure time, which is relatively simple to calculate and has a low implementation cost.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0017] Figure 1 A schematic diagram of a system architecture in this exemplary embodiment is shown; Figure 2 This diagram illustrates the structure of an electronic device according to this exemplary embodiment. Figure 3 This diagram illustrates a flowchart of a method for determining the degree of image blur in this exemplary embodiment; Figure 4 This diagram illustrates the sub-exposure time and data readout time in this exemplary embodiment. Figure 5 This illustrates a flowchart of determining a degree of ambiguity value in this exemplary embodiment; Figure 6 This diagram illustrates a flowchart of a method for constructing a blurred image dataset in this exemplary embodiment; Figure 7 This diagram illustrates the structure of a fuzziness perception network in this exemplary embodiment. Figure 8 This diagram illustrates the structure of a deblurring network and a feature-aware network in this exemplary embodiment. Figure 9 This diagram illustrates a flowchart of an image deblurring method according to this exemplary embodiment; Figure 10 A flowchart illustrating another image deblurring method in this exemplary embodiment is shown; Figure 11 This illustration shows a schematic flowchart of an image deblurring method according to this exemplary embodiment; Figure 12 A flowchart of a training network in this exemplary embodiment is shown; Figure 13 This diagram illustrates the structure of an image blur degree determination device according to this exemplary embodiment. Figure 14 This diagram illustrates the structure of a fuzzy image dataset construction apparatus according to this exemplary embodiment. Figure 15 This diagram illustrates the structure of an image deblurring apparatus according to this exemplary embodiment. Figure 16 A schematic diagram of another image deblurring apparatus in this exemplary embodiment is shown. Detailed Implementation

[0018] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0019] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0020] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0021] One approach in related technologies involves calculating the overall blur level of the entire image and then generating a corresponding blur kernel, which is then used to deblur the entire image. However, differences in scene depth and texture sparsity in different regions of the image can lead to variations in the degree of blur in different areas. Using a single blur kernel to deblur different regions cannot achieve optimal deblurring for each region, resulting in numerous defects in the deblurred image. For example, noise may appear in some areas, blur may still exist in others, and even artifacts and ringing may occur.

[0022] In view of the above problems, the exemplary embodiments of this disclosure first provide a method for determining the degree of image blur, a method for constructing a blurred image dataset, and a method for deblurring images. The system architecture of the operating environment of this exemplary embodiment will be described below.

[0023] Figure 1A schematic diagram of the system architecture is shown. System architecture 100 may include a terminal 110 and a server 120. The terminal 110 may be a smartphone, tablet, digital camera, drone, virtual reality / augmented reality glasses, or other terminal devices. The server 120 may be a single server providing image processing services, or a cluster of multiple servers. The terminal 110 and server 120 can be connected via wired or wireless communication links for data interaction. The terminal 110 can capture a target image and acquire relevant data from the capture process. Both the terminal 110 and server 120 can execute the image blur degree determination method, the blurred image dataset construction method, and the image deblurring method described in this exemplary embodiment.

[0024] In one implementation, terminal 110 can execute an image blur degree determination method to determine the blur degree data of the target image, and further execute a blurred image dataset construction method and an image deblurring method based on the blur degree data.

[0025] In one implementation, after the terminal 110 executes the image blur degree determination method, it can send the obtained blur degree data to the server 120, which will then further execute the blur image dataset construction method and the image deblurring method.

[0026] In one implementation, after the server 120 executes the method for constructing a blurred image dataset, it constructs a blur degree perception network using the blurred image dataset and sends the network to the terminal 110 for deployment. The terminal 110 can then execute an image deblurring method.

[0027] In one implementation, terminal 110 can send the target image and related data to server 120, where server 120 executes an image blur degree determination method. Furthermore, server 120 can execute a blurred image dataset construction method and an image deblurring method. Alternatively, it can return the blur degree data of the target image to terminal 110, where terminal executes the blurred image dataset construction method and the image deblurring method.

[0028] The above are just a few examples for illustration. It should be understood that any of the above methods for determining the degree of image blur, constructing a blurred image dataset, and deblurring the image can be executed by the above terminal 110 or server 120.

[0029] An exemplary embodiment of this disclosure also provides an electronic device for performing an image blur degree determination method. This electronic device may be the aforementioned terminal 110 or server 120. Generally, the electronic device may include a processor and a memory. The memory stores executable instructions for the processor and may also store application data, such as images and videos. The processor is configured to implement various programs, such as the image blur degree determination method of this exemplary embodiment, by executing the executable instructions.

[0030] The following is based on Figure 2 Taking the mobile terminal 200 as an example, the structure of the above-mentioned electronic device will be described by way of example. Those skilled in the art should understand that, apart from components specifically designed for mobile purposes, Figure 2 The structure can also be applied to fixed types of equipment.

[0031] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 201, a memory 202, a bus 203, a mobile communication module 204, an antenna 1, a wireless communication module 205, an antenna 2, a display screen 206, a camera module 207, an audio module 208, a power module 209, and a sensor module 210.

[0032] Processor 201 may include one or more processing units, such as: application processor (AP), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit), etc.

[0033] An encoder can encode (i.e., compress) images or videos; for example, it can encode a target image into a bitstream for storage or transmission. A decoder can decode (i.e., decompress) the bitstream data of an image or video to restore the image or video data; for example, it can decode the bitstream data of a target image to restore the target image data. The mobile terminal 200 can support one or more encoders and decoders. Thus, the mobile terminal 200 can handle images or videos in various encoding formats, such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bitmap), and MPEG (Moving Picture Experts Group) 1, MPEG2, H.263, H.264, HEVC (High Efficiency Video Coding), and other video formats.

[0034] The processor 201 can be connected to the memory 202 or other components via the bus 203.

[0035] The memory 202 can be used to store computer executable program code, which includes instructions. The processor 201 executes various functional applications and data processing of the mobile terminal 200 by running the instructions stored in the memory 202. The memory 202 can also store application data, such as images, videos, and other files.

[0036] The communication function of mobile terminal 200 can be implemented through mobile communication module 204, antenna 1, wireless communication module 205, antenna 2, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 204 can provide 2G, 3G, 4G, and 5G mobile communication solutions for use on mobile terminal 200. Wireless communication module 205 can provide wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication for use on mobile terminal 200.

[0037] The display screen 206 is used to implement display functions, such as displaying user interfaces, images, videos, etc.

[0038] The camera module 207 is used to perform shooting functions, such as capturing images and videos. The camera module 207 may include a lens, an image sensor, and related circuitry. The image sensor may be a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device), which generates a digital image by sensing incident light as an electrical signal and then converting it into a digital signal.

[0039] The audio module 208 is used to implement audio functions, such as playing audio and capturing voice.

[0040] The power module 209 is used to implement power management functions, such as charging the battery, powering the device, and monitoring the battery status.

[0041] The sensor module 210 may include a depth sensor 2101, a pressure sensor 2102, a gyroscope sensor 2103, a barometric pressure sensor 2104, etc., to realize corresponding sensing and detection functions.

[0042] The following is combined with Figure 3 The method for determining the degree of image blur in this exemplary embodiment will be described. Figure 3 An exemplary flow of a method for determining the degree of image blur is shown, which may include: Step S310: Obtain the pixel values ​​corresponding to multiple sub-exposure times within the exposure time of the target image. Step S320: Determine the blur level value of the pixel in the target image based on the degree of deviation between the pixel values ​​corresponding to the above multiple sub-exposure times, and determine the blur level data of the target image based on the blur level value of each pixel in the target image.

[0043] The pixel values ​​of the target image are obtained by merging the pixel values ​​of the aforementioned pixels at multiple discrete times.

[0044] The above method provides a technical solution for determining the blur level value of each pixel in an image. On one hand, compared with related techniques that calculate the overall blur level of an image, this solution can more precisely represent the blur level values ​​of different pixels in the image, reflecting the differences in blur level among different pixels. This facilitates targeted deblurring of different regions in the image; for example, different parameters can be used for deblurring different regions of the image, which helps improve the deblurring effect. On the other hand, this solution determines the blur level data based on the pixel value deviation within the exposure time, making the calculation process relatively simple and having a low implementation cost.

[0045] The following sections respectively address... Figure 3 Each step in the process will be explained in detail.

[0046] refer to Figure 3 In step S310, the pixel values ​​corresponding to multiple sub-exposure times of the pixel within the exposure time of the target image are obtained.

[0047] A pixel can be a photosensitive element in an image sensor, which is the basic unit for recording photoelectric signals. Each photosensitive element generates a corresponding pixel in the target image. Therefore, this paper equates the pixel of the image sensor with the pixel of the target image, without making any special distinction in the description.

[0048] In this exemplary embodiment, the complete exposure time of the target image is divided into multiple sub-exposure times, and the photoelectric signal within each sub-exposure time is read and processed to obtain the pixel value corresponding to each pixel point at each sub-exposure time. After the exposure of the target image is completed, each pixel point has the same number of pixel values ​​as the number of sub-exposure times. For example, if the exposure time of the target image includes n sub-exposure times, where n is a positive integer not less than 2, then for each pixel point, n pixel values ​​corresponding to the n sub-exposure times can be obtained.

[0049] In one embodiment, obtaining the pixel values ​​corresponding to multiple sub-exposure times within the exposure time of the target image may include the following steps: Obtain the pixel value of the pixel point read within the data readout time corresponding to each sub-exposure time.

[0050] refer to Figure 4 As shown, the exposure time of the target image is divided into three sub-exposure times: sub-exposure time 1, sub-exposure time 2, and sub-exposure time 3. A short period following each sub-exposure time is the corresponding data readout time: data readout time 1, data readout time 2, and data readout time 3. When capturing the target image, after opening the shutter, the image is first exposed during sub-exposure time 1. During data readout time 1, the photoelectric signals collected by the pixels within sub-exposure time 1 are read and converted into pixel values. Then, the image is exposed during sub-exposure time 2, and the photoelectric signals collected by the pixels within sub-exposure time 2 are read and converted into pixel values. Finally, the image is exposed during sub-exposure time 3, and the photoelectric signals collected by the pixels within sub-exposure time 3 are read and converted into pixel values. For each pixel, three corresponding pixel values ​​are obtained.

[0051] The target image can be viewed as an image obtained by merging pixel values ​​corresponding to multiple sub-exposure times. For example, when shooting long-exposure or high dynamic range images, multiple exposures can be performed, each exposure being a sub-exposure time. The target image is then output by combining the results of these multiple exposures. Specifically, the pixel values ​​corresponding to each pixel at the aforementioned multiple sub-exposure times can be merged to obtain the pixel value of that pixel in the target image. For example, after obtaining the n pixel values ​​corresponding to each pixel at n sub-exposure times, the sum, average, or maximum values ​​of the n pixel values ​​for each pixel can be calculated to obtain the pixel value of each pixel in the target image.

[0052] For example, the number of pixels in an image sensor is W. Starting from a pixel at a certain position, the n pixel values ​​corresponding to the n sub-exposure times are weighted and fused. For example, starting from the top-left pixel coordinate (1,1), the (1,1) pixel values ​​corresponding to the n sub-exposure times are obtained, weighted, and fused to obtain the (1,1) fused pixel value. This disclosure does not limit the weight values ​​used for weighted fusion. For example, equal weights can be set for the n sub-exposure times, then weighted fusion is equivalent to calculating the average pixel value of the n pixel values. Different weights can also be set for the n sub-exposure times according to the actual scene requirements, such as setting corresponding weights according to the duration of the n sub-exposure times, with longer durations having larger weights. Using the pixel value weighted fusion method, the pixels at each position are traversed in a certain order. For example, starting from the (1,1) pixel, the traversal proceeds from left to right and from top to bottom to the (W,H) pixel, and pixel value weighted fusion is performed in the same way as the (1,1) pixel to obtain the fused pixel value of each pixel. Thus, the fused pixel values ​​of all pixels form the target image.

[0053] Typically, image sensors incorporate Bayer filters, which filter light into monochromatic light before it reaches each pixel. Therefore, in one implementation, the pixel value obtained in step S310 can be a single-channel value, such as an R-channel value, G-channel value, or B-channel value. In another implementation, the single-channel value corresponding to each sub-exposure time can be demosaiced to obtain a three-channel value for each pixel. For ease of subsequent calculations, the three-channel values ​​can be converted to grayscale values.

[0054] Continue to refer to Figure 3 In step S320, the blur degree value of the pixel in the target image is determined according to the degree of deviation between the pixel values ​​corresponding to the above multiple sub-exposure times, and the blur degree data of the target image is determined according to the blur degree value of each pixel in the target image.

[0055] The degree of deviation between pixel values ​​refers to the dispersion of pixel value distribution, which can be quantitatively represented by a preset deviation index, such as variance or standard deviation. The n pixel values ​​corresponding to a specific pixel in the target image quantitatively represent the light signal sensed by that pixel at different sub-exposure times during the exposure process. If the subject being photographed is unstable during exposure, including camera shake causing movement of the shooting area, changes in the subject, or changes in ambient lighting, the light signal sensed by the pixel will change, making the pixel's position in the target image prone to blurring. Therefore, the degree of deviation between pixel values ​​corresponding to different sub-exposure times can measure the change in light signal, and thus measure the blurring value of the pixel. Generally, the higher the degree of deviation between the n pixel values ​​corresponding to a pixel, the higher the blurring value of that pixel; the two can satisfy a linear or non-linear positive correlation. For example, the standard deviation between the n pixel values ​​corresponding to each pixel can be used as the blurring value of each pixel, as shown below: (1) Where (x,y) represents the position coordinates of the pixel, D(x,y) represents the blur level of the (x,y) pixel in the target image, and L i (x,y) represents the pixel value of pixel (x,y) at the i-th sub-exposure time. Formula (1) represents the standard deviation of the pixel values ​​of pixel (x,y) at n sub-exposure times, which is used as the blur level of pixel (x,y) in the target image.

[0056] In one implementation, reference Figure 5 As shown, determining the blur level of a pixel in the target image based on the degree of deviation between pixel values ​​corresponding to the multiple sub-exposure times can include the following steps S510 and S520: Step S510: Calculate the deviation index value between the pixel values ​​corresponding to the above multiple sub-exposure times; Step S520: Quantize the deviation index value according to the blur degree value range to obtain the blur degree value of the pixel in the target image.

[0057] The blur level range can be a preset quantization range for blur level values, such as [0, 10], representing a minimum blur level of 0 and a maximum blur level of 10. After calculating the pixel value deviation index, such as variance or standard deviation, the deviation index value can be quantized to the blur level range, for example, through linear or non-linear transformation. For instance, the deviation index value can be normalized to [0, 1], then multiplied by the maximum value of the blur level range (e.g., 10) to obtain the quantized deviation index value, which is the blur level value.

[0058] Therefore, regardless of the numerical format of the target image, it can be quantized to a uniform range of blur levels to obtain a standardized blur level value, which facilitates horizontal comparison between different images, constructs a standardized blur image dataset, and achieves standardized deblurring processing.

[0059] In one implementation, the duration of the multiple sub-exposure times can be the same to eliminate pixel value differences caused by different sub-exposure times and improve the accuracy of blur values.

[0060] After obtaining the blur level value of each pixel in the target image, the blur level values ​​are formed into a set or other specific data format to obtain the blur level data of the target image. The blur level data of the target image includes the blur level value of each pixel.

[0061] In one embodiment, the image blur degree determination method may further include the following steps: Based on the blur level data of the target image, generate a blur level image corresponding to the target image.

[0062] In the blur level image, the pixel value of each pixel represents the degree of blur of that pixel in the target image. In other words, the blur level image is a visual representation of the blur level of the target image, showing the blur level of the target image in a graphical form. The blur level image has the same number of pixels as the target image, making it easy to save and process.

[0063] Exemplary embodiments of this disclosure also provide a method for constructing a blurred image dataset. Figure 6 An exemplary flow of the method for constructing the blurred image dataset is shown, including the following steps S610 and S620: Step S610: Obtain the target image and the blur level data of the target image determined by the above image blur level determination method; Step S620: Use the target image as a sample image and the blur level data of the target image as the first label to construct a blurred image dataset.

[0064] For example, it will be through Figure 3 The target image obtained by the image blur determination method is regarded as a blurred image, which can be used as a sample image. The blur degree data of the target image is used as the corresponding label (Ground Truth). To facilitate the distinction of other types of labels, it is referred to as the first label here. The sample image and the first label are combined to form a training array. By obtaining a large number of training arrays, a blurred image dataset can be constructed.

[0065] Blurry image datasets can be used to train blur-aware networks, which determine the blur level of images input to the network. Blur-aware networks can be end-to-end structures. Figure 7 A schematic diagram of a blur-aware network is shown, which can employ a U-Net structure. For example, after a sample image is input into the blur-aware network, one or more convolutional operations are performed by convolutional layer 1. Figure 7 The diagram shows convolutional layer 1 performing two convolution operations (this disclosure does not limit the specific number of convolution operations in each convolutional layer), followed by pooling to obtain a feature image with a reduced size. Convolutional layer 2 then performs another round of convolution and pooling operations to obtain a feature image with an even smaller size. Convolutional layer 3 then performs another round of convolution and pooling operations to obtain an even smaller feature image. Convolutional layer 4 performs a convolution operation but no pooling operation. Then, the image enters transposed convolutional layer 1, where a transposed convolution operation is performed, followed by concatenation with the feature image from convolutional layer 3, and then one or more convolution operations to obtain a feature image with an increased size. Transposed convolutional layer 2 then performs another round of transposed convolution, concatenation with the feature image from convolutional layer 2, and another convolution operation to obtain a feature image with an even larger size. Finally, transposed convolutional layer 3 performs the above operations once more, outputting blur level data in the form of a blur level image. It should be noted that this disclosure does not limit the number of convolutional layers and transposed convolutional layers in the fuzziness perception network. Depending on the actual needs of the scenario, other types of intermediate layers, such as Dropout layers and fully connected layers, can also be added to the fuzziness perception network.

[0066] The loss function is calculated based on the difference between the sample blur level data output by the blur level perception network (here, the output is the blur level data of the sample image, hence called sample blur level data) and the blur level data used as the first label. For example, a loss function can be established based on the MSE (Mean Square Error) between the sample blur level data and the first label, and the loss function value is obtained by substituting the sample blur level data and the first label. The parameters of the blur level perception network are updated using the loss function value, such as by backpropagation. Through multiple iterations, the network reaches a certain accuracy or the loss function value converges, thus completing the training of the blur level perception network.

[0067] In one implementation, the method for constructing a blurred image dataset may further include the following steps: The clear image corresponding to the target image is used as the second label and added to the blurred image dataset.

[0068] The clear image corresponding to the target image can be a manually captured image or a clear image of the target image after processing. This clear image is used as another type of label corresponding to the sample image (i.e., the target image), and for ease of distinction from the first label mentioned above, it is referred to as the second label. Adding the second label to the blurred image dataset can form a binary training array with the sample images, or a ternary training array with the sample images and the first label.

[0069] In one implementation, the exposure time can be shortened before or after capturing the target image to capture another image as the clear image corresponding to the target image.

[0070] In one implementation, when capturing a target image, the pixel values ​​of each pixel corresponding to a sub-exposure time can be used to form an image. The exposure time of this image is shorter than that of the target image, and it can be used as the clear image corresponding to the target image.

[0071] In one implementation, the target image can be deblurred based on its blur level data to obtain a clear image. For example, different blur kernels can be set for different regions of the target image based on its blur level data, and different regions of the target image can be deblurred according to the blur kernels to obtain a clear image.

[0072] The blurred image dataset containing the second label can also be used to train a deblurring network. The deblurring network is used to deblur the image input to the deblurring network and output the corresponding deblurred image.

[0073] In one implementation, the deblurring network can be an end-to-end network, such as a U-Net network. For example, the deblurring network can also employ... Figure 7 The network structure shown, in which the specific settings of convolutional layers, transposed convolutional layers, or other types of intermediate layers can be compared with... Figure 7 There are some differences. Based on the difference between the deblurred image output by the deblurring network and the second label, the parameters of the deblurring network are updated, thereby achieving training of the deblurring network. It should be understood that in this exemplary embodiment, both the blur perception network and the deblurring network can adopt a U-Net structure, but they can differ in their detailed structural details, and the parameters of the two networks will differ after training, resulting in different functions implemented by the two networks.

[0074] In one implementation, the blur level data can be integrated into the deblurring network. Generally, the blur level data can be processed and then input into the intermediate layer of the deblurring network for fusion with the image information. Figure 8 A schematic diagram of the deblurring network and the feature-aware network is shown. The feature-aware network processes the blurred data, and its output is connected to the intermediate layers of the deblurring network. For example, a sample image is input into the deblurring network, undergoes convolution and pooling operations in convolutional layers 1 through 3, resulting in a feature image with progressively smaller dimensions, which then enters convolutional layer 4. The sample image is then input into the blur-aware network (…). Figure 8 The specific structure of the blur perception network is omitted here. The input sample image corresponds to the sample blur data (usually in the form of a blur image), which is then input into the feature perception network and first passes through convolutional layer 1'. Figure 8 The diagram shows that convolutional layer 1' includes four convolution operations (this disclosure does not specifically limit the number of operations), which yields a multi-channel feature ψ representing the degree of fuzziness; then ψ is input into multiple different convolutional layers, such as... Figure 8 The convolutional layers 2' and 3' shown are... Figure 8 The diagram shows that both convolutional layers 2' and 3' include two convolution operations, which are not specifically limited in this disclosure. The feature image output from convolutional layer 2' is input into fully connected layer 1 to obtain sample modulation parameters α, and the feature image output from convolutional layer 3' is input into fully connected layer 2 to obtain sample modulation parameters β (here, α and β are the modulation parameters corresponding to the sample images, hence the name sample modulation parameters). Modulation refers to fusing blur information with image information. In one embodiment, modulation can be an affine transformation of the image, where α and β can be parameters of different types of transformation operations. For example, affine transformations typically include operations such as rotation, translation, and scaling; for example, α can be a scaling parameter, and β can be a translation parameter. Fully connected layers 1 and 2 can be connected to any intermediate layer in the deblurring network, for example, in... Figure 8The connection to convolutional layer 4 indicates that an affine transformation is performed on the feature image in convolutional layer 4, as shown below: (2) Here, F is the feature image, and the dimensions of α and β are the same as those of F. It is a feature image that has undergone affine transformation. This indicates element-wise multiplication. The feature image after affine transformation is then processed by transposed convolutional layers 1 to 3 to output a sample deblurred image (the deblurred image here is the deblurred image corresponding to the sample image, hence it is called the sample deblurred image).

[0075] Alternatively, the first label can be used to replace the sample fuzziness data output by the fuzziness perception network and input into the feature perception network to obtain the sample modulation parameters.

[0076] exist Figure 8 In this study, a feature-aware network is used to represent the blur level data as modulation parameters and modulate the feature image of the sample image. This achieves the fusion of image and blur level information, which is beneficial to improving the deblurring quality of the deblurring network.

[0077] It should be understood that Figure 8 The feature-aware network in this example uses an SFT (Spatial Feature Transformation) layer structure, which is only illustrative. In other implementations, other feature-aware network structures can be used, such as a U-Net structure. After the blurred image is input into the feature-aware network, it is processed through multiple convolutional layers and transposed convolutional layers. One or more of these convolutional or transposed convolutional layers can be connected to corresponding convolutional or transposed convolutional layers in the deblurring network. This allows the feature image from the feature-aware network to be input into the deblurring network and stitched together with the feature image in the deblurring network, thus achieving the fusion of image and blur information.

[0078] In the aforementioned structure of the deblurring network and feature perception network, a three-element training array such as sample image-first label-second label can be used for training. The sample image and the first label are respectively input into two channels of the deblurring network. After processing, the deblurred sample image is output. The parameters of the deblurring network are updated according to the difference between the deblurred sample image and the second label (such as the loss function value of MSE), or the parameters of the deblurring network and the feature perception network are updated simultaneously to achieve training.

[0079] Existing public blurred image datasets (such as the GoPro dataset) typically calculate the average of multiple consecutive clear images to obtain blurred images. Using the image blur degree determination method in this exemplary embodiment, blur degree data can be determined for blurred images in public blurred image datasets, thereby expanding these datasets and constructing a more comprehensive blurred image dataset. This facilitates its application in training the aforementioned blur degree perception network and deblurring network.

[0080] Exemplary embodiments of this disclosure also provide an image deblurring method. Figure 9 An exemplary flow of the image deblurring method is shown, including the following steps S910 to S930: Step S910: Obtain the image to be processed.

[0081] The image to be processed is the image that needs to be deblurred; it is an image captured by multiple sub-exposure times.

[0082] Step S920: Based on the above image blur degree determination method, using the image to be processed as the target image, determine the blur degree data of the image to be processed.

[0083] For example, using the image to be processed as the target image, perform... Figure 3 The method for determining the blur level of an image obtains blur level data of the image to be processed, including the blur level value of each pixel in the image to be processed.

[0084] Step S930: Based on the blur level data of the image to be processed, the image to be processed is deblurred to obtain the deblurred image corresponding to the image to be processed.

[0085] The blur level data of the image to be processed can provide auxiliary or reference information for the deblurring process. For example, based on the blur level data of the target image, different blur kernels can be set for different regions of the target image, and the different regions of the target image can be deblurred according to the blur kernels to obtain the model image. Compared with using a single blur kernel, this exemplary embodiment can perform targeted deblurring processing on four different regions, thereby improving the deblurring effect.

[0086] In one implementation, step S930 may include the following steps: The feature perception network is used to process the blur level data of the image to be processed to obtain the modulation parameters; The image to be processed is deblurred according to the modulation parameters to obtain the deblurred image corresponding to the image to be processed.

[0087] The feature-aware network is used to convert the blurriness data into a specific form of modulation parameters, which can then be fused with the image information of the image to be processed. The feature-aware network and modulation parameters can be found in [reference needed]. Figure 8 As shown, it can also be in other forms, such as a feature-aware network with a U-Net structure, and the modulation parameters being feature images of fuzziness level data. This disclosure does not limit it in this way.

[0088] Based on the modulation parameters, blur information can be fused with image information, making the information dimensions of the image to be processed more comprehensive and facilitating high-quality image deblurring.

[0089] In one implementation, the deblurring network described above can be used for deblurring. For example, the process of deblurring the image to be processed according to modulation parameters to obtain a deblurred image may include the following steps: The image to be processed is input into the input layer of the deblurring network, the modulation parameters are input into the intermediate layer of the deblurring network, and the deblurred image corresponding to the image to be processed is output by the deblurring network.

[0090] The specific processing steps of the deblurring network can be referred to the above. Figure 8 Part of the content. Figure 8 The first part describes the process by which the deblurring network processes sample images and their degree of blurriness, i.e., the training process. In the deblurring of the image to be processed, the deblurring network follows the same process, the difference being that the network has been fully trained and can directly output a high-quality deblurred image.

[0091] Exemplary embodiments of this disclosure also provide another method for image deblurring. Figure 10 An exemplary flow of the image deblurring method is shown, including the following steps S1010 to S1030: Step S1010: Obtain the image to be processed.

[0092] The image to be processed is the image that needs to be deblurred. It can be an image obtained through any means, such as the currently captured image or an image selected by the user. Compared with step S910, step S1010 does not require that the image to be processed is an image obtained by capturing multiple sub-exposure times. Specifically, it does not require obtaining the pixel values ​​corresponding to multiple sub-exposure times when capturing the image to be processed. For example, it can be an image obtained by a single exposure and data reading.

[0093] Step S1020: Use a blur degree perception network to process the image to be processed to obtain blur degree data of the image to be processed.

[0094] The blurriness perception network is trained using the blurry image dataset constructed using the aforementioned blurry image dataset construction method. For example, it could be... Figure 7 The network shown. For example, via... Figure 6 A method for constructing a blurred image dataset is described, which is then used to train a blur perception network. In step S1020, the image to be processed is input into the trained blur perception network, which outputs the blur level data of the image to be processed. The blur level data of the image to be processed can be a blur level image corresponding to the image to be processed.

[0095] It should be noted that the blur level perception network can learn the feature association between a target image and its blur level data, and this feature association is applicable to blurred images in all scenes. After training, the blur level perception network can apply the learned feature association to blurred images in any scene, thus processing blurred images in any scenario. For this reason, this exemplary embodiment does not specifically limit the image to be processed, especially the method of capturing the image; the image to be processed can even be an image obtained through non-capture methods, such as a synthetic image.

[0096] Step S1030: Based on the blur level data of the image to be processed, the image to be processed is deblurred to obtain the deblurred image corresponding to the image to be processed.

[0097] The blur level data of the image to be processed can provide auxiliary or reference information for the deblurring process. For example, based on the blur level data of the image to be processed, different blur kernels can be set for different regions in the image, and then the blur kernels can be used to deblur different regions to obtain a deblurred image. Compared with using a single blur kernel, targeted deblurring can be performed on different regions, thus improving the deblurring effect.

[0098] In one implementation, step S1030 may include the following steps: The feature perception network is used to process the blur level data of the image to be processed to obtain the modulation parameters; The image to be processed is deblurred according to the modulation parameters to obtain the deblurred image corresponding to the image to be processed.

[0099] The feature-aware network is used to convert the blurriness data into a specific form of modulation parameters, which can then be fused with the image information of the image to be processed. The feature-aware network and modulation parameters can be found in [reference needed]. Figure 8 As shown, it can also be in other forms, such as a feature-aware network with a U-Net structure, and the modulation parameters being feature images of fuzziness level data. This disclosure does not limit it in this way.

[0100] Based on the modulation parameters, blur information can be fused with image information, making the information dimensions of the image to be processed more comprehensive and facilitating high-quality image deblurring.

[0101] In one implementation, the deblurring network described above can be used for deblurring. For example, the process of deblurring the image to be processed according to modulation parameters to obtain a deblurred image may include the following steps: The image to be processed is input into the input layer of the deblurring network, the modulation parameters are input into the intermediate layer of the deblurring network, and the deblurred image corresponding to the image to be processed is output by the deblurring network.

[0102] The specific processing steps of the deblurring network can be referred to the above. Figure 8 Part of the content. Figure 8 The first part describes the process by which the deblurring network processes sample images and their degree of blurriness, i.e., the training process. In the deblurring of the image to be processed, the deblurring network follows the same process, the difference being that the network has been fully trained and can directly output a high-quality deblurred image.

[0103] Figure 11 This diagram illustrates a process for deblurring an image using a blur level perception network, a feature perception network, and a deblurring network. The image to be processed is first input into the blur level perception network, which outputs a blur level image corresponding to the image being processed. Next, the blur level image is input into the feature perception network, which outputs modulation parameters. Finally, the image to be processed is input into the deblurring network, and the modulation parameters are input into the intermediate layer of the deblurring network. The deblurring network then outputs a deblurred image corresponding to the image being processed. Figure 11 The process incorporates image information from the image itself and pixel-level blur information, thereby enabling targeted deblurring of different regions of the image to obtain high-quality deblurred images.

[0104] In the aforementioned network training process, a blur perception network is trained using sample images and their first labels, and a deblurring network is trained using sample images, their first labels, and their second labels; alternatively, a deblurring network and a feature perception network are trained separately. Based on... Figure 11 As can be seen, the three networks have input-output connections, allowing for the joint training of the fuzziness perception network, the defuzzification network, and the feature perception network. The complete training process is illustrated below. (Reference) Figure 12 As shown, the image deblurring method may further include the following steps S1210 to S1240: Step S1210: Obtain a blurred image dataset, including sample images, a first label, and a second label; the sample images are target images, the first label is the blur level data of the target images, and the second label is the clear image corresponding to the target images; Step S1220: Pre-train the blur perception network using sample images and the first label; Step S1230: Pre-train the deblurring network using sample images, the first label, and the second label, or pre-train the deblurring network and the feature perception network. Step S1240: Input the sample images into the blur perception network and the deblurring network respectively. Input the sample blur data output by the blur perception network into the feature perception network. Input the sample modulation parameters output by the feature perception network into the intermediate layer of the deblurring network. Based on the difference between the sample deblurred image output by the deblurring network and the second label, fine-tune the parameters of the blur perception network, the deblurring network and the feature perception network.

[0105] The entire training process is divided into two stages: pre-training and fine-tuning. In pre-training, the fuzziness perception network is trained separately, the deblurring network is trained separately, or the deblurring network and the feature perception network are trained as a whole and then separately. For example, refer to... Figure 8 As shown, the blur perception network can be pre-trained based on the difference between the blur level data output by the blur perception network and the first label. Similarly, the deblurring network and the feature perception network can be pre-trained based on the difference between the deblurred image output by the deblurring network and the second label. Each pre-trained network structure is relatively small, facilitating rapid parameter adjustments to obtain preliminary training results. In the parameter fine-tuning stage, the three networks are treated as a single large network structure. Sample images are input to the blur perception network and the deblurring network. The blur perception network outputs sample blur level data, which is then processed by the feature perception network to output sample modulation parameters, which are input to the intermediate layer of the deblurring network. Finally, the deblurring network outputs the deblurred image, serving as the overall network output. Based on the difference between this image and the second label (e.g., the MSE loss function value), the parameters of all three networks are updated together, achieving comprehensive global training. Further fine-tuning of the parameters based on pre-training results in a more accurate network, which is beneficial for further improving image deblurring quality.

[0106] Exemplary embodiments of this disclosure also provide an image blur degree determination apparatus. (See reference...) Figure 13 As shown, the image blur level determination device 1300 may include: The data acquisition module 1310 is configured to acquire the pixel values ​​corresponding to multiple sub-exposure times within the exposure time of the target image. The blur level data determination module 1320 is configured to determine the blur level value of a pixel in the target image based on the degree of deviation between the pixel values ​​corresponding to the plurality of sub-exposure times, and to determine the blur level data of the target image based on the blur level value of each pixel in the target image. The pixel values ​​of the target image are obtained by merging the pixel values ​​corresponding to the pixels at the aforementioned multiple sub-exposure times.

[0107] In one implementation, the data acquisition module 1310 is configured to: Obtain the pixel value of the pixel point read within the data readout time corresponding to each sub-exposure time.

[0108] In one implementation, the ambiguity level data determination module 1320 is configured to: Calculate the deviation index value between pixel values ​​corresponding to the above multiple sub-exposure times; The deviation index value is quantified based on the range of blur degree values ​​to obtain the blur degree value of the pixel in the target image.

[0109] In one implementation, the ambiguity level data determination module 1320 is further configured to: Based on the blur level data of the target image, a blur level image corresponding to the target image is generated. The pixel value of each pixel in the blur level image is the blur level value of each pixel in the target image.

[0110] Exemplary embodiments of this disclosure also provide an apparatus for constructing a blurred image dataset. (See reference...) Figure 14 As shown, the blurred image dataset construction apparatus 1400 may include: The data acquisition module 1410 is configured to acquire the target image and the blur level data of the target image determined according to the above-described image blur level determination method; The dataset construction module 1420 is configured to use the target image as the sample image and the blur level data of the target image as the first label to construct a blurred image dataset. The blurred image dataset is used to train the blur level perception network, which is used to determine the blur level data of the image input to the blur level perception network.

[0111] In one implementation, the dataset construction module 1420 is further configured to: The clear image corresponding to the target image is used as the second label and added to the blurred image dataset; the blurred image dataset is also used to train the deblurring network, which is used to deblur the images input to the deblurring network.

[0112] In one embodiment, the blurred image dataset construction apparatus 1400 may further include a deblurring processing module configured to: Based on the blur level data of the target image, the target image is deblurred to obtain the corresponding clear image.

[0113] Exemplary embodiments of this disclosure also provide an image deblurring determination apparatus. (See reference...) Figure 15 As shown, the image deblurring device 1500 may include: The data acquisition module 1510 is configured to acquire the image to be processed; The blur degree data determination module 1520 is configured to determine the blur degree data of the image to be processed, using the image to be processed as the target image, according to the above image blur degree determination method. The deblurring module 1530 is configured to perform deblurring on the image to be processed based on the blur level data of the image to be processed, and obtain the deblurred image corresponding to the image to be processed.

[0114] Exemplary embodiments of this disclosure also provide another image deblurring determination apparatus. (Reference) Figure 16 As shown, the image deblurring device 1600 may include: The data acquisition module 1610 is configured to acquire the image to be processed; The blur level data determination module 1620 is configured to process the image to be processed using a blur level perception network to obtain blur level data of the image to be processed. The deblurring module 1630 is configured to perform deblurring on the image to be processed based on the blur level data of the image to be processed, and obtain the deblurred image corresponding to the image to be processed. The blur perception network is trained using the blur image dataset constructed using the aforementioned blur image dataset construction method.

[0115] In one implementation, the deblurring module 1630 is configured to: The feature perception network is used to process the blur level data of the image to be processed to obtain the modulation parameters; The image to be processed is deblurred according to the modulation parameters to obtain the deblurred image corresponding to the image to be processed.

[0116] In one implementation, the deblurring module 1630 is configured to: The image to be processed is input into the input layer of the deblurring network, the modulation parameters are input into the intermediate layer of the deblurring network, and the deblurred image is output through the deblurring network.

[0117] In one embodiment, the image deblurring device 1600 may further include a network training module configured to: Obtain a blurred image dataset, which includes sample images, a first label, and a second label. The sample images are the target images, the first label is the blur level data of the target images, and the second label is the clear image corresponding to the target images. The blur perception network was pre-trained using sample images and the first label; The deblurring network can be pre-trained using sample images, first labels, and second labels, or the deblurring network and feature perception network can be pre-trained together. The sample images are input into the blur perception network and the deblurring network respectively. The blur perception network outputs the sample blur data into the feature perception network, and the feature perception network outputs the sample modulation parameters into the intermediate layer of the deblurring network. Based on the difference between the deblurred sample image output by the deblurring network and the second label, the parameters of the blur perception network, the deblurring network and the feature perception network are fine-tuned.

[0118] The details of each part of the above-described apparatus have been described in detail in the method section of the embodiments, and therefore will not be repeated here.

[0119] Exemplary embodiments of this disclosure also provide a computer-readable storage medium that can be implemented as a program product including program code, which, when run on an electronic device, causes the electronic device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. In one embodiment, the program product can be implemented as a portable compact disc read-only memory (CD-ROM) and include program code, and can run on an electronic device, such as a personal computer. However, the program product of this disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0120] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0121] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0122] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0123] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0124] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0125] Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be embodied in entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuit,” “module,” or “system.” Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0126] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is defined only by the appended claims.

Claims

1. A method for constructing a fuzzy image dataset, characterized in that, include: Acquire the target image and the blur level data of the target image; Using the target image as a sample image and the blur level data of the target image as a first label, a blurred image dataset is constructed; the blurred image dataset is used to train a blur level perception network, and the blur level perception network is used to determine the blur level data of the images input to the blur level perception network; The blur level data of the target image is determined in the following way: Obtain the pixel value corresponding to multiple sub-exposure times within the exposure time of the target image; Based on the degree of deviation between the pixel values ​​corresponding to the plurality of sub-exposure times, the blur degree value of the pixel in the target image is determined, and the blur degree data of the target image is determined based on the blur degree value of each pixel in the target image; wherein, the pixel value of the target image is obtained by merging the pixel values ​​corresponding to the pixel in the plurality of sub-exposure times.

2. The method according to claim 1, characterized in that, The step of obtaining the pixel values ​​corresponding to multiple sub-exposure times within the exposure time of the target image includes: Obtain the pixel value of the pixel point read within the data readout time corresponding to each of the sub-exposure times.

3. The method according to claim 1, characterized in that, Determining the blur level value of a pixel in the target image based on the degree of deviation between the pixel values ​​corresponding to the plurality of sub-exposure times includes: Calculate the deviation index value of the pixel among the pixel values ​​corresponding to the plurality of sub-exposure times; The deviation index value is quantified according to the blur degree value range to obtain the blur degree value of the pixel in the target image.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Based on the blur level data of the target image, a blur level image corresponding to the target image is generated, wherein the pixel value of each pixel in the blur level image is the blur level value of each pixel in the target image.

5. The method according to claim 1, characterized in that, The method further includes: The clear image corresponding to the target image is used as the second label and added to the blurred image dataset; the blurred image dataset is also used to train a deblurring network, which is used to deblur the images input to the deblurring network.

6. The method according to claim 5, characterized in that, The method further includes: Based on the blur level data of the target image, the target image is deblurred to obtain a clear image corresponding to the target image.

7. An image deblurring method, characterized in that, include: Obtain the image to be processed; The image to be processed is processed using a blur degree perception network to obtain blur degree data of the image to be processed; Based on the blur level data of the image to be processed, the image to be processed is deblurred to obtain the deblurred image corresponding to the image to be processed. The blur perception network is trained using a blur image dataset constructed using the blur image dataset construction method described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that, The process of deblurring the image based on its blur level data to obtain a deblurred image includes: The blur level data of the image to be processed is processed using a feature-aware network to obtain modulation parameters; The image to be processed is deblurred according to the modulation parameters to obtain the deblurred image corresponding to the image to be processed.

9. The method according to claim 8, characterized in that, The step of deblurring the image to be processed according to the modulation parameters to obtain the deblurred image corresponding to the image to be processed includes: The image to be processed is input into the input layer of the deblurring network, the modulation parameters are input into the intermediate layer of the deblurring network, and the deblurred image is output through the deblurring network.

10. The method according to claim 9, characterized in that, The method further includes: Obtain a blurred image dataset, which includes sample images, a first label, and a second label. The sample images are target images, the first label is the blur level data of the target image, and the second label is the clear image corresponding to the target image. The blur perception network is pre-trained using the sample images and the first label; The deblurring network can be pre-trained using the sample image, the first label, and the second label, or the deblurring network and the feature perception network can be pre-trained. The sample images are input into the blur perception network and the deblurring network respectively. The sample blur data output by the blur perception network is input into the feature perception network. The sample modulation parameters output by the feature perception network are input into the intermediate layer of the deblurring network. Based on the difference between the deblurred sample image output by the deblurring network and the second label, the parameters of the blur perception network, the deblurring network and the feature perception network are fine-tuned.

11. A device for constructing a fuzzy image dataset, characterized in that, include: The data acquisition module is configured to acquire a target image and blur data of the target image; The dataset construction module is configured to construct a blurred image dataset by using the target image as a sample image and the blur level data of the target image as a first label; the blurred image dataset is used to train a blur level perception network, and the blur level perception network is used to determine the blur level data of the image input to the blur level perception network; The blur level data of the target image is determined in the following way: Obtain the pixel value corresponding to multiple sub-exposure times within the exposure time of the target image; Based on the degree of deviation between the pixel values ​​corresponding to the plurality of sub-exposure times, the blur degree value of the pixel in the target image is determined, and the blur degree data of the target image is determined based on the blur degree value of each pixel in the target image; wherein, the pixel value of the target image is obtained by merging the pixel values ​​corresponding to the pixel in the plurality of sub-exposure times.

12. An image deblurring device, characterized in that, include: The data acquisition module is configured to acquire the image to be processed; The blur level data determination module is configured to process the image to be processed using a blur level perception network to obtain blur level data of the image to be processed. The deblurring module is configured to perform deblurring on the image to be processed based on the blur level data of the image to be processed, so as to obtain a deblurred image corresponding to the image to be processed. The blur perception network is trained using a blur image dataset constructed using the blur image dataset construction method described in any one of claims 1 to 6.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 10.

14. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 10 by executing the executable instructions.

Images