A multi-level patch-based image deblurring method and system

Abstract

The application provides a multi-level piece image deblurring method and system, and relates to the field of image processing.The method comprises the following steps: cutting any blurred image in a blurred image training set in a space pyramid manner to J-level image pieces; inputting the image pieces to a J-level decoder at the Jth level to obtain corresponding convolution features; performing spatial aggregation on the convolution features of adjacent image pieces at the Jth level, so that the spatial size of the aggregated image pieces is the same as the spatial size of the image pieces at the J-1th level, and obtaining the output of the J-level decoding-decoding network; performing matrix addition on the image pieces and the output of the J-level decoding-decoding network at the J-1th level to obtain corresponding convolution features, and finally obtaining the output of the decoding-decoding network at the 1th level as a deblurring output result.The application effectively improves the deblurring performance, reduces the model parameter size, and accelerates the model operation efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a multi-level fragmented image deblurring method and system. BACKGROUND

[0002] The research goal of the non-uniform image deblurring method is to remove the image blur caused by camera motion or scene dynamics in the target image. Before the introduction of the deep learning model, the traditional deblurring method uses various constraints or regularizations to estimate the motion blur filter, but this process is a non-convex nonlinear optimization problem with huge cost. In addition, the uniformity assumption of the blur kernel space adopted by such methods is too loose, which leads to the inability to effectively solve the complex blur in the real environment.

[0003] The image deblurring method based on the deep convolutional neural network is to realize the deblurring process by learning how to reconstruct a clear image end to end. In order to more effectively process the blur information at different scales, a "coarse-grained-fine-grained" multi-scale mechanism is proposed for the deep image deblurring problem. Compared with the early deep model, the performance of such method has been significantly improved.

[0004] In recent years, although the end-to-end deep learning method has shown excellent performance in removing non-uniform motion blur, the early model is mainly based on the "coarse-grained-fine-grained" multi-scale mechanism to carry out model design. Such deep deblurring model also has technical bottlenecks in efficiency, performance, etc., such as: ① under the multi-scale mechanism, since a larger filter is used, the parameter size of most network models is also larger, which further leads to high running time; ② under the multi-scale architecture, linearly increasing the number of scale levels cannot continuously improve the performance of the deblurring network, that is, there is a performance bottleneck. SUMMARY

[0005] To solve the above technical problems, the present application provides a multi-level fragmented image deblurring technical solution.

[0006] The first aspect of the present application discloses a multi-level fragmented image deblurring method; the method comprises:

[0007] Any blurred image B in the blurred image training set is cut in a spatial pyramid structure manner i (i = 1, 2 …… N), to obtain J level image fragments Wherein, i represents the i-th blurred image in the blurred image training set, N represents the total number of samples in the blurred image training set, J represents the total number of levels of the spatial pyramid structure, J≥3, (j, k) represents the k-th image fragment of the j-th level;

[0008] At the Jth level, the image fragments are input the Jth level encoder e J (·), to obtain the corresponding convolutional feature k∈{1…2 J-1}, to spatially aggregate the adjacent image patch convolutional features of the Jth level, to obtain k∈{1…2 J-2}, such that the spatial size of is the same as the spatial size of the image patch convolutional feature of the J-1th level; and input the Jth level decoder d J (·), to obtain the Jth level encoder-decoder network output k∈{1…2 J-2};

[0009] At the J-1th level, the image patch k∈{1…2 J-2} is matrix-added with the Jth level encoder-decoder network output k∈{1…2 J-2}, and the result of the matrix addition is input into the J-1th level encoder e J-1 (·), to obtain the corresponding convolutional feature k∈{1…2 J-2}, and the corresponding convolutional feature of the J-1th level encoder is added with the Jth level image patch convolutional feature k∈{1…2 J-2}, to obtain the corresponding convolutional feature;

[0010] The adjacent image patch convolutional features of the J-1th level are spatially aggregated, to obtain k∈{1…2 J-3}, such that k∈{1…2 J-3} is the same as the spatial size of the image patch convolutional feature of the J-2th level; and input the J-1th level decoder d J-1 (·), to obtain the J-1th level encoder-decoder network output k∈{1…2 J-3};

[0011] When J≥5, the manner of obtaining k∈{1…2 J-3} and k∈{1…2 J-3} at the J-1th level is respectively used to obtain the spatially aggregated image patch convolutional features k∈{1…2 J-4}、…、 and the coding-decoding network output at the J-2, …, 2th level k∈{1…2 J -4}、…、

[0012] When J=4, the image patch convolutional features after spatial aggregation are obtained at the 2th level in the manner of referring to the J-1th level k∈{1…2 J-3} and k∈{1…2 J-3} are obtained at the 2th level The coding-decoding network output at the 2th level

[0013] The image patch convolutional features after spatial aggregation are obtained at the 1th level in the manner of referring to the J-1th level k∈{1…2 J-2} are obtained at the 1th level The image patch convolutional features after spatial aggregation are obtained at the 1th level in the manner of referring to the J-1th level The coding-decoding network output at the 1th level The image patch convolutional features after spatial aggregation are obtained at the 1th level in the manner of referring to the J-1th level as any blurred image B i in the blurred image training set.

[0014] According to the method of the first aspect of the present application, the loss function of the blurred image training model is constructed by the root mean square error function: The parameter gradient of the blurred image training model is generated by the back propagation algorithm, so that the parameters of the blurred image training model are updated in the direction of minimizing the loss function, wherein G i is the corresponding clear image. i

[0015] According to the method of the first aspect of the present application, the multi-level patch image deblurring model training is completed until the loss function value reaches the minimum value.

[0016] According to the method of the first aspect of the present application, for any blurred image B i in the blurred image training set, by obtaining the corresponding The deblurred reconstructed clear sample is obtained.

[0017] According to the method of the first aspect of the present application, the total number of levels of the spatial pyramid structure is J=4.

[0018] The second aspect of the present application discloses a multi-level patch image deblurring system; the system comprises:​

[0019] a first processing module configured to, for any blurred image B in the blurred image training set i (i = 1, 2 … N) are cut in a spatial pyramid structure to obtain J levels of image patches wherein i represents the i-th blurred image in the blurred image training set, N represents the total number of samples in the blurred image training set, J represents the total number of levels of the spatial pyramid structure, J≥3, (j, k) represents the k-th image patch in the j-th level;

[0020] a second processing module configured to, at the J-th level, add the image patch input the J-th level encoder e J (·) to obtain the corresponding convolutional feature k∈{1…2 J-1}, the spatially aggregated convolutional features of adjacent image patches in the J-th level are obtained k∈{1…2 J-2}, so that the spatial size of is the same as that of the convolutional feature of the image patch in the J-1-th level; and input the J-th level decoder d J (·) to obtain the J-th level encoder-decoder network output k∈{1…2 J-2};

[0021] a third processing module configured to, at the J-1-th level, add the image patch k∈{1…2 J-2} and the J-th level encoder-decoder network output k∈{1…2 J-2}, and input the result of the matrix addition to the J-1-th level encoder e J-1 (·) to obtain the corresponding convolutional feature k∈{1…2 J-2}, the corresponding convolutional feature of the J-1-th level encoder is added to the J-th level image patch k∈{1…2 J-2} convolutional feature to obtain the corresponding convolutional feature;

[0022] a fourth processing module configured to, at the J-1-th level, spatially aggregate the convolutional features of adjacent image patches to obtain k∈{1…2 J-3}, so that k∈{1…2 J-3the spatial size of the image patch convolutional feature of the J-2th level is the same as the spatial size of the image patch convolutional feature of the J-1th level; and inputting the J-1th level decoder d J-1 (·) to obtain the J-1th level encoder-decoder network output k∈{1…2 J-3};

[0023] the fifth processing module is configured to, when J≥5, refer to the J-1th level to obtain k∈{1…2 J-3} and k∈{1…2 J-3} in a manner of spatial aggregation of the image patch convolutional feature k∈{1…2 J-4} of the J-2th level, …, the 2th level, and the J-2th level, …, the 2th level encoder-decoder network output k∈{1…2 J-4 {, …, when J=4, refer to the J-1th level to obtain k∈{1…2 J-3} and k∈{1…2 J-3} in a manner of spatial aggregation of the image patch convolutional feature the 2th level encoder-decoder network output

[0024] the sixth processing module is configured to, refer to the J-1th level to obtain k∈{1…2 J-2} in a manner of the 1th level decoder e1(·) to obtain inputting the 1st level decoder d1(·) to obtain the 1th level encoder-decoder network output inputting the 1st level decoder d1(·) to obtain the 1th level encoder-decoder network output inputting the 1st level decoder d1(·) to obtain the 1th level encoder-decoder network output as any blurred image B i (i=1,2……N) in the blurred image training set.

[0025] According to the system of the second aspect of the present application, the steps in the multi-level patch image deblurring method of any one of the first aspect of the present application are implemented.

[0026] The third aspect of the present application discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in the multi-level patch image deblurring method of any one of the first aspect of the present application.

[0027] The fourth aspect of the present application discloses a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the multi-level fragmented image deblurring method in any one of the first aspect of the present application.

[0028] To sum up, the scheme provided by the present application has the following technical effects: first, the present application effectively solves the efficiency problem based on the multi-scale mechanism in the early stage, and by performing spatial pyramid image segmentation under the condition of maintaining image resolution, a smaller convolution kernel can be used for end-to-end image deblurring learning, effectively reducing the model parameter size and accelerating the model running time; in addition, compared with the multi-scale hierarchical structure, the multi-level fragmented image deblurring model proposed in the present application can maintain significant growth in performance when the number of levels is linearly increased, so that the growth relationship between the model size and the deblurring performance is proportional. The multi-level fragmented model topology proposed in the present application can play a role in many low-level computer vision tasks, effectively improving the deblurring performance for deep learning in the fields of image processing such as deblurring and super-resolution. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the specific embodiments or prior art of the present application, the drawings needed in the specific embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0030] Figure 1 The flow chart of the multi-level fragmented image deblurring method according to the embodiment of the present application;

[0031] Figure 2 The multi-level fragmented image deblurring model training and prediction structure diagram according to the embodiment of the present application;

[0032] Figure 3 The structure diagram of the multi-level fragmented image deblurring system according to the embodiment of the present application;

[0033] Figure 4 The structure diagram of the electronic device according to the embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0035] The first aspect of the present application discloses a multi-level patchy image deblurring method. Figure 1 A flow chart of a multi-level patchy image deblurring method according to an embodiment of the present application is shown in Figure 1 The method comprises the following steps:

[0036] Step S1, any blurred image B i (i=1, 2…N) in the blurred image training set is cut in a spatial pyramid structure manner to obtain J levels of image patches Wherein, i represents the i-th blurred image in the blurred image training set, N represents the total number of samples in the blurred image training set, J represents the total number of levels of the spatial pyramid structure, J≥3, (j, k) represents the k-th image patch in the j-th level;

[0037] Step S2, at the J-th level, the image patches are input into the J-th level encoder e J (·) to obtain the corresponding convolutional features k∈{1…2 J-1}, the spatial aggregation is performed on the convolutional features of adjacent image patches in the J-th level to obtain k∈{1…2 J-2}, so that the spatial size of is the same as that of the convolutional features of the image patches in the J-1-th level; the is input into the J-th level decoder d J (·) to obtain the output of the J-th level encoder-decoder network k∈{1…2 J-2};

[0038] Step S3, at the J-1-th level, the image patches k∈{1…2 J-2} and the output of the J-th level encoder-decoder network k∈{1…2 J-2} are subjected to matrix addition, and the result after the matrix addition is input into the J-1-th level encoder e J-1 (·) to obtain the corresponding convolutional features k∈{1…2J-2} and the convolutional features corresponding to the J-1th level decoder are added to obtain the corresponding convolutional features; k∈{1…2 J-2} are added to obtain the corresponding convolutional features;

[0039] Step S4, the spatial aggregation of the adjacent image patch convolutional features of the J-1th level is performed to obtain k∈{1…2 J-3} such that k∈{1…2 J-3} have the same spatial size as the image patch convolutional features of the J-2th level; and the J-1th level decoder d J-1 (·) is inputted to obtain the J-1th level decoder-decoder network output k∈{1…2 J-3};

[0040] Step S5, when J≥5, the manner of referring to the J-1th level to obtain k∈{1…2 J-3} and k∈{1…2 J-3} is used to obtain the spatially aggregated image patch convolutional features k∈{1…2 J-4}…, and the J-2th level…, the 2th level decoder-decoder network output k∈{1…2 J-4}…, When J=4, the manner of referring to the J-1th level to obtain k∈{1…2 J-3} and k∈{1…2 J -3} is used to obtain the spatially aggregated image patch convolutional features the 2th level decoder-decoder network output

[0041] Step S6, the manner of referring to the J-1th level to obtain k∈{1…2 J-2} is used to obtain at the 1th level decoder e1(·); the 1th level decoder d1(·) is inputted, and the 1th level decoder-decoder network output is obtained; as any blurred image B in the blurred image training seti the deblurring output result of (i = 1, 2, …, N).

[0042] According to the method of the first aspect of the present application, the loss function of the training model of the blurred image is constructed by the root mean square error function: The parameter gradient of the training model of the blurred image is generated by the back propagation algorithm, so that the parameters of the training model of the blurred image are updated in the direction of minimizing the loss function.

[0043] Specifically, the loss function of the model is constructed by the root mean square error function where G i is the B i corresponding clear image. The model parameter gradient is generated by the back propagation algorithm, and the model parameters are updated using the learning rate a, which continuously reduces the value of the loss function L. The root mean square error, also known as the standard error, is the square root of the ratio of the square of the deviation of the predicted value from the true value to the number of observations, and is mainly used to measure the deviation between the observed value and the true value.

[0044] According to the method of the first aspect of the present application, the multi-level fragmented image deblurring model training is completed until the loss function value reaches the minimum value.

[0045] Specifically, the multi-level fragmented image deblurring model training is completed until the value of the loss function L no longer decreases or reaches the maximum training iteration number.

[0046] According to the method of the first aspect of the present application, for any blurred image B i (i = 1, 2, …, N) in the blurred image training set, the corresponding obtain the deblurred reconstructed clear sample.

[0047] According to the method of the first aspect of the present application, the total number of layers of the spatial pyramid structure is J = 4.

[0048] Figure 2 is a training and prediction structure diagram of a multi-level fragmented image deblurring model according to an embodiment of the present application; as shown in Figure 2 the model includes: a multi-level fragmented image deblurring model of a 4-layer pyramid structure, the 4-layer pyramid structure in the model includes a decoder, a decoder, a network output end, the decoder is respectively: Encoder1, Encoder2, Encoder3, Encoder4, the decoder is respectively: Decoder1, Decoder2, Decoder3, Decoder4, the network output end is respectively: S1, S2, S3, S4.

[0049] Training structure diagram of the model: For any blurred image B in the blurred image training set... i (i = 1, 2, ..., N) Four levels of image segmentation are performed. Specifically, the first level is the original blurred image, the second level is fragments of size 2, the third level is fragments of size 4, and the fourth level is fragments of size 8. Subsequently, the multi-level image fragments are input into a multi-level decoder-decoder network for feature extraction and fusion. In the fourth level, the image fragments are... Inputting the fourth-level decoder e4(·) yields the corresponding convolutional features. k∈{1…8}; then, the convolutional features of adjacent fragments are spatially aggregated to obtain k∈{1…4}. Through the above operations, we can achieve... The spatial dimensions are the same as the fragment characteristics of level 3. Then, Inputting the fourth-level decoder d4(·) yields the output of the fourth-level decoder-decoder network. k∈{1…4}.

[0050] At level 3, image fragments With the output of the Level 4 decoder-decoder network Matrix addition is performed, inputting the third-level decoder e3(·), and its output is simultaneously added to the fourth-level fragment features to obtain the corresponding convolutional features. k∈{1…4}; then, the convolutional features of adjacent fragments are spatially aggregated to obtain k∈{1…2}. Through the above operations, we can achieve... The spatial dimensions are the same as the fragment characteristics of level 2. Then, Inputting the third-level decoder d3(·) yields the output of the third-level decoder-decoder network. k∈{1…2}.

[0051] Following the implementation method of level 3, the fragment representation of level 2 is obtained. and decoder-decoder network output Referring to the implementation method of level 3, the first-level decoder e1(·) obtains Will Input to the first-level decoder d1(·), output to the first-level decoder-decoder network The network output S1 outputs the final image of the entire multi-level fragmented image deblurring model, and the loss function of the model is constructed using the root mean square error function. By referencing the clear image ground truth values ​​of the training samples, the parameters of the multi-level fragmented image deblurring training model are adjusted to construct a multi-level fragmented image deblurring model.

[0052] Predictive structure diagram of the model: any blurred image B in the blurred image training set i (i=1, 2…N) into the multi-level fragmented image deblurring model, and the prediction result Si of the deblurred image is output.

[0053] The second aspect of the present application discloses a multi-level fragmented image deblurring system. Figure 3 The structure diagram of a multi-level fragmented image deblurring system according to an embodiment of the present application is shown in FIG. 3. Figure 3 As shown in FIG. 3, the system 300 comprises:

[0054] A first processing module 301 is configured to cut any blurred image B in the blurred image training set i (i=1, 2…N) in a spatial pyramid structure to obtain J levels of image fragments Wherein, i represents the i-th blurred image in the blurred image training set, N represents the total number of samples in the blurred image training set, J represents the total number of levels of the spatial pyramid structure, J≥3, and (j, k) represents the k-th image fragment in the j-th level.

[0055] A second processing module 302 is configured to, at the J-th level, input the image fragments to the J-th level decoder e J (·) to obtain the corresponding convolution features k∈{1…2 J-1}, and perform spatial aggregation on the convolution features of adjacent image fragments in the J-th level to obtain k∈{1…2 J-2}, so that the spatial size of is the same as the spatial size of the convolution features of the image fragments in the J-1-th level; and input to the J-th level decoder d J (·) to obtain the J-th level decoder-decoder network output k∈{1…2 J-2};

[0056] A third processing module 303 is configured to, at the J-1-th level, perform matrix addition on the image fragments k∈{1…2 J-2} and the J-th level decoder-decoder network output k∈{1…2 J-2}, and input the result after the matrix addition to the J-1-th level decoder e J-1 (·) to obtain the corresponding convolution features k∈{1…2 J-2} and the convolution feature corresponding to the J-1th decoder is added to the convolution feature of the Jth image patch k∈{1…2 J-2} to obtain the corresponding convolution feature;

[0057] The fourth processing module 304 is configured to perform spatial aggregation on the convolution features of the adjacent image patches at the J-1th level to obtain k∈{1…2 J-3} such that the spatial dimensions of k∈{1…2 J-3} are the same as the spatial dimensions of the convolution features of the image patches at the J-2th level; and inputting the J-1th decoder d J-1 (·) to obtain the decoder-decoder network output k∈{1…2 J-3} at the J-1th level;

[0058] The fifth processing module 305 is configured to, when J≥5, refer to the J-1th level to obtain k∈{1…2 J-3} and k∈{1…2 J-3} in a manner that the spatial aggregation is performed on the convolution features of the image patches at the J-2th, …, 2th level to obtain k∈{1…2 J-4}, …, and the decoder-decoder network output k∈{1…2 J-4}, …, at the J-2th, …, 2th level, respectively. When J=4, refer to the J-1th level to obtain k∈{1…2 J-3} and k∈{1…2 J-3} in a manner that the spatial aggregation is performed on the convolution features of the image patches at the 2th level to obtain the decoder-decoder network output

[0059] The sixth processing module 306 is configured to refer to the J-1th level to obtain k∈{1…2 J-2} in a manner that the 1st level decoder e1(·) obtains inputting the 1st decoder d1(·) to obtain the decoder-decoder network output at the 1st level; taking as any blurred image B in the blurred image training set iThe deblurring output result of (i = 1, 2, …, N).

[0060] The third aspect of the present application discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps of the multi-level fragmented image deblurring method of any one of the first aspect of the present application.

[0061] Figure 4 The structure diagram of the electronic device according to the embodiment of the present application is shown in FIG. 1. Figure 4 As shown in FIG. 1, the electronic device comprises a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0062] Those skilled in the art can understand that, Figure 4 The structure shown in FIG. 1 is only a structure diagram of part of the technical solution of the present application, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0063] The fourth aspect of the present application discloses a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the method for multi-level fragmented image deblurring.

[0064] Please note that the technical features of the above embodiments can be combined in any manner, and for the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered as within the scope of the present disclosure. The above embodiments only express several implementation manners of the present application, which are described in a more specific and detailed manner, but should not be understood as a limitation on the patent scope of the present application. It should be noted that for those skilled in the art, some modifications and improvements can be made without departing from the concept of the present application, and these are within the scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A multi-level fragmented image deblurring method, characterized in that, The method includes: For any blurred image B in the blurred image training set i (i = 1, 2, ..., N), are cut in a spatial pyramid structure to obtain J levels of image fragments. Where i represents the i-th blurred image in the blurred image training set, N represents the total number of samples in the blurred image training set, J represents the total number of levels in the spatial pyramid structure, J≥3, and (j,k) represents the k-th image fragment of the j-th level. At the Jth level, image fragments Input to the J-th level decoder e J (·), to obtain the corresponding convolutional features. Spatially aggregate the convolutional features of adjacent image fragments at the J-th level to obtain Make The spatial dimensions are the same as the spatial dimensions of the image fragment convolution features at level J-1; Input to the J-th level decoder d J (·), to obtain the output of the J-th level decoder-decoder network. At level J-1, image fragments With the output of the J-th level decoder-decoder network Perform matrix addition, and input the result of the matrix addition into the (J-1)th stage decoder e. J-1 (·), to obtain the corresponding convolutional features. The convolutional features corresponding to the J-1 level decoder are added to the convolutional features of the J-th level image fragment to obtain the corresponding convolutional features. Spatially aggregate the convolutional features of adjacent image fragments at the (J-1)th level to obtain Make The spatial dimensions are the same as the spatial dimensions of the image fragment convolution features at level J-2; Input to the (J-1)th level decoder d J-1 (·), to obtain the output of the decoder-decoder network at level J-1. When J≥5, refer to level J-1 to obtain as well as In this manner, spatially aggregated image fragment convolutional features are obtained from level J-2 to level 2. and the decoder-decoder network output from level J-2 to level 2. When J=4, refer to level J-1 to obtain as well as In this way, the image fragment convolutional features after spatial aggregation are obtained at the second level. Output of the Level 2 decoder-decoder network Obtained by referring to level J-1 In this way, the first-level decoder e1(·) obtains Will Input to the first-level decoder d1(·), output to the first-level decoder-decoder network Will As any blurred image B in the blurred image training set i The deblurred output of (i = 1, 2, ..., N).

2. The multi-level fragmented image deblurring method according to claim 1, characterized in that, The loss function for training a blurred image model is constructed using the root mean square error function: The backpropagation algorithm generates the parameter gradients of the blurred image training model, enabling the parameters of the blurred image training model to be updated in the direction of minimizing the loss function, where G... i For B i The corresponding clear image.

3. The multi-level fragmented image deblurring method according to claim 2, characterized in that, The training of the multi-level fragmented image deblurring model is completed when the loss function value reaches its minimum.

4. The multi-level fragmented image deblurring method according to claim 1, characterized in that, For any blurred image B in the blurred image training set i (i = 1, 2, ..., N), by obtaining their corresponding Obtain the clear reconstructed sample after deblurring.

5. The multi-level fragmented image deblurring method according to claim 4, characterized in that, The total number of levels in the spatial pyramid structure is J = 4.

6. A multi-level fragmented image deblurring system, characterized in that, The system includes: The first processing module is configured to process any blurred image B in the blurred image training set. i (i = 1, 2, ..., N), are cut in a spatial pyramid structure to obtain J levels of image fragments. Where i represents the i-th blurred image in the blurred image training set, N represents the total number of samples in the blurred image training set, J represents the total number of levels in the spatial pyramid structure, J≥3, and (j,k) represents the k-th image fragment of the j-th level. The second processing module is configured to, at the J-th level, process image fragments. Input to the J-th level decoder e J (·), to obtain the corresponding convolutional features. Spatially aggregate the convolutional features of adjacent image fragments at the J-th level to obtain Make The spatial dimensions are the same as the spatial dimensions of the image fragment convolution features at level J-1; Input to the J-th level decoder d J (·), to obtain the output of the J-th level decoder-decoder network. The third processing module is configured to, at level J-1, process image fragments. With the output of the J-th level decoder-decoder network Perform matrix addition, and input the result of the matrix addition into the (J-1)th stage decoder e. J-1 (·), to obtain the corresponding convolutional features. The convolutional features corresponding to the J-1 level decoder are combined with the J-level image fragments The convolutional features of the slices are added together to obtain the corresponding convolutional features; The fourth processing module is configured to spatially aggregate the convolutional features of adjacent image fragments at the (J-1)th level to obtain... Make The spatial dimensions are the same as the spatial dimensions of the image fragment convolution features at level J-2; Input to the (J-1)th level decoder d J-1 (·), to obtain the output of the decoder-decoder network at level J-1. The fifth processing module is configured to, when J≥5, refer to the (J-1)th level to obtain... as well as The image fragment convolutional features after spatial aggregation are obtained at levels J-2, ..., 2 respectively. and the decoder-decoder network outputs at levels J-2, ..., 2. When J=4, refer to level J-1 to obtain as well as In this way, the image fragment convolutional features after spatial aggregation are obtained at the second level. Output of the Level 2 decoder-decoder network The sixth processing module is configured to obtain, with reference to level J-1 In this way, the first-level decoder e1(·) obtains Will Input to the first-level decoder d1(·), output to the first-level decoder-decoder network Will As any blurred image B in the blurred image training set i The deblurred output of (i = 1, 2, ..., N).

7. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the multi-level fragmented image deblurring method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-level fragmented image deblurring method according to any one of claims 1 to 5.

Images