Image processing method and device

Abstract

The invention discloses an image processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: determining blurring kernels respectively corresponding to M grayscale image blocks in a grayscale image of a first image, wherein M is a positive integer; based on the blurring kernels corresponding to the M gray level image blocks, deblurring and deconvolution processing is carried out on the gray level image, and M second images are obtained; the M second images are fused to obtain a third image, and the resolution ratio of the third image is larger than that of the first image.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to an image processing method and apparatus thereof. Background Art

[0002] With the rapid development of shooting technology, more and more people use electronic devices to take pictures to assist their daily work and life.

[0003] During the process of taking pictures with an electronic device, if the camera focus cannot be focused on the subject, it will cause the captured image to be blurred, thereby affecting the user's viewing or use of the image. For example, during the process of using an electronic device to scan a QR code, if the camera focus cannot be focused on the QR code, it will cause the captured QR code to be blurred, and thus the QR code cannot be accurately recognized. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide an image processing method and apparatus thereof to solve the problem that an electronic device cannot recognize a blurred image during shooting.

[0005] In a first aspect, the embodiments of this application provide an image processing method, which includes:

[0006] Determine the blur kernels respectively corresponding to M gray image blocks in the gray image of the first image, where M is a positive integer;

[0007] Based on the blur kernels respectively corresponding to the M gray image blocks, perform deblurring and deconvolution processing on the gray image respectively to obtain M second images;

[0008] Fuse the M second images to obtain a third image, and the clarity of the third image is greater than that of the first image.

[0009] In a second aspect, the embodiments of this application provide an image processing apparatus, which includes:

[0010] A first determination module, configured to determine the blur kernels respectively corresponding to M gray image blocks in the gray image of the first image, where M is a positive integer;

[0011] A second determination module, configured to perform deblurring and deconvolution processing on the gray image respectively based on the blur kernels respectively corresponding to the M gray image blocks to obtain M second images;

[0012] A third determination module, configured to fuse the M second images to obtain a third image, and the clarity of the third image is greater than that of the first image.

[0013] In a third aspect, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the method described in the first aspect.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0016] In the embodiment of the present application, by using the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image, the grayscale image of the first image is respectively subjected to deblurring deconvolution processing to obtain M second images, and then the M second images are fused to obtain a third image with a higher resolution than the first image. In this way, through the image processing method provided by the present application, the first image with a lower resolution can be deblurred to obtain a third image with a higher resolution, improving the clarity of the blurred image. In addition, by using the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image, the grayscale image of the first image is respectively subjected to deblurring deconvolution processing. Compared with using a single blur kernel to perform deblurring deconvolution processing on a blurred image, the image processing method provided by the present application can deblur blurred images formed in various complex scenarios, further improving the deblurring effect on blurred images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic flowchart of an image processing method provided by some embodiments of the present application;

[0018] Figure 2 is a schematic diagram of segmenting the grayscale image of the first image provided by some embodiments of the present application;

[0019] Figure 3 is a schematic flowchart of image processing provided by some embodiments of the present application;

[0020] Figure 4 is a schematic flowchart of a fast two-dimensional code detection and recognition method provided by some embodiments of the present application;

[0021] Figure 5 is a schematic flowchart of a blind deconvolution deblurring two-dimensional code detection and recognition method provided by some embodiments of the present application;

[0022] Figure 6It is a schematic structural diagram of an image processing device shown in some embodiments of the present application;

[0023] Figure 7 It is a schematic structural diagram of an electronic device shown in some embodiments of the present application;

[0024] Figure 8 It is a schematic hardware structure diagram of an electronic device shown in some embodiments of the present application. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and the number of objects is not limited. For example, the first object can be one or N. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means an "or" relationship between the associated objects before and after.

[0027] Before introducing the technical solutions of the embodiments of the present application, the background technology of the embodiments of the present application will be introduced first:

[0028] Currently, when deblurring a blurred image, a convolutional blur kernel model is usually constructed, and the deblurring of the blurred image is realized through the operation of deconvolving the blur kernel. Since the blur kernel of this convolutional blur kernel model is invariant, this model can only deblur the blurred image caused by linear shift. In actual shooting, the blurred image may be caused by various factors. At this time, the current convolutional blur kernel model cannot handle the blurred image well.

[0029] To solve the above problems, the embodiments of the present application provide an image processing method, apparatus, electronic device, and storage medium. By using the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image, the grayscale image of the first image is respectively subjected to deblurring deconvolution processing to obtain M second images, and then the M second images are fused to obtain a third image with a higher resolution than the first image. In this way, through the image processing method provided by the present application, the first image with a lower resolution can be deblurred to obtain a third image with a higher resolution, improving the clarity of the blurred image. In addition, by using the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image to respectively perform deblurring deconvolution processing on the grayscale image of the first image, compared with using a single blur kernel to perform deblurring deconvolution processing on the blurred image, the image processing method provided by the present application can deblur the blurred images formed in various complex scenarios, further improving the deblurring effect on the blurred images.

[0030] The technical solution of the embodiments of the present application can be applied to the processing scenario of blurred images, such as the processing scenario of blurred images captured by cameras, extended reality (XR) devices, etc.

[0031] Next, with reference to the accompanying drawings, the image processing method provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0032] Figure 1 FIG. is a schematic flowchart of an image processing method provided by the embodiments of the present application. The execution subject of this image processing method can be an electronic device, such as but not limited to a personal computer (PC), a smart phone, a tablet computer, or a personal digital assistant (PDA), etc.

[0033] As Figure 1 shown, the image processing method provided by the embodiments of the present application may include S110 - S130.

[0034] S110. Determine the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image.

[0035] Among them, the first image may be a color image that needs to be processed for clarity, such as a blurred two-dimensional code image for payment.

[0036] It should be noted that in the following embodiments, a blurred two-dimensional code image for payment is used as an example of the first image for illustration.

[0037] It should be noted that the first image can be an image captured in real time by the image capture device of the electronic device, or can be directly obtained from the local of the electronic device, which is not limited in the embodiments of the present application.

[0038] The above-mentioned M is a positive integer.

[0039] In some embodiments of the present application, the blur kernel can be a convolution kernel used to simulate the motion blur effect. When an object is captured by a camera during movement, due to factors such as movement speed and direction, the pixel values in the image will change in a specific way, resulting in motion blur. The blur kernel is a mathematical model used to describe this blur change.

[0040] The blur kernel is actually a matrix. After the clear image is convolved with the blur kernel, the image becomes blurred, so it is called the blur kernel. The blur kernel is a type of convolution kernel.

[0041] In some embodiments of the present application, in order to accurately obtain the blur kernels corresponding to the M grayscale image blocks respectively, S110 may specifically include:

[0042] Segment the grayscale image of the first image to obtain M grayscale image blocks;

[0043] Perform deconvolution processing on the M grayscale image blocks respectively to obtain the blur kernels corresponding to the M grayscale image blocks respectively.

[0044] Among them, the grayscale image block can be an image block obtained by segmenting the grayscale image of the first image.

[0045] In some embodiments of the present application, after obtaining the first image, the first image can be subjected to grayscale conversion processing to obtain the grayscale image of the first image, and then the grayscale image of the first image can be segmented to obtain M grayscale image blocks, and then deconvolution processing can be performed on the M grayscale image blocks respectively to obtain the blur kernels corresponding to the M grayscale image blocks respectively.

[0046] It should be noted that after obtaining the first image, it is also possible not to perform grayscale conversion processing on the first image, that is, directly segment the first image. Specifically, whether to perform grayscale conversion processing on the first image can be set according to user needs, which is not limited in the embodiments of the present application.

[0047] In addition, when segmenting the grayscale image of the first image, the specific number of segmented image blocks can be set according to user needs, which is not limited in the embodiments of the present application.

[0048] In one example, refer to Figure 2, taking M as 10 as an example, after segmenting the grayscale image 21 of the first image, 10 grayscale image blocks can be obtained. Then, deconvolution processing is performed on each grayscale image block respectively, and the blur kernels corresponding to the 10 grayscale image blocks can be obtained.

[0049] In the embodiments of the present application, by performing grayscale conversion processing on the first image, the subsequent computational complexity can be reduced, and the efficiency of the clarity processing of the first image can be improved. In addition, by segmenting the grayscale image of the first image, M grayscale image blocks can be obtained. Then, deconvolution processing is performed on the M grayscale image blocks respectively, and the blur kernels corresponding to the M grayscale image blocks can be accurately obtained.

[0050] S120. Based on the blur kernels corresponding to the M grayscale image blocks respectively, perform deblurring deconvolution processing on the grayscale image to obtain M second images.

[0051] Among them, the second image can be the image obtained by performing deblurring deconvolution processing on the grayscale image using the blur kernel corresponding to the grayscale image block.

[0052] In some embodiments of the present application, the blur kernels corresponding to the M grayscale image blocks can be used to perform deblurring deconvolution processing on the grayscale image of the first image respectively, and M second images can be obtained.

[0053] Continuing to refer to the above example, the blur kernels corresponding to the 10 grayscale image blocks are obtained: blur kernel 1, blur kernel 2, blur kernel 3, blur kernel 4, blur kernel 5, blur kernel 6, blur kernel 7, blur kernel 8, blur kernel 9, and blur kernel 10. Then, use blur kernel 1 to perform deblurring deconvolution processing on the grayscale image of the first image to obtain a second image, use blur kernel 2 to perform deblurring deconvolution processing on the grayscale image of the first image to obtain a second image, and so on. A total of 10 second images are obtained.

[0054] In some embodiments of the present application, in order to improve the accuracy of the clarity processing of the first image, before S120, the above-mentioned method may further include:

[0055] Cluster the blur kernels corresponding to the M grayscale image blocks to obtain N grayscale image blocks;

[0056] Perform deconvolution processing on the N grayscale image blocks respectively to obtain the blur kernels corresponding to the N grayscale image blocks respectively;

[0057] S120 may specifically include:

[0058] Based on the blur kernels corresponding to the N grayscale image blocks respectively, perform deblurring deconvolution processing on the grayscale image to obtain N second images.

[0059] Among them, the N grayscale image blocks can be the grayscale image blocks obtained by clustering the blur kernels corresponding to the M grayscale image blocks respectively. Here, N is a positive integer, and N < M.

[0060] In some embodiments of the present application, the blur kernels corresponding to the M grayscale image blocks can be clustered to obtain N grayscale image blocks, and then the N grayscale image blocks are respectively deconvolved to obtain the blur kernels corresponding to the N grayscale image blocks. Furthermore, based on the blur kernels corresponding to the N grayscale image blocks, the grayscale images are respectively deblurred by deconvolution to obtain N second images.

[0061] Continuing to refer to the above example, the blur kernels corresponding to 10 grayscale image blocks are obtained: blur kernel 1, blur kernel 2, blur kernel 3, blur kernel 4, blur kernel 5, blur kernel 6, blur kernel 7, blur kernel 8, blur kernel 9, and blur kernel 10. The blur kernels 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 can be clustered. After clustering, the grayscale image blocks with the same clustering results among the 10 grayscale image blocks can be stitched together to form larger grayscale image blocks. For example, the grayscale image blocks with the same clustering results among the 10 grayscale blocks can be stitched together to form larger grayscale image blocks, and finally 3 grayscale image blocks are obtained.

[0062] It should be noted that when clustering the blur kernels corresponding to the M grayscale image blocks, a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering method that does not require setting the number of clusters obtained finally in advance can be used. This DBSCAN clustering method first needs to calculate the neighborhood radius value, which is determined by the k-distance graph method. Specifically, the average distance from each point to its k nearest neighbor points is first calculated, and then the distance matrix is obtained. Each element in the distance matrix is fitted to obtain the k-distance graph, and then the inflection point of the curve is found in the k-distance graph as the neighborhood radius value. Furthermore, based on this neighborhood radius value, the sample points can be clustered.

[0063] It should be noted that the clusters finally obtained by the DBSCAN clustering method do not contain all the previous grayscale image blocks. For the grayscale image blocks that do not form a cluster with other grayscale image blocks, they can be discarded, so as to improve the calculation speed.

[0064] Continuing to refer to the above example, taking M = 10 as an example, when clustering 10 grayscale image blocks using the DBSCAN clustering method, if grayscale image block 2 and grayscale image block 3 can form a cluster, but it is required that a cluster should contain at least 3 grayscale image blocks, so grayscale image block 2 and grayscale image block 3 cannot finally form a cluster and can be discarded. Or, if grayscale image block 4 cannot form a cluster with any other grayscale image blocks, then grayscale image block 4 can be discarded.

[0065] In addition, when using the DBSCAN clustering method for clustering, it is also possible that the same grayscale image block is clustered into different clusters respectively. For example, taking M = 10 as an example, when clustering 10 grayscale image blocks using the DBSCAN clustering method, grayscale image block 2, grayscale image block 3, and grayscale image block 4 can form a cluster, and grayscale image block 2, grayscale image block 4, and grayscale image block 6 can also form a cluster.

[0066] In one example, taking M = 10 and N = 3 as an example, when clustering the blur kernel 1 of grayscale image block 1, the blur kernel 2 of grayscale image block 2, the blur kernel 3 of grayscale image block 3, ……, the blur kernel 10 of grayscale image block 10 using the DBSCAN clustering method, grayscale image block 1, grayscale image block 3, and grayscale image block 5 form cluster 1, grayscale image block 2, grayscale image block 3, grayscale image block 7, and grayscale image block 8 form cluster 2, and grayscale image block 3, grayscale image block 4, and grayscale image block 9 form cluster 3. Then, recalculate the blur kernel 11 corresponding to cluster 1, the blur kernel 12 corresponding to cluster 2, and the blur kernel 13 corresponding to cluster 3. Then, use the blur kernel 11 to perform deblurring and deconvolution processing on the grayscale image of the first image to obtain image 1, use the blur kernel 12 to perform deblurring and deconvolution processing on the grayscale image of the first image to obtain image 2, and use the blur kernel 13 to perform deblurring and deconvolution processing on the grayscale image of the first image to obtain image 3. Here, image 1, image 2, and image 3 are the second images.

[0067] In the embodiments of the present application, by clustering the blur kernels corresponding to M grayscale image blocks respectively, that is, clustering different types of blur kernels generated by different reasons, it can, to a certain extent, suppress the influence of abnormal blur kernels, and at the same time, unreliable blur kernels can be removed, improving the accuracy of the clarity processing of the first image.

[0068] In some embodiments of the present application, in order to improve the flexibility of clustering the blur kernels corresponding to M grayscale image blocks, the clustering of the blur kernels corresponding to M grayscale image blocks to obtain N grayscale image blocks may specifically include:

[0069] For the blur kernel corresponding to the i-th grayscale image block among M grayscale image blocks, calculate the distance between the blur kernels corresponding to other grayscale image blocks and the blur kernel corresponding to the i-th grayscale image block;

[0070] Sort the distances in ascending order to obtain the first sorting;

[0071] Select the distance at the K-th order in the first sorting as an element in the distance matrix;

[0072] Update i to i + 1, and return to execute calculating the distance between the blur kernels corresponding to other grayscale image blocks and the blur kernel corresponding to the i-th grayscale image block; sort the distances in ascending order to obtain the first sorting; select the distance at the K-th order in the first sorting as an element in the distance matrix, until i = M, to obtain the distance matrix, and the elements in the distance matrix are sorted according to the value of i;

[0073] Based on the elements in the distance matrix, determine the neighborhood radius value for clustering the blur kernels corresponding to M grayscale image blocks;

[0074] Based on the neighborhood radius value and the K value, cluster the blur kernels corresponding to M grayscale image blocks to obtain N grayscale image blocks.

[0075] Among them, the i-th grayscale image block can be any grayscale image block among M grayscale image blocks, and the initial value of i is 1.

[0076] The first sorting can be the sorting obtained by sorting the distances between the blur kernels corresponding to other grayscale image blocks and the blur kernel corresponding to the i-th grayscale image block in ascending order.

[0077] The K-th order can be the K-th order position in the first sorting. The value of K here can be the number of grayscale image blocks that should be included at least in each cluster when clustering the blur kernels corresponding to M grayscale image blocks, that is, the minimum sample points. That is to say, when clustering the blur kernels corresponding to M grayscale image blocks, each cluster in the finally obtained clusters should contain at least M grayscale image blocks, that is, a large cluster should contain at least K grayscale image blocks among M grayscale image blocks.

[0078] It should be noted that the value of K mentioned above is generally the data dimension of the image plus 1. Since the grayscale image is two-dimensional data, the value of K here is 2 + 1 = 3.

[0079] In some embodiments of the present application, for the blur kernel corresponding to the i-th grayscale image block among M grayscale image blocks, calculate the distances between the blur kernels corresponding to other grayscale image blocks and the blur kernel corresponding to the i-th grayscale image block. The initial value of i is 1, that is, first for the first grayscale block among M grayscale image blocks, calculate the distances between the blur kernels of other grayscale image blocks and the blur kernel of the first grayscale image block respectively, obtaining M - 1 distances. Then sort the M - 1 distances in ascending order to obtain the first sorting. Select the distance at the 3rd position in the first sorting as an element in the distance matrix.

[0080] Then continue to calculate the distances between the blur kernels of other grayscale image blocks and the blur kernel of the 2nd grayscale image block respectively, obtaining M - 1 distances. Then sort the M - 1 distances in ascending order to obtain the first sorting. Select the distance at the 3rd position in the first sorting as an element in the distance matrix. And so on, until calculating the distances between the blur kernels of other grayscale image blocks and the blur kernel of the M-th grayscale image block respectively, obtaining M - 1 distances. Then sort the M - 1 distances in ascending order to obtain the first sorting. Select the distance at the 3rd position in the first sorting as an element in the distance matrix.

[0081] Then, based on the elements in the distance matrix, determine the neighborhood radius value for clustering the blur kernels corresponding to M grayscale image blocks. Furthermore, based on the neighborhood radius value and the K value, the DBSCAN clustering method can be used to cluster the blur kernels corresponding to M grayscale image blocks, obtaining N grayscale image blocks.

[0082] It should be noted that the elements in the distance matrix are sorted in sequence according to the value of i. That is, the first element in the distance matrix is the distance at the 3rd position selected from the first sorting when i takes the value of 1, the second element in the distance matrix is the distance at the 3rd position selected from the first sorting when i takes the value of 2, and so on. For example, if the distance matrix is D = {d1, d2, d3, … d M}, then d1 is the distance at the 3rd position selected from the first sorting when i takes the value of 1, and d2 is the distance at the 3rd position selected from the first sorting when i takes the value of 1.

[0083] In the embodiments of the present application, by using the DBSCAN clustering method to cluster the blur kernels corresponding to M grayscale image blocks, the number of finally obtained clusters is not limited, which improves the flexibility of clustering the blur kernels corresponding to M grayscale image blocks.

[0084] In some embodiments of the present application, in order to improve the accuracy of clustering the blur kernels corresponding to M grayscale image patches, determining the neighborhood radius value for clustering the blur kernels corresponding to M grayscale image patches based on each element in the distance matrix may specifically include:

[0085] Determine the slope between two adjacent elements in the distance matrix;

[0086] Determine the neighborhood radius value for clustering the blur kernels corresponding to M grayscale image patches according to the distance corresponding to the maximum value in the slopes.

[0087] In some embodiments of the present application, since the DBSCAN clustering method determines the neighborhood radius value according to the k-distance graph method when calculating the neighborhood radius value, that is, fitting each element in the distance matrix to obtain the k-distance graph, and then finding the inflection point of the curve in the k-distance graph as the neighborhood radius value. Therefore, the slope between two adjacent elements in the distance matrix can be determined first, and then the maximum value in the slopes can be used as the neighborhood radius value for clustering the blur kernels corresponding to M grayscale image patches. Specifically, the neighborhood radius value can be obtained according to the following formula (1):

[0088]

[0089] In the above formula (1), diff(·) is the difference function, that is, to calculate the slope between two adjacent elements in the distance matrix, max(·) is the maximum value function in the set, and the find(·) function is to find the index corresponding to the maximum value. The k-distance corresponding to this index is the neighborhood radius value.

[0090] Continuing with the above example, taking M = 10 as an example, the distance matrix is D = {d1, d2, d3,... d 10}, calculate the slope between two adjacent distances in the distance matrix to obtain the slope set x = {x1, x2, x3, x4, x5, x6, x7, x8, x9}, where x1 is the slope between d1 and d2, x2 is the slope between d2 and d3, and so on, x9 is the slope between d9 and d 10 If x2 is the maximum value in the slope set, that is, index = 2, then the neighborhood radius value is d index+1 = d3.

[0091] In the embodiments of the present application, through the slope between two adjacent elements in the distance matrix, the neighborhood radius value for clustering the blur kernels corresponding to M grayscale image patches can be accurately determined, and then the blur kernels corresponding to M grayscale image patches can be clustered based on this neighborhood radius value, improving the accuracy of clustering the blur kernels corresponding to M grayscale image patches.

[0092] S130. Fuse the M second images to obtain a third image.

[0093] Among them, the third image can be the image obtained after fusing the M second images. The resolution of the third image is greater than the resolution of the first image.

[0094] In some embodiments of the present application, the resolution of an image can be used to reflect the clarity of the image. The resolution is the number of pixels contained in each width unit or height unit of the image or video. The higher the resolution of the image acquisition device, the more pixels there are, and the clearer the image. Therefore, when the resolution of the third image is greater than the resolution of the first image, the clarity of the third image is greater than the clarity of the first image.

[0095] In some embodiments of the present application, in order to accurately obtain a third image with better clarity, S130 may specifically include:

[0096] For each pixel point in the M second images, determine the target gray value of the pixel point according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images;

[0097] Obtain the third image according to the target gray value of each pixel point.

[0098] Among them, the target gray value can be the gray value of the pixel point after fusing the M second images.

[0099] In some embodiments of the present application, for each pixel point in the M second images, the target gray value of the pixel point after fusing the M second images can be determined according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images. Then, the third image can be obtained according to the target gray value of each pixel point.

[0100] In the embodiments of the present application, for each pixel point in the M second images, the target gray value of the pixel point after fusing the M second images can be determined according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images, so as to accurately obtain a third image with better clarity.

[0101] In some embodiments of the present application, in order to improve the flexibility of determining the target gray value of the pixel point, the step of determining the target gray value of the pixel point according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images may specifically include:

[0102] In the case where the pixel point exists in any one of the M second images, determine the gray value of the pixel point in the second image as the target gray value;

[0103] When a pixel exists in at least two of the M second images, determine the target gray value of the pixel according to the gray values of the pixel in the at least two second images respectively.

[0104] In some embodiments of the present application, when clustering the blur kernels of M gray image blocks using the DBSCAN clustering method, the finally obtained clusters do not contain all the previous gray image blocks. Therefore, for a certain pixel in the M second images, it does not exist in other second images.

[0105] Continuing to refer to the above example, Image 1 is obtained by deblurring and deconvolving the gray image of the first image using the blur kernel of Cluster 1, and Cluster 1 is composed of Gray Image Block 1, Gray Image Block 3, and Gray Image Block 5. Therefore, Image 1 will contain all the pixels in Gray Image Block 1, Gray Image Block 3, and Gray Image Block 5. Similarly, Image 2 is obtained by deblurring and deconvolving the gray image of the first image using the blur kernel of Cluster 2, and Cluster 2 is composed of Gray Image Block 2, Gray Image Block 3, Gray Image Block 7, and Gray Image Block 8. Therefore, Image 2 will contain all the pixels in Gray Image Block 2, Gray Image Block 3, Gray Image Block 7, and Gray Image Block 8. Image 3 is obtained by deblurring and deconvolving the gray image of the first image using the blur kernel of Cluster 3, and Cluster 3 is composed of Gray Image Block 3, Gray Image Block 4, and Gray Image Block 9. Therefore, Image 3 will contain all the pixels in Gray Image Block 3, Gray Image Block 4, and Gray Image Block 9. That is, all the pixels in Gray Image Block 1 only exist in Image 1, and all the pixels in Gray Image Block 3 exist in Image 1, Image 2, and Image 3.

[0106] In this way, the target gray value of the pixel can be determined according to the relationship between the pixel and the M second images. That is, when the pixel exists in any one of the M second images, the gray value of the pixel can be determined as the target gray value.

[0107] Continuing to refer to the above example, for a certain pixel A in Image 1, this pixel A is a pixel in Gray Image Block 1 and only exists in Image 1. Therefore, when fusing Image 1, Image 2, and Image 3, the target gray value of this pixel A is the gray value of pixel A in Image 1.

[0108] When a pixel exists in at least two of the M second images, the target gray value of the pixel can be determined according to the gray values of the pixel in the at least two second images respectively. Specifically, the gray values of the pixel in the at least two second images can be weighted and averaged to obtain the target gray value of the pixel.

[0109] Continuing to refer to the above example, for a certain pixel point B in Image 1, this pixel point B is a pixel point in the grayscale image block 3, and this pixel point B exists in Image 1, Image 2, and Image 3. Therefore, when fusing Image 1, Image 2, and Image 3, the target grayscale value of this pixel point B is the grayscale value obtained by performing weighted average calculation on the grayscale value of pixel point B in Image 1, the grayscale value of pixel point B in Image 2, and the grayscale value of pixel point B in Image 3.

[0110] In the embodiments of the present application, according to the relationship between the pixel point and the M second images, the target grayscale value of the pixel point can be determined, thus improving the flexibility in determining the target grayscale value of the pixel point.

[0111] In some embodiments of the present application, in order to avoid affecting the subsequent processing of the third image, after S130, the above-mentioned method may further include:

[0112] Determine the resolution of the third image;

[0113] When the resolution is less than the first threshold, update the third image to the first image, and return to execute the steps of determining the blur kernels corresponding to the M grayscale image blocks in the grayscale image of the first image respectively, performing deblurring deconvolution processing on the grayscale image respectively based on the blur kernels corresponding to the M grayscale image blocks, obtaining M second images; fusing the M second images to obtain the third image, until the resolution is greater than or equal to the first threshold.

[0114] Among them, the first threshold can be a preset threshold for the resolution of the third image, and this first threshold can be determined according to prior experience. Specifically, this first threshold can be a threshold for ensuring that the third image will not affect subsequent operations due to clarity. For example, when the first image is a two-dimensional code image for payment, this first threshold should be a clarity threshold that can clearly identify the two-dimensional code image.

[0115] In some embodiments of the present application, after obtaining the third image, the resolution of the third image can be determined. When the resolution is less than the first threshold, update the third image to the first image, and return to execute the above S110 - S130 process until the resolution is greater than or equal to the first threshold.

[0116] In the embodiments of the present application, when it is determined that the resolution of the third image is less than the first threshold, the third image can be updated to the first image, and the clarity optimization process for the first image can be repeated, so as to ensure the clarity of the obtained third image and avoid affecting the subsequent processing of the third image.

[0117] The following takes the first image as a two-dimensional code image for user payment as an example to illustrate in detail the image processing method provided by the embodiments of the present application.

[0118] Since the "scanning code" scenario requires the system to respond quickly with results, when scanning a QR code, a fast QR code detection and recognition method with less system occupancy is first used. This method can achieve the detection and recognition function for most clear QR codes. When the fast QR code detection and recognition method fails to successfully detect the QR code, the image processing method of S110 - S130 above can be used. The method of S110 - S130 above can also be called the blind deconvolution deblurring QR code detection and recognition method in complex scenarios.

[0119] Figure 3 It is a schematic flowchart of an image processing method provided by an embodiment of the present application. As Figure 3 shown, the image processing method provided by the embodiment of the present application may include S31 - S36.

[0120] S31. Obtain a QR code image.

[0121] S32. Perform grayscale conversion on the QR code image to obtain a grayscale image of the QR code image.

[0122] S33. Use the fast QR code detection and recognition method to recognize the grayscale image of the QR code image.

[0123] In this S33, the fast QR code detection and recognition method is a currently relatively mature method for recognizing QR codes. The specific fast QR code detection and recognition method will be introduced in detail in the following embodiments.

[0124] S34. Determine whether the QR code image can be successfully recognized. If so, execute S35; if not, execute S36.

[0125] S35. Output the QR code recognition result.

[0126] In S35, since the QR code image is a QR code image for payment, after successfully recognizing the QR code image, the recognition result of successful payment can be directly displayed.

[0127] S36. Use the blind deconvolution deblurring QR code detection and recognition method to recognize the grayscale image of the QR code image.

[0128] In this S36, by using the processes of S120 - S130 in the above embodiment, a clear QR code image can be obtained, and then the fast QR code detection and recognition method can be used to recognize and detect the clear QR code image, and the QR code image can be successfully recognized.

[0129] The following details the fast QR code detection and recognition method in S33 above. As Figure 4 shown, this fast QR code detection and recognition method includes S41 - S44.

[0130] S41. Binarize the grayscale image of the QR code image to obtain a binarized image.

[0131] In this S41, the grayscale image can be converted into a black-and-white image through threshold segmentation to obtain a binarized image.

[0132] After converting the grayscale image into a binarized image, the computational complexity can be reduced. That is, whether S41 is executed can be set according to user needs and is not limited in the embodiments of this application.

[0133] S42. Remove the noise in the binarized image.

[0134] In S42, the noise in the binarized image can be removed through a noise removal method. For example, median filtering can be performed on the binarized image to remove isolated noise points in the binarized image, so as to maintain good edge details of the binarized image.

[0135] S43. Perform edge detection on the binarized image after removing noise to determine the position of the QR code.

[0136] In S43, after determining the position of the QR code, if there is a certain angular deviation between the QR code and the image acquisition device of the electronic device, the position of the QR code can be rotationally corrected so that the QR code is horizontally aligned with the image acquisition device of the electronic device.

[0137] S44. Detect and decode the QR code.

[0138] In S44, after processing the QR code, the image acquisition device of the electronic device can be directly used to scan the QR code to obtain a scan result.

[0139] The blind deconvolution deblurring QR code detection and recognition method in the above S36 is introduced in detail below. As Figure 5 shown, this blind deconvolution deblurring QR code detection and recognition method includes S51 - S56.

[0140] S51. Divide the grayscale image of the QR code image into M grayscale image blocks.

[0141] S52. Perform deconvolution processing on each of the M grayscale image blocks to obtain the blur kernels corresponding to each of the M grayscale image blocks.

[0142] S53. Cluster the blur kernels corresponding to each of the M grayscale image blocks to obtain N grayscale image blocks.

[0143] S54. Perform deconvolution processing on each of the N grayscale image blocks to obtain the blur kernels corresponding to each of the N grayscale image blocks.

[0144] S55. Use the blur kernels corresponding to N grayscale image patches respectively to perform deblurring deconvolution processing on the grayscale image of the QR code image, and obtain N second images.

[0145] S56. Fuse the N second images to obtain a third image.

[0146] The blind deconvolution deblurring QR code detection and recognition method provided by the embodiment of the present application can handle image blurring caused by space-variant, space-invariant, and various combined reasons, without making any parameter assumptions about the movement of the image acquisition object or object, and can restore clear images through clustering and fusion in various complex scenarios.

[0147] For the image processing method provided by the embodiment of the present application, the execution subject may be an image processing device. In the embodiment of the present application, taking the image processing device executing the image processing method as an example, the image processing device provided by the embodiment of the present application is described.

[0148] Figure 6 It is a schematic structural diagram of an image processing device shown according to an exemplary embodiment. As Figure 6 shown, the image processing device 600 may include:

[0149] A first determination module 610, configured to determine the blur kernels corresponding to M grayscale image patches in the grayscale image of the first image, where M is a positive integer;

[0150] A second determination module 620, configured to perform deblurring deconvolution processing on the grayscale image respectively based on the blur kernels corresponding to the M grayscale image patches, and obtain M second images;

[0151] A third determination module 630, configured to fuse the M second images to obtain a third image, and the resolution of the third image is greater than the resolution of the first image.

[0152] In the embodiments of the present application, by using the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image, the grayscale image of the first image is respectively subjected to deblurring deconvolution processing to obtain M second images, and then the M second images are fused to obtain a third image with a higher resolution than the first image. Thus, through the image processing method provided by the present application, the first image with a lower resolution can be deblurred to obtain a third image with a higher resolution, improving the clarity of the blurred image. In addition, by using the blur kernels respectively corresponding to M grayscale image blocks in the grayscale image of the first image, the grayscale image of the first image is respectively subjected to deblurring deconvolution processing. Thus, compared with using a single blur kernel to perform deblurring deconvolution processing on a blurred image, the image processing method provided by the present application can deblur a blurred image formed in various complex scenarios, further improving the deblurring effect on the blurred image.

[0153] In some embodiments of the present application, the first determination module 610 is specifically configured to:

[0154] Segment the grayscale image of the first image to obtain M grayscale image blocks;

[0155] Perform deconvolution processing on the M grayscale image blocks respectively to obtain the blur kernels respectively corresponding to the M grayscale image blocks.

[0156] In some embodiments of the present application, the above-mentioned device may further include:

[0157] A clustering module, configured to cluster the blur kernels respectively corresponding to the M grayscale image blocks to obtain N grayscale image blocks before performing deblurring deconvolution processing on the grayscale image respectively based on the blur kernels respectively corresponding to the M grayscale image blocks to obtain M second images;

[0158] A fourth determination module, configured to perform deconvolution processing on the N grayscale image blocks respectively to obtain the blur kernels respectively corresponding to the N grayscale image blocks;

[0159] The second determination module 620 is specifically configured to:

[0160] Perform deblurring deconvolution processing on the grayscale image respectively based on the blur kernels respectively corresponding to the N grayscale image blocks to obtain N second images.

[0161] In some embodiments of the present application, the clustering module may include:

[0162] A first determination unit, configured to calculate the distance between the blur kernel corresponding to the i-th grayscale image block among the M grayscale image blocks and the blur kernels corresponding to other grayscale image blocks, and the initial value of i is 1;

[0163] A sorting unit, configured to sort the distances in ascending order to obtain a first sorting.

[0164] A selection unit, configured to select the distance at the K-th order in the first sorting as an element in the distance matrix, where the value of K is the minimum number of grayscale image blocks included in each cluster when clustering the blur kernels corresponding to the M grayscale image blocks, which is preset.

[0165] A second determination unit, configured to update i to i + 1, return to execute calculating the distances between the blur kernels corresponding to other grayscale image blocks and the blur kernel corresponding to the i-th grayscale image block; sort the distances in ascending order to obtain a first sorting; select the distance at the K-th order in the first sorting as an element in the distance matrix, until i = M, to obtain the distance matrix, and the elements in the distance matrix are sorted according to the value of i.

[0166] A third determination unit, configured to determine a neighborhood radius value for clustering the blur kernels corresponding to the M grayscale image blocks based on the elements in the distance matrix.

[0167] A clustering unit, configured to cluster the blur kernels corresponding to the M grayscale image blocks based on the neighborhood radius value and the K value to obtain N grayscale image blocks.

[0168] In some embodiments of the present application, the third determination module 630 may include:

[0169] A fourth determination unit, configured to, for each pixel point in the M second images, determine a target grayscale value of the pixel point according to the grayscale value of the pixel point in the M second images and the relationship between the pixel point and the M second images.

[0170] A fifth determination unit, configured to obtain a third image according to the target grayscale value of each pixel point.

[0171] The image processing apparatus in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0172] The image processing apparatus in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0173] The image processing apparatus provided by the embodiments of the present application can implement Figure 1 each process implemented by the method embodiments. To avoid repetition, it will not be described in detail here.

[0174] Optionally, as Figure 7 shown, the embodiments of the present application further provide an electronic device 700, including a processor 701 and a memory 702. A program or instruction that can run on the processor 701 is stored on the memory 702. When the program or instruction is executed by the processor 701, it implements each step of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[0175] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0176] Figure 8 A schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0177] The electronic device 800 includes, but is not limited to, components such as a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810.

[0178] Those skilled in the art can understand that the electronic device 800 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 810 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 8 The structure of the electronic device shown does not limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0179] Among them, the processor 810 is used to determine the blur kernels corresponding to M gray image blocks in the gray image of the first image, where M is a positive integer; based on the blur kernels corresponding to the M gray image blocks respectively, perform deblurring deconvolution processing on the gray image respectively to obtain M second images; fuse the M second images to obtain a third image, and the resolution of the third image is greater than the resolution of the first image.

[0180] In this way, by using the blur kernels corresponding to M gray image blocks in the gray image of the first image to perform deblurring deconvolution processing on the gray image of the first image respectively to obtain M second images, and then fusing the M second images, a third image with a higher resolution than the first image can be obtained. In this way, through the image processing method provided by this application, the first image with a lower resolution can be deblurred to obtain a third image with a higher resolution, improving the clarity of the blurred image. In addition, by using the blur kernels corresponding to M gray image blocks in the gray image of the first image to perform deblurring deconvolution processing on the gray image of the first image respectively, compared with using a single blur kernel to perform deblurring deconvolution processing on the blurred image, the image processing method provided by this application can deblur the blurred image formed in various complex scenarios, further improving the deblurring effect on the blurred image.

[0181] Optionally, the processor 810 is further used to segment the gray image of the first image to obtain M gray image blocks; perform deconvolution processing on the M gray image blocks respectively to obtain the blur kernels corresponding to the M gray image blocks respectively.

[0182] Thus, by performing grayscale conversion processing on the first image, the subsequent computational complexity can be reduced, and the efficiency of the clarity processing of the first image can be improved. In addition, by segmenting the grayscale image of the first image, M grayscale image blocks can be obtained. Then, deconvolution processing is performed on the M grayscale image blocks respectively, and the blur kernels corresponding to the M grayscale image blocks can be accurately obtained.

[0183] Optionally, the processor 810 is further configured to cluster the blur kernels corresponding to the M grayscale image blocks respectively to obtain N grayscale image blocks; perform deconvolution processing on the N grayscale image blocks respectively to obtain the blur kernels corresponding to the N grayscale image blocks respectively; and perform deblurring deconvolution processing on the grayscale image respectively based on the blur kernels corresponding to the N grayscale image blocks respectively to obtain N second images.

[0184] Thus, by clustering the blur kernels corresponding to the M grayscale image blocks respectively, that is, clustering different types of blur kernels caused by different reasons, the influence of abnormal blur kernels can be suppressed to a certain extent, and at the same time, unreliable blur kernels can be removed, improving the accuracy of the clarity processing of the first image.

[0185] Optionally, the processor 810 is further configured to calculate the distance between the blur kernel corresponding to the i-th grayscale image block among the M grayscale image blocks and the blur kernels corresponding to the other grayscale image blocks, where the initial value of i is 1; sort the distances in ascending order to obtain a first sorting; select the distance at the K-th order in the first sorting as an element in the distance matrix, where the value of K is the minimum number of grayscale image blocks included in each cluster when clustering the blur kernels corresponding to the M grayscale image blocks as preset; update i to i + 1, and return to execute the calculation of the distance between the blur kernel corresponding to the i-th grayscale image block and the blur kernels corresponding to the other grayscale image blocks; sort the distances in ascending order to obtain a first sorting; select the distance at the K-th order in the first sorting as an element in the distance matrix until i = M, to obtain the distance matrix, and the elements in the distance matrix are sorted according to the value of i; determine the neighborhood radius value for clustering the blur kernels corresponding to the M grayscale image blocks based on the elements in the distance matrix; and cluster the blur kernels corresponding to the M grayscale image blocks based on the neighborhood radius value and the value of K to obtain N grayscale image blocks.

[0186] Thus, by clustering the blur kernels corresponding to the M grayscale image blocks by the DBSCAN clustering method, the number of finally obtained clusters is not limited, improving the flexibility of clustering the blur kernels corresponding to the M grayscale image blocks.

[0187] Optionally, the processor 810 is further configured to determine, for each pixel point in the M second images, a target gray value of the pixel point according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images; and obtain a third image according to the target gray value of each pixel point.

[0188] In this way, for each pixel point in the M second images, the target gray value of the pixel point after the M second images are fused can be determined according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images, so that a third image with better clarity can be accurately obtained.

[0189] It should be understood that in the embodiments of the present application, the input unit 804 may include a Graphics Processing Unit (GPU) 8041 and a microphone 8042. The graphics processor 8041 processes the image data of static pictures or videos obtained by an image capture device (such as a color camera) in a video capture mode or an image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also referred to as a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. The other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0190] The memory 809 can be used to store software programs and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 809 may include a volatile memory or a non-volatile memory, or the memory 809 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 809 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0191] The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 810 either.

[0192] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above embodiment of the image processing method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0193] Among them, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks or optical discs, etc.

[0194] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above embodiment of the image processing method, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0195] It should be understood that the chip mentioned in the embodiment of the present application may also be referred to as a system-on-chip, system chip, chip system or system-on-chip, etc.

[0196] The embodiment of the present application provides a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above embodiment of the image processing method, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0197] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0198] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0199] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Determining the blur kernels corresponding to M grayscale image patches in the grayscale image of the first image, where M is a positive integer; Based on the blur kernels corresponding to the M grayscale image patches respectively, performing deblurring deconvolution processing on the grayscale image respectively to obtain M second images; Fusing the M second images to obtain a third image, and the resolution of the third image is greater than that of the first image.

2. The method according to claim 1, wherein The determining the blur kernels corresponding to M grayscale image patches in the grayscale image of the first image includes: Segmenting the grayscale image of the first image to obtain M grayscale image patches; Performing deconvolution processing on the M grayscale image patches respectively to obtain the blur kernels corresponding to the M grayscale image patches respectively.

3. The method according to claim 1, characterized in that, Before the step of, based on the blur kernels corresponding to the M grayscale image patches respectively, performing deblurring deconvolution processing on the grayscale image respectively to obtain M second images, the method further includes: Clustering the blur kernels corresponding to the M grayscale image patches respectively to obtain N grayscale image patches; Performing deconvolution processing on the N grayscale image patches respectively to obtain the blur kernels corresponding to the N grayscale image patches respectively; The step of, based on the blur kernels corresponding to the M grayscale image patches respectively, performing deblurring deconvolution processing on the grayscale image respectively to obtain M second images includes: Based on the blur kernels corresponding to the N grayscale image patches respectively, performing deblurring deconvolution processing on the grayscale image respectively to obtain N second images.

4. The method according to claim 3, wherein The step of clustering the blur kernels corresponding to the M grayscale image patches to obtain N grayscale image patches includes: For the blur kernel corresponding to the i-th grayscale image patch among the M grayscale image patches, calculating the distance between the blur kernels corresponding to other grayscale image patches and the blur kernel corresponding to the i-th grayscale image patch, and the initial value of i is 1; Sorting the distances in ascending order to obtain a first sorting; Selecting the distance at the K-th order in the first sorting as an element in the distance matrix, where the value of K is the minimum number of grayscale image patches included in each cluster when clustering the blur kernels corresponding to the M grayscale image patches; Updating i to i + 1, returning to execute calculating the distance between the blur kernels corresponding to other grayscale image patches and the blur kernel corresponding to the i-th grayscale image patch; sorting the distances in ascending order to obtain a first sorting; selecting the distance at the K-th order in the first sorting as an element in the distance matrix until i = M, obtaining the distance matrix, and the elements in the distance matrix are sorted according to the value of i; Based on the elements in the distance matrix, determining the neighborhood radius value for clustering the blur kernels corresponding to the M grayscale image patches; Based on the neighborhood radius value and the value of K, clustering the blur kernels corresponding to the M grayscale image patches to obtain N grayscale image patches.

5. The method according to claim 1, wherein The step of fusing the M second images to obtain a third image includes: For each pixel point in the M second images, determine the target gray value of the pixel point according to the gray value of the pixel point in the M second images and the relationship between the pixel point and the M second images; Obtain a third image according to the target gray value of each pixel point.

6. An image processing apparatus, characterized in that, The device includes: A first determination module, configured to determine the blur kernels respectively corresponding to M gray image blocks in the gray image of the first image, where M is a positive integer; A second determination module, configured to perform deblurring and deconvolution processing on the gray image respectively based on the blur kernels respectively corresponding to the M gray image blocks to obtain M second images; A third determination module, configured to fuse the M second images to obtain a third image, and the resolution of the third image is greater than the resolution of the first image.

7. The device according to claim 6, characterized in that, The first determination module is specifically configured to: Segment the gray image of the first image to obtain M gray image blocks; Perform deconvolution processing on the M gray image blocks respectively to obtain the blur kernels respectively corresponding to the M gray image blocks.

8. The device according to claim 6, characterized in that, The device further includes: A clustering module, configured to cluster the blur kernels respectively corresponding to the M gray image blocks to obtain N gray image blocks before performing deblurring and deconvolution processing on the gray image respectively based on the blur kernels respectively corresponding to the M gray image blocks to obtain M second images; A fourth determination module, configured to perform deconvolution processing on the N gray image blocks respectively to obtain the blur kernels respectively corresponding to the N gray image blocks; The second determination module is specifically configured to: Perform deblurring and deconvolution processing on the gray image respectively based on the blur kernels respectively corresponding to the N gray image blocks to obtain N second images.

9. The device according to claim 8, characterized in that, The clustering module includes: A first determination unit, configured to calculate the distance between the blur kernel corresponding to the i-th gray image block among the M gray image blocks and the blur kernels corresponding to other gray image blocks, and the initial value of i is 1; A sorting unit, configured to sort the distances in ascending order to obtain a first sorting; A selection unit, configured to select the distance at the K-th order in the first sorting as an element in the distance matrix, where the value of K is the minimum number of gray image blocks included in each cluster when clustering the blur kernels corresponding to the M gray image blocks; A second determination unit, configured to update i to i + 1, return to execute calculating the distance between the blur kernel corresponding to the i-th gray image block and the blur kernels corresponding to other gray image blocks; sort the distances in ascending order to obtain a first sorting; select the distance at the K-th order in the first sorting as an element in the distance matrix until i = M to obtain the distance matrix, and the elements in the distance matrix are sorted according to the value of i; A third determination unit, configured to determine the neighborhood radius value for clustering the blur kernels corresponding to the M gray image blocks based on the elements in the distance matrix; A clustering unit, configured to cluster the blur kernels corresponding to the M grayscale image blocks based on the neighborhood radius value and the K value, so as to obtain N grayscale image blocks.

10. The device according to claim 6, characterized in that, The third determination module includes: A fourth determination unit, configured to determine a target grayscale value of each pixel point in the M second images according to the grayscale value of the pixel point in the M second images and the relationship between the pixel point and the M second images; A fifth determination unit, configured to obtain a third image according to the target grayscale value of each pixel point.

Images