Image reconstruction method, device, computer equipment and storage medium

Abstract

The present application relates to an image reconstruction method, apparatus, computer equipment, and storage medium, and relates to the field of image processing, and can be used in the field of financial technology or other related fields. The method comprises: in the i-th round of image reconstruction, segmenting the image to be processed using a target image segmentation method to obtain a plurality of first image blocks; performing reconstruction processing on each first image block separately to obtain a reconstructed image block corresponding to each first image block, and merging each reconstructed image block to obtain a reconstructed image; downsampling the reconstructed image to obtain a downsampled image, and segmenting the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks; and obtaining a target super-resolution image based on the loss between the first image block and the second image block and the reconstructed image. The use of this method can expand the scope of application of the image reconstruction method.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image reconstruction method, apparatus, computer device, and storage medium. Background Art

[0002] Image resolution refers to the number of pixels per inch and is a metric used to evaluate image quality. The higher the image resolution, the better the image quality. Currently, many business scenarios, such as facial recognition, require high image clarity. However, the image clarity of different devices varies. Low-resolution images can negatively impact business processing.

[0003] Currently, super-resolution reconstruction using the Enhanced Deep Super-Resolution Network (EDSR) neural network is a mainstream approach for reconstructing low-resolution images. The ESDR neural network optimizes the residual block structure, removes the normalization module, and increases network depth by stacking 65 convolutional layers, effectively improving image reconstruction.

[0004] However, since the EDSR neural network is trained using image pairs of low-resolution and high-resolution images, the trained model is prone to overfitting problems, which means that the model only performs well on images similar to the training set images and has limited applicability. Summary of the Invention

[0005] Based on this, it is necessary to provide an image reconstruction method, apparatus, computer device and storage medium to address the above technical problems.

[0006] In a first aspect, the present application provides an image reconstruction method. The method comprises:

[0007] In the i-th round of image reconstruction, a target image segmentation method is used to segment the image to be processed to obtain a plurality of first image blocks. The target image segmentation method is used to indicate the size and image content overlap of the first image blocks. The number and / or size of the first image blocks obtained by segmentation using the target image segmentation method are different from the number and / or size of the image blocks obtained by segmentation using the image segmentation method in the previous i-1 rounds of image reconstruction. There is overlap between adjacent first image blocks. The image content overlap is used to characterize the degree of overlap between adjacent first image blocks. Wherein, i is a positive integer.

[0008] Performing reconstruction processing on each of the first image blocks to obtain a reconstructed image block corresponding to each of the first image blocks, and merging the reconstructed image blocks to obtain a reconstructed image;

[0009] Downsampling the reconstructed image to obtain a downsampled image, and segmenting the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks, wherein the size of the downsampled image is the same as the size of the image to be processed;

[0010] A target super-resolution image is obtained according to the loss between the first image block and the second image block, and the reconstructed image.

[0011] In one embodiment, obtaining a target super-resolution image based on the loss between the first image block and the second image block and the reconstructed image includes:

[0012] determining a loss between the first image block and the second image block;

[0013] When the loss is less than a loss threshold, the reconstructed image is used as a target super-resolution image; or

[0014] When the loss is greater than or equal to the loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0015] In one embodiment, in the i-th round of image reconstruction, before segmenting the image to be processed using the target image segmentation method, the method further includes:

[0016] When i is greater than 1, the size of the image block and / or the image content overlap corresponding to the image segmentation method used in the (i-1)th round of image reconstruction is adjusted to obtain the target image segmentation method.

[0017] In one embodiment, in the i-th round of image reconstruction, before segmenting the image to be processed using the target image segmentation method, the method further includes:

[0018] When i is equal to 1, determining the complexity of the image to be processed;

[0019] determining, according to the complexity of the image to be processed, an image content overlap between image blocks, wherein the image content overlap is positively correlated with the complexity of the image to be processed;

[0020] The target image segmentation method is obtained according to the image content overlap and the preset size of the image block.

[0021] In one embodiment, when the loss is less than a loss threshold, using the reconstructed image as a target super-resolution image includes:

[0022] determining an image perception loss between the first image block and the second image block when the loss is less than a loss threshold;

[0023] When the image perceptual loss is less than a perceptual loss threshold, using the reconstructed image as a target super-resolution image; or

[0024] When the image perceptual loss is greater than or equal to the perceptual loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the image perceptual loss between the first image block and the second image block is less than the perceptual loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0025] In one embodiment, the merging of the reconstructed image blocks to obtain the reconstructed image includes:

[0026] determining an overlapping portion between the reconstructed image blocks according to overlapping portions between adjacent first image blocks;

[0027] The reconstructed image blocks are spliced together according to overlapping portions between the reconstructed image blocks to obtain a reconstructed image.

[0028] In one embodiment, the performing reconstruction processing on each of the first image blocks to obtain a reconstructed image block corresponding to each of the first image blocks includes:

[0029] Identifying image content in the first image block to obtain a content classification corresponding to the first image block;

[0030] determining an image reconstruction network corresponding to the first image block according to the content classification corresponding to the first image block;

[0031] The image reconstruction network is used to reconstruct the first image block to obtain a reconstructed image block.

[0032] In one embodiment, the ratio of the size of the reconstructed image block to the size of the first image block is the same as the ratio of the size of the target super-resolution image to the size of the image to be processed.

[0033] In a second aspect, the present application further provides an image reconstruction device. The device comprises:

[0034] a segmentation module configured to segment the image to be processed using a target image segmentation method in an i-th round of image reconstruction to obtain a plurality of first image blocks, where the target image segmentation method is used to indicate the size and image content overlap of the first image blocks, the number and / or size of the first image blocks obtained by segmentation using the target image segmentation method being different from the number and / or size of the image blocks obtained by segmentation using the image segmentation method in the previous i-1 rounds of image reconstruction, there being overlap between adjacent first image blocks, and the image content overlap being used to characterize the degree of overlap between adjacent first image blocks, where i is a positive integer;

[0035] a reconstruction module, configured to perform reconstruction processing on each of the first image blocks respectively to obtain a reconstructed image block corresponding to each of the first image blocks, and to merge the reconstructed image blocks to obtain a reconstructed image;

[0036] a downsampling module, configured to perform downsampling processing on the reconstructed image to obtain a downsampled image, and segment the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks, wherein the size of the downsampled image is the same as the size of the image to be processed;

[0037] An acquisition module is configured to obtain a target super-resolution image based on the loss between the first image block and the second image block and the reconstructed image.

[0038] In one embodiment, the acquisition module is further configured to:

[0039] determining a loss between the first image block and the second image block;

[0040] When the loss is less than a loss threshold, the reconstructed image is used as a target super-resolution image; or

[0041] When the loss is greater than or equal to the loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0042] In one embodiment, the apparatus further comprises:

[0043] The adjustment module is used to adjust the size of the image block and / or the image content overlap corresponding to the image segmentation method used in the (i-1)th round of image reconstruction when i is greater than 1, so as to obtain the target image segmentation method.

[0044] In one embodiment, the apparatus further comprises:

[0045] A first determining module, configured to determine the complexity of the image to be processed when i is equal to 1;

[0046] a second determining module, configured to determine an image content overlap between image blocks according to the complexity of the image to be processed, wherein the image content overlap is positively correlated with the complexity of the image to be processed;

[0047] The segmentation method acquisition module is used to obtain the target image segmentation method according to the image content overlap and the preset size of the image block.

[0048] In one embodiment, the acquisition module is further configured to:

[0049] determining an image perception loss between the first image block and the second image block when the loss is less than a loss threshold;

[0050] When the image perceptual loss is less than a perceptual loss threshold, using the reconstructed image as a target super-resolution image; or

[0051] When the image perceptual loss is greater than or equal to the perceptual loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the image perceptual loss between the first image block and the second image block is less than the perceptual loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0052] In one embodiment, the reconstruction module is further configured to:

[0053] determining an overlapping portion between the reconstructed image blocks according to overlapping portions between adjacent first image blocks;

[0054] The reconstructed image blocks are spliced together according to overlapping portions between the reconstructed image blocks to obtain a reconstructed image.

[0055] In one embodiment, the reconstruction module is further configured to:

[0056] Identifying image content in the first image block to obtain a content classification corresponding to the first image block;

[0057] determining an image reconstruction network corresponding to the first image block according to the content classification corresponding to the first image block;

[0058] The image reconstruction network is used to reconstruct the first image block to obtain a reconstructed image block.

[0059] In one embodiment, the ratio of the size of the reconstructed image block to the size of the first image block is the same as the ratio of the size of the target super-resolution image to the size of the image to be processed.

[0060] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above methods when executing the computer program.

[0061] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any of the above methods is implemented.

[0062] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program implements any of the above methods when executed by a processor.

[0063] The above-mentioned image reconstruction method, device, computer equipment and storage medium, through multiple rounds of image reconstruction, and in each round, a target image segmentation method that is different from the image segmentation method in all previous rounds is used to segment the image to be processed, to obtain multiple overlapping first image blocks, and then the first image blocks are reconstructed and spliced to obtain a reconstructed image, and the reconstructed image is downsampled, and the downsampled reconstructed image is segmented by the target image segmentation method to obtain multiple second image blocks, and then the target super-resolution image is obtained based on the loss between the first image block and the second image block and the reconstructed image. The embodiment of the present application divides the image to be processed into multiple overlapping first image blocks, so when the first image block is reconstructed into the reconstructed image block, the overlapping part is equivalent to being reconstructed twice or even more times, so when the reconstructed image blocks are spliced, it is equivalent to fusing the results of multiple reconstructions, so that a better image reconstruction effect can be obtained. The embodiment of the present application only requires one image to be processed, and the target super-resolution image can be obtained by performing multiple iterative reconstructions on the image to be processed. Therefore, there is no need to provide a training set containing low-resolution image and high-resolution image pairs, and there is no risk of overfitting due to training on the training set. The embodiments of the present application not only have a wide range of applications, but also can achieve good performance in various scenes and various types of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a schematic flow chart of an image reconstruction method in one embodiment;

[0065] Figure 2 is a schematic diagram of an image reconstruction method in one embodiment;

[0066] Figure 3 108 is a flow chart of step 108 in one embodiment;

[0067] Figure 4 102 is a flow chart of step 102 in one embodiment;

[0068] Figure 5 304 is a flowchart of an embodiment;

[0069] Figure 6 104 is a flow chart of step 104 in one embodiment;

[0070] Figure 7 is a schematic diagram of an image reconstruction method in one embodiment;

[0071] Figure 8 104 is a flow chart of step 104 in one embodiment;

[0072] Figure 9 is a schematic diagram of an image reconstruction method in one embodiment;

[0073] Figure 10 is a schematic diagram of an image reconstruction method in one embodiment;

[0074] Figure 11 is a schematic diagram of an image reconstruction method in one embodiment;

[0075] Figure 12 is a schematic diagram of an image reconstruction method in one embodiment;

[0076] Figure 13 is a structural block diagram of an image reconstruction device in one embodiment;

[0077] Figure 14 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0079] In one embodiment, Figure 1 As shown, an image reconstruction method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0080] Step 102: In the i-th round of image reconstruction, the image to be processed is segmented using a target image segmentation method to obtain a plurality of first image blocks. The target image segmentation method is used to indicate the size of the first image blocks and the image content overlap. The number and / or size of the first image blocks segmented using the target image segmentation method are different from the number and / or size of the image blocks segmented using the image segmentation method in the previous i-1 rounds of image reconstruction. There is overlap between adjacent first image blocks. The image content overlap is used to characterize the degree of overlap between adjacent first image blocks, where i is a positive integer.

[0081] In the embodiment of the present application, image reconstruction is performed in multiple rounds until the reconstructed target super-resolution image meets the requirements. In the i-th round of image reconstruction, the image to be processed can be segmented using a target image segmentation method to obtain multiple first image blocks. The image to be processed is the image to be super-resolution reconstructed, and the target image segmentation method is used to indicate the size of the first image blocks and the image content overlap. The image content overlap is used to represent the degree of overlap between adjacent first image blocks.

[0082] For example, the size of the first image block indicated by the target image segmentation method may be 40 pixels × 40 pixels, and the image content overlap may be 30%. That is, for any first image block, the size of the first image block is 40 pixels × 40 pixels, and the size of the overlapping portion between the first image block and any adjacent first image block should occupy 30% of the total size of the first image block. For example, taking the first image block at the upper left corner of the image to be processed as an example, the first image block is adjacent to the first image block to its right and the first image block below it. Then, the size of the overlapping portion between the first image block and the first image block to its right should occupy 30% of the total size of the first image block, that is, 12 pixels × 40 pixels; the size of the overlapping portion between the first image block and the first image block below it should also occupy 30% of the total size of the first image block, that is, 40 pixels × 12 pixels.

[0083] It should be noted that, see Figure 2As shown, if the segmentation of the image to be processed cannot be completed according to the size of the first image block indicated by the target image segmentation method and the image content overlap (for example, if the size of the first image block indicated by the target image segmentation method is 40 pixels × 40 pixels, the image content overlap is 30%, and the size of the image to be processed is 110 pixels × 60 pixels, then the image to be processed cannot be exactly divided into multiple first image blocks of 40 pixels × 40 pixels with an image content overlap of 30%), then the image to be processed can be segmented in any of the following ways: reducing the size of the image block located at the edge of the image to be processed, increasing the image content overlap between the image block located at the edge of the image to be processed and the image blocks adjacent to it, or simultaneously reducing the size of the image block located at the edge of the image to be processed and increasing the image content overlap between the image block located at the edge of the image to be processed and the image blocks adjacent to it, etc. The embodiments of the present application do not make specific limitations on this. For example, taking the above example, the size of the image block located at the right edge of the image to be processed can be reduced to 26 pixels × 40 pixels, or the size of the overlapping part between the image block located at the right edge of the image to be processed and the first image block to its left can be increased to 26 pixels × 40 pixels, that is, the image content overlap is increased to 65%, so as to complete the segmentation of the image to be processed.

[0084] The number and / or size of the first image blocks obtained by segmenting the target image using the target image segmentation method should be different from the number and / or size of the image blocks obtained by segmenting the target image using the target image segmentation method in the first through (i-1) image reconstruction rounds. For example, if i equals 3, and if in the first image reconstruction round the image blocks obtained by segmenting the target image are 32 pixels × 32 pixels and the number of image blocks obtained is 8, and in the second image reconstruction round the image blocks obtained by segmenting the target image are 40 pixels × 40 pixels and the number of image blocks obtained is 10, then in the third image reconstruction round the image blocks obtained by segmenting the target image cannot be 32 pixels × 32 pixels, or 8, or 40 pixels × 40 pixels, or 10.

[0085] It should be noted that the number of first image blocks obtained by using the target image segmentation method can be different from the number of image blocks obtained by using the image segmentation method in the previous i rounds of image reconstruction, the size of the first image blocks obtained by using the target image segmentation method can be different from the size of the image blocks obtained by using the image segmentation method in the previous i rounds of image reconstruction, or the number and size of the first image blocks obtained by using the target image segmentation method can be different from the number and size of the image blocks obtained by using the image segmentation method in the previous i rounds of image reconstruction. The embodiments of the present application do not make specific restrictions on this. Taking the above example where i is equal to 3, in the third round of image reconstruction, the size and number of image blocks obtained by segmenting the image to be processed can be 32 pixels × 32 pixels, 10 (that is, different from the number of image blocks segmented by the image segmentation method used in the first round of image reconstruction, and different from the size of the image blocks segmented by the image segmentation method used in the second round of image reconstruction), or 48 pixels × 48 pixels, 12 (that is, different from the number and size of image blocks segmented by the image segmentation method used in the first round of image reconstruction, and different from the number and size of image blocks segmented by the image segmentation method used in the second round of image reconstruction).

[0086] Step 104 : Reconstruct each first image block to obtain a reconstructed image block corresponding to each first image block, and merge each reconstructed image block to obtain a reconstructed image.

[0087] In the embodiment of the present application, each first image block can be reconstructed separately to obtain a reconstructed image block corresponding to each first image block, and each reconstructed image block can be merged to obtain a reconstructed image. The reconstructed image is a super-resolution image, and the pixel size of the reconstructed image can be set by those skilled in the art according to actual needs. Alternatively, the pixel size of the reconstructed image can be set without setting the pixel size of the reconstructed image, and the ratio between the size of the reconstructed image and the size of the image to be processed can be set, etc. This embodiment of the present application does not specifically limit this.

[0088] The embodiment of the present application does not specifically limit the method of performing reconstruction processing on each first image block to obtain a reconstructed image block corresponding to each first image block. Any method that can perform reconstruction processing on the first image block to obtain a reconstructed image block is applicable to the embodiment of the present application, for example: bicubic interpolation method, SRCNN (Super-Resolution Convolutional Neural Network) neural network, EDSR neural network, etc.

[0089] After reconstructing each first image block to obtain a reconstructed image block, the reconstructed image blocks can be merged to obtain a reconstructed image. The embodiment of the present application does not specifically limit the method for merging the reconstructed image blocks. Any method that can merge the reconstructed image blocks is applicable to the embodiment of the present application, for example: SURF (Speeded Up Robust Features, accelerated robust features) algorithm, splicing method based on FAST (Features from Accelerated Segment Test) corner detector, splicing method based on SIFT (Scale-Invariant Feature Transform) feature detector, etc.

[0090] Step 106 , downsampling the reconstructed image to obtain a downsampled image, and segmenting the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks. The size of the downsampled image is the same as the size of the image to be processed.

[0091] In an embodiment of the present application, the reconstructed image can be downsampled to obtain a downsampled image, and the downsampled image can be segmented according to the target image segmentation method to obtain multiple second image blocks. The size of the downsampled image is the same as the size of the image to be processed. That is, the downsampled image is an image with the same pixel size as the image to be processed. For example, if the pixel size of the image to be processed is 295 pixels × 413 pixels, the pixel size of the downsampled image should also be 295 pixels × 413 pixels. The embodiment of the present application does not specifically limit the method of downsampling. Any method that can downsample the reconstructed image to obtain a downsampled image is applicable to the embodiment of the present application.

[0092] After obtaining the downsampled image, the downsampled image can be segmented according to the target image segmentation method to obtain multiple second image blocks. That is, the number of second image blocks should be the same as the number of first image blocks, the size of the second image blocks should be the same as the size of the first image blocks, and the image content overlap of the second image blocks should also be the same as the image content overlap of the first image blocks, so that there is a one-to-one correspondence between the second image blocks and the first image blocks, which facilitates the calculation of the loss of the image reconstruction round.

[0093] Step 108 : Obtain a target super-resolution image based on the loss between the first image block and the second image block and the reconstructed image.

[0094] In embodiments of the present application, a target super-resolution image can be obtained based on the loss between the first image block and the second image block and the reconstructed image. For example, when the loss between the first image block and the second image block is less than a certain value, the reconstructed image can be directly used as the target super-resolution image, and when the loss between the first image block and the second image block is greater than or equal to the certain value, the next round of image reconstruction can be continued, etc. This embodiment of the present application does not specifically limit this.

[0095] The image reconstruction method provided in the embodiment of the present application is a method for image reconstruction that uses multiple rounds of image reconstruction, and in each round, a target image segmentation method that is different from the image segmentation method in all previous rounds is used to segment the image to be processed, thereby obtaining multiple overlapping first image blocks, and then reconstructing and splicing the first image blocks to obtain a reconstructed image, and downsampling the reconstructed image. The downsampled reconstructed image is segmented using the target image segmentation method to obtain multiple second image blocks, and then a target super-resolution image is obtained based on the loss between the first image block and the second image block and the reconstructed image. The embodiment of the present application divides the image to be processed into multiple overlapping first image blocks, so when the first image block is reconstructed into a reconstructed image block, the overlapping part is equivalent to being reconstructed twice or even more times. Therefore, when splicing the reconstructed image blocks, it is equivalent to fusing the results of multiple reconstructions, thereby achieving a better image reconstruction effect. The embodiment of the present application only requires one image to be processed, and the target super-resolution image can be obtained by iteratively reconstructing the image to be processed multiple times. Therefore, there is no need to provide a training set containing low-resolution and high-resolution image pairs, and there is no risk of overfitting due to training on the training set. The embodiments of the present application not only have a wide range of applications, but also can achieve good performance in various scenes and various types of images.

[0096] In one embodiment, Figure 3 As shown, in step 108, a target super-resolution image is obtained based on the loss between the first image block and the second image block and the reconstructed image, including:

[0097] Step 302: Determine the loss between the first image block and the second image block.

[0098] In an embodiment of the present application, the loss between the first image block and the second image block can be determined. The embodiment of the present application does not specifically limit the type of loss and the specific method of determining the loss, as long as the loss can characterize the difference between the first image block and the second image block. For example: the type of loss can be reconstruction accuracy loss, that is, the difference between each pixel of the first image block and each corresponding pixel of the second image block. In the case where the loss is reconstruction accuracy loss, the reconstruction accuracy loss can be determined by a loss function such as the L1 loss function (also known as MAE (Mean Absolute Error)) or the L2 loss function (also known as MSE (Mean Square Error)). The type of loss can also be image perception loss. In the case where the loss is image perception loss, the image perception loss can be determined by the image perception loss function. The embodiment of the present application does not specifically limit this.

[0099] Step 304: When the loss is less than the loss threshold, the reconstructed image is used as the target super-resolution image.

[0100] Step 306, or, when the loss is greater than or equal to the loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0101] In an embodiment of the present application, when the loss is less than the loss threshold, the reconstructed image can be used as the target super-resolution image, or when the loss is greater than or equal to the loss threshold, the downsampled image can be used as the new image to be processed, and the next round of image reconstruction can be performed until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in this round of image reconstruction is used as the target super-resolution image. The embodiment of the present application does not specifically limit the specific value of the loss threshold, and it can be selected by those skilled in the art according to actual needs. For example, when the image reconstruction accuracy is required to be high, the loss threshold can be set lower, and when the image reconstruction accuracy is not required to be high, the loss threshold can be set higher.

[0102] If the loss is less than the loss threshold, the reconstructed image obtained in the current round of image reconstruction can be considered to have met the requirements, and thus the reconstructed image obtained in the current round of image reconstruction can be used as the target super-resolution image. If the loss is greater than or equal to the loss threshold, the reconstructed image obtained in the current round of image reconstruction can be considered to have not yet met the requirements, and in this case the downsampled image obtained in the current round of image reconstruction can be used as the image to be processed in the next round of image reconstruction (i+1th round), and image reconstruction of the i+1th round can be continued, i.e., returning to step 102 and repeating steps 102 to 108.

[0103] If, in the i+1th round of image reconstruction, the loss between the first image block and the second image block is less than the loss threshold, the reconstructed image obtained in the i+1th round of image reconstruction can be used as the target super-resolution image; if the loss between the first image block and the second image block is greater than or equal to the loss threshold, the downsampled image obtained in the i+1th round of image reconstruction can be used as the image to be processed in the next round (i+2th round) of image reconstruction, and the i+2th round of image reconstruction is continued... and so on, until in a certain round of image reconstruction, the loss between the first image block and the second image block is less than the loss threshold, at which time the reconstructed image obtained in that round of image reconstruction can be used as the target super-resolution image.

[0104] The image reconstruction method provided in the embodiment of the present application can determine the loss between the first image block and the second image block, and when the loss is less than a loss threshold, use the reconstructed image obtained in the current round of image reconstruction as the target super-resolution image. When the loss is greater than or equal to the loss threshold, the next round of image reconstruction is entered. The embodiment of the present application evaluates the image reconstruction effect of the current round by loss, and only requires one image to be processed. By performing multiple iterative reconstructions on the image to be processed, the target super-resolution image can be obtained. Therefore, there is no need to provide a training set containing low-resolution and high-resolution image pairs, and there is no risk of overfitting due to training on the training set. Therefore, the scope of application of the image reconstruction method can be expanded.

[0105] In one embodiment, in step 102, in the i-th round of image reconstruction, before segmenting the image to be processed using the target image segmentation method, the method further includes:

[0106] When i is greater than 1, the size of the image block and / or the image content overlap corresponding to the image segmentation method used in the (i-1)th round of image reconstruction is adjusted to obtain the target image segmentation method.

[0107] In the embodiment of the present application, when i is greater than 1, the target image segmentation method can be obtained by adjusting the size of the image block and / or the image content overlap corresponding to the image segmentation method adopted in the i-1th round of image reconstruction. For example, if the size of the image block corresponding to the image segmentation method adopted in the i-1th round of image reconstruction is 32 pixels × 32 pixels and the image content overlap is 30%, then in the i-th round, the target image segmentation method can be obtained by adjusting the size of the image block to 48 pixels × 48 pixels, or by adjusting the image content overlap to 40%, or by adjusting the size of the image block to 48 pixels × 48 pixels and the image content overlap to 40%. The embodiment of the present application does not specifically limit this.

[0108] Since adjusting the image content overlap corresponding to the image segmentation method will cause the number of first image blocks obtained by segmenting using the image segmentation method to change, the image content overlap corresponding to the image segmentation method is adjusted. After obtaining the target image segmentation method, the number of first image blocks segmented using the target image segmentation method will be different from the number of image blocks segmented using the image segmentation method used in the previous i-1 rounds of image reconstruction.

[0109] The image reconstruction method provided in the embodiment of the present application can obtain a target image segmentation method by adjusting the size of the image blocks and / or the image content overlap corresponding to the image segmentation method. Therefore, the number and / or size of the first image blocks obtained by segmenting using the target image segmentation method will be different from the number and / or size of the image blocks obtained by segmenting using the image segmentation method used in the previous i-1 rounds of image reconstruction, thereby changing the effect of image reconstruction in the i-th round, and then determining whether to perform the next round of image reconstruction by evaluating whether the reconstructed image in the i-th round meets the requirements.

[0110] In one embodiment, Figure 4 As shown, in step 102, in the i-th round of image reconstruction, before segmenting the image to be processed using the target image segmentation method, the method further includes:

[0111] Step 402: when i is equal to 1, determine the complexity of the image to be processed.

[0112] In the embodiments of the present application, complexity is used to characterize the richness of information in the image to be processed. For example, the more objects in the image to be processed, the higher the complexity of the image to be processed; the higher the color richness of the image to be processed, the higher the complexity of the image to be processed. The embodiments of the present application do not specifically limit the method for determining the complexity of the image to be processed. Any method that can identify the richness of information in the image to be processed and use this to determine the complexity of the image to be processed is applicable to the embodiments of the present application.

[0113] Step 404 : determining the image content overlap between the image blocks according to the complexity of the image to be processed, wherein the image content overlap is positively correlated with the complexity of the image to be processed.

[0114] In an embodiment of the present application, the image content overlap between image blocks can be determined based on the complexity of the image to be processed. For example, a reference image can be set, and the image content overlap between the image blocks of the reference image can be set to 10%, and then the image content overlap between the image blocks of the image to be processed can be increased proportionally based on the ratio of the complexity of the image to be processed to the complexity of the reference image. It is also possible to set multiple image groups, determine to which image group the image to be processed belongs based on the complexity of the image to be processed, and then determine the image content overlap between the image blocks of the image to be processed based on the image content overlap corresponding to the image group. Since setting a higher image content overlap for an image to be processed with higher complexity can make the reconstruction effect of the image to be processed better, the image content overlap should be positively correlated with the complexity of the image to be processed.

[0115] Taking the example that the complexity of the image to be processed is determined by the number of objects contained in the image to be processed, for example, the complexity of the image to be processed containing more than or equal to 15 objects can be set to level one, the complexity of the image to be processed containing more than 5 and less than 15 objects can be set to level two, and the complexity of the image to be processed containing less than or equal to 5 objects can be set to level three, where the image to be processed with level one complexity corresponds to the first image group, the image to be processed with level two complexity corresponds to the second image group, and the image to be processed with level three complexity corresponds to the third image group. The image content overlap corresponding to the first image group is 60%, the image content overlap corresponding to the second image group is 40%, and the image content overlap corresponding to the third image group is 20%. If the complexity of the image to be processed is level three, it can be determined that the image to be processed belongs to the third image group, and further it can be determined that the image content overlap between the image blocks of the image to be processed should be 20%.

[0116] Step 406 : Obtain a target image segmentation method based on the image content overlap and the preset size of the image blocks.

[0117] In an embodiment of the present application, a target image segmentation method can be obtained based on the image content overlap and the preset size of the image block. The preset size can be a pre-set value, such as 32 pixels × 32 pixels, 40 pixels × 40 pixels, etc. The preset sizes corresponding to different images to be processed can be the same or different, and the embodiment of the present application does not specifically limit this. For example, if it is determined that the image content overlap is 20% and the preset size of the image block is 32 pixels × 32 pixels, then the size of the image block corresponding to the target image segmentation method is 32 pixels × 32 pixels, and the image content overlap is 20%.

[0118] The image reconstruction method provided in the embodiments of the present application can determine the image content overlap between image blocks corresponding to the image to be processed in the first round of image reconstruction based on the complexity of the image to be processed. The image content overlap should be positively correlated with the complexity of the image to be processed. The embodiments of the present application can set a higher image content overlap for more complex images to be processed, thereby improving the reconstruction effect of the image to be processed.

[0119] In one embodiment, Figure 5 As shown, in step 304, when the loss is less than the loss threshold, the reconstructed image is used as the target super-resolution image, including:

[0120] Step 502 : When the loss is less than a loss threshold, determine the image perception loss between the first image block and the second image block.

[0121] Step 504 : When the image perceptual loss is less than the perceptual loss threshold, the reconstructed image is used as the target super-resolution image.

[0122] Alternatively, in step 506, when the image perceptual loss is greater than or equal to the perceptual loss threshold, the downsampled image is used as a new image to be processed and the image reconstruction process is continued in the (i+1)th round until the perceptual loss between the first image block and the second image block is less than the perceptual loss threshold. The reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0123] In the embodiment of the present application, the loss is used to characterize the loss of reconstruction accuracy, and the image perception loss is used to characterize the difference between the features of the first image block and the features of the second image block. When the loss between the first image block and the second image block is less than the loss threshold, the image perception loss between the first image block and the second image block can be determined: when the image perception loss is less than the perception loss threshold, it can be considered that the reconstructed image obtained in the current round of image reconstruction has met the requirements, and therefore the reconstructed image obtained in the current round of image reconstruction can be used as the target super-resolution image. When the image perception loss is greater than or equal to the perception loss threshold, it can be considered that the reconstructed image obtained in the current round of image reconstruction does not meet the requirements. At this time, the downsampled image obtained in the current round of image reconstruction can be used as the image to be processed in the next round (i+1th round) of image reconstruction, and the i+1th round of image reconstruction can be continued. When the image perception loss is greater than or equal to the perception loss threshold, the method of continuing the i+1th round of image reconstruction can refer to the relevant description in the aforementioned embodiment, and the embodiment of the present application will not be repeated here.

[0124] The image reconstruction method provided in the embodiment of the present application can evaluate the effect of the current round of image reconstruction through loss once, and then perform a second evaluation of the effect of the current round of image reconstruction through image perception loss. Therefore, the target super-resolution image finally output can meet the requirements not only in terms of reconstruction accuracy, but also in terms of retaining the feature level of the original image to be processed, thereby improving the effect of image reconstruction.

[0125] In one embodiment, Figure 6 As shown, in step 104, each reconstructed image block is merged to obtain a reconstructed image, including:

[0126] Step 602: Determine the overlapping portion between the reconstructed image blocks according to the overlapping portion between adjacent first image blocks.

[0127] Step 604 : splicing the reconstructed image blocks according to the overlapping parts between the reconstructed image blocks to obtain a reconstructed image.

[0128] In the embodiment of the present application, the overlapping portion between the reconstructed image blocks can be determined based on the overlapping portion between adjacent first image blocks, and then the reconstructed image blocks can be spliced based on the overlapping portion between the reconstructed image blocks to obtain a reconstructed image. Figure 7As shown, since there are overlapping parts between adjacent first image blocks, after the reconstruction processing of each first image block, there will also be overlapping parts in the adjacent reconstructed image blocks. At this time, an image merging algorithm can be used to merge the overlapping parts in the adjacent reconstructed image blocks to splice the reconstructed image blocks to obtain a reconstructed image. The embodiment of the present application does not specifically limit the image merging algorithm. Any image merging algorithm that can merge adjacent reconstructed image blocks to display parts of the same area is applicable to the embodiment of the present application, such as: SURF algorithm, splicing method based on FAST corner detector, splicing method based on SIFT feature detector, etc.

[0129] The image reconstruction algorithm provided in the embodiments of the present application can determine the overlapping portion between each reconstructed image block based on the overlapping portion between adjacent first image blocks, and then stitch each reconstructed image block together based on the overlapping portion between the reconstructed image blocks to obtain a reconstructed image. By merging the overlapping portions of adjacent reconstructed image blocks, the embodiments of the present application can fuse the different reconstruction results for the overlapping portions in adjacent reconstructed image blocks. Therefore, the reconstructed image of the overlapping portion is equivalent to the combination of multiple reconstruction results, which can further improve the image reconstruction effect.

[0130] In one embodiment, Figure 8 As shown, in step 104, reconstruction processing is performed on each first image block to obtain a reconstructed image block corresponding to each first image block, including:

[0131] Step 802: Identify the image content in the first image block to obtain a content classification corresponding to the first image block.

[0132] Step 804: Determine an image reconstruction network corresponding to the first image block according to the content classification corresponding to the first image block.

[0133] Step 806: reconstruct the first image block using an image reconstruction network to obtain a reconstructed image block.

[0134] In an embodiment of the present application, the image content in each first image block can be identified to determine an image reconstruction network for reconstructing each first image block according to the content classification corresponding to each first image block. For example, the content classification may include: person classification, animal classification, scenery classification, etc. For any content classification, the image reconstruction network corresponding to the content classification can be used to reconstruct the first image block belonging to the content classification to obtain a reconstructed image block. For example, the image reconstruction network corresponding to the person classification can be an image reconstruction network trained on a person image training set, the image reconstruction network corresponding to the animal classification can be an image reconstruction network trained on an animal image training set, the image reconstruction network corresponding to the scenery classification can be an image reconstruction network trained on a scenery image training set, etc. The embodiment of the present application does not specifically limit this.

[0135] After determining the content classification corresponding to the first image block and then determining the image reconstruction network corresponding to the first image block based on the image reconstruction network corresponding to the content classification, the first image block can be reconstructed using the image reconstruction network to obtain a reconstructed image block.

[0136] For example, if the image to be processed includes both animals and people, after segmenting the image to be processed using the target image segmentation method, some first image blocks may include animals, while other first image blocks may include people. In this case, the content in each first image block can be identified to determine the content category corresponding to each first image block. For example, the content category corresponding to the first image block containing an animal can be the animal category, while the content category corresponding to the first image block containing a person can be the person category. During image reconstruction, the first image block containing the animal can be reconstructed using the image reconstruction network corresponding to the animal category, while the first image block containing the person can be reconstructed using the image reconstruction network corresponding to the person category, to obtain reconstructed image blocks.

[0137] The image reconstruction method provided in the embodiment of the present application can reconstruct each first image block according to the content classification of the first image block and adopt the corresponding image reconstruction network, thereby improving the reconstruction quality of the first image block of each content classification and further improving the image reconstruction effect.

[0138] In one embodiment, the ratio of the size of the reconstructed image block to the size of the first image block is the same as the ratio of the size of the target super-resolution image to the size of the image to be processed.

[0139] In an embodiment of the present application, the ratio of the size of the reconstructed image block to the size of the first image block should be the same as the ratio of the size of the target super-resolution image to the size of the image to be processed. For example, if the ratio of the size of the target super-resolution image to the size of the image to be processed is 4:1, then the ratio of the size of the reconstructed image block to the size of the first image block should also be 4:1. That is, if the size of the first image block is 32 pixels × 32 pixels, then the size of the reconstructed image block should be 128 pixels × 128 pixels, so that when the reconstructed image blocks are merged to obtain a reconstructed image, the ratio of the size of the obtained reconstructed image to the size of the image to be processed is also 4:1.

[0140] The image reconstruction method provided in the embodiment of the present application can make the ratio of the size of the reconstructed image block to the size of the first image block the same as the ratio of the size of the target super-resolution image to the size of the image to be processed. Therefore, when the reconstructed image blocks are merged to obtain a reconstructed image, the size of the obtained reconstructed image will also be the same as the size of the target super-resolution image.

[0141] In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below with reference to specific examples.

[0142] Reference Figure 9 , which shows a flow chart of an image reconstruction method.

[0143] In the present application, refer to Figure 10 As shown, in the i-th round of image reconstruction, the input image to be processed can be segmented using the target image segmentation method to obtain multiple first image blocks. Adjacent first image blocks should overlap. The method for obtaining the target image segmentation method and the method for segmenting the image to be processed using the target image segmentation method can be referred to the relevant description of the above embodiments, and will not be repeated in this embodiment of the present application.

[0144] After obtaining a plurality of first image blocks, referring to Figure 11 、 Figure 12 As shown, each first image block can be reconstructed separately to obtain multiple reconstructed image blocks. The ratio between the size of the reconstructed image block and the size of the first image block should be the same as the ratio between the size of the target super-resolution image and the size of the image to be processed. Because there is overlap between adjacent first image blocks, there is also overlap between adjacent reconstructed image blocks. Therefore, the overlapping portions of the reconstructed image blocks can be merged and spliced together to obtain a reconstructed image.

[0145] Furthermore, the reconstructed image may be downsampled to obtain a downsampled image, and the downsampled image may be segmented using a target image segmentation method to obtain a plurality of second image blocks to calculate the loss between the first image block and the second image block.

[0146] Exemplarily, the loss between the first image block and the second image block can be first calculated using the L1 loss function and the L2 loss function. A first loss threshold can be set for the L1 loss function, and a second loss threshold can be set for the L2 loss function. When the loss between the first image block and the second image block obtained by the L1 loss function is greater than or equal to the first loss threshold, and / or the loss between the first image block and the second image block obtained by the L2 loss function is greater than or equal to the second loss threshold, it can be considered that the reconstructed image obtained in the current round of image reconstruction does not meet the requirements. At this time, the downsampled image obtained in the current round of image reconstruction can be used as the image to be processed in the next round (i+1th round) of image reconstruction, and the i+1th round of image reconstruction is continued.

[0147] When the loss between the first image block and the second image block obtained by the L1 loss function is less than the first loss threshold, and the loss between the first image block and the second image block obtained by the L2 loss function is less than the second loss threshold, it can be considered that the reconstructed image obtained in the current round of image reconstruction has met the requirements. At this time, the image perception loss between the first image block and the second image block can be further evaluated using the perceptual loss function. When the image perception loss between the first image block and the second image block is greater than or equal to the perceptual loss threshold, it can be considered that the reconstructed image obtained in the current round of image reconstruction has met the requirements. Therefore, the reconstructed image obtained in the current round of image reconstruction can be used as the target super-resolution image. When the image perception loss between the first image block and the second image block is less than the perceptual loss threshold, it can be considered that the reconstructed image obtained in the current round of image reconstruction does not meet the requirements. At this time, the downsampled image obtained in the current round of image reconstruction can be used as the image to be processed in the next round (i+1th round) of image reconstruction, and the i+1th round of image reconstruction can be continued.

[0148] The image reconstruction method provided in the embodiments of the present application uses only the input image to be processed and, through multiple iterative reconstructions, reconstructs the image to be processed into a target super-resolution image. This method can be used to reconstruct blurred, noisy images. The embodiments of the present application do not require the use of a training set of low-resolution and high-resolution image pairs to train the algorithm, and do not suffer from overfitting due to training on a single type of image. Therefore, the algorithm's fitting capability can be greatly improved, significantly expanding its scope of application.

[0149] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0150] Based on the same inventive concept, embodiments of the present application also provide an image reconstruction device for implementing the aforementioned image reconstruction method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the following image reconstruction device embodiments can be found in the above-described limitations on the image reconstruction method and will not be further elaborated here.

[0151] In one embodiment, Figure 13 As shown, an image reconstruction device is provided, including: a segmentation module 1302, a reconstruction module 1304, a downsampling module 1306, and an acquisition module 1308, wherein:

[0152] a segmentation module 1302 configured to segment the image to be processed using a target image segmentation method in an i-th round of image reconstruction to obtain a plurality of first image blocks, wherein the target image segmentation method is used to indicate the size and image content overlap of the first image blocks, the number and / or size of the first image blocks obtained by segmentation using the target image segmentation method being different from the number and / or size of the image blocks obtained by segmentation using the image segmentation method in the previous i-1 round of image reconstruction, and there is overlap between adjacent first image blocks. The image content overlap is used to indicate the degree of overlap between adjacent first image blocks, where i is a positive integer;

[0153] a reconstruction module 1304 configured to perform reconstruction processing on each of the first image blocks to obtain a reconstructed image block corresponding to each of the first image blocks, and to merge the reconstructed image blocks to obtain a reconstructed image;

[0154] a downsampling module 1306 configured to perform downsampling processing on the reconstructed image to obtain a downsampled image, and segment the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks, wherein the size of the downsampled image is the same as the size of the image to be processed;

[0155] The acquisition module 1308 is configured to obtain a target super-resolution image according to the loss between the first image block and the second image block and the reconstructed image.

[0156] The image reconstruction device provided by the embodiment of the present application, through multiple rounds of image reconstruction, and in each round, a target image segmentation method different from the image segmentation method in all previous rounds is used to segment the image to be processed, thereby obtaining multiple overlapping first image blocks, and then reconstructing and splicing the first image blocks to obtain a reconstructed image, and downsampling the reconstructed image, and then segmenting the downsampled reconstructed image by the target image segmentation method to obtain multiple second image blocks, and then obtaining a target super-resolution image based on the loss between the first image block and the second image block and the reconstructed image. The embodiment of the present application divides the image to be processed into multiple overlapping first image blocks, so when the first image block is reconstructed into a reconstructed image block, the overlapping part is equivalent to being reconstructed twice or even more times, so when splicing the reconstructed image blocks, it is equivalent to fusing the results of multiple reconstructions, thereby achieving a better image reconstruction effect. The embodiment of the present application only requires one image to be processed, and the target super-resolution image can be obtained by iteratively reconstructing the image to be processed multiple times. Therefore, there is no need to provide a training set containing low-resolution and high-resolution image pairs, and there is no risk of overfitting due to training on the training set. The embodiments of the present application not only have a wide range of applications, but also can achieve good performance in various scenes and various types of images.

[0157] In one embodiment, the acquisition module 1308 is further configured to:

[0158] determining a loss between the first image block and the second image block;

[0159] When the loss is less than a loss threshold, the reconstructed image is used as a target super-resolution image; or

[0160] When the loss is greater than or equal to the loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0161] In one embodiment, the apparatus further comprises:

[0162] The adjustment module is used to adjust the size of the image block and / or the image content overlap corresponding to the image segmentation method used in the (i-1)th round of image reconstruction when i is greater than 1, so as to obtain the target image segmentation method.

[0163] In one embodiment, the apparatus further comprises:

[0164] A first determining module, configured to determine the complexity of the image to be processed when i is equal to 1;

[0165] a second determining module, configured to determine an image content overlap between image blocks according to the complexity of the image to be processed, wherein the image content overlap is positively correlated with the complexity of the image to be processed;

[0166] The segmentation method acquisition module is used to obtain the target image segmentation method according to the image content overlap and the preset size of the image block.

[0167] In one embodiment, the acquisition module 1308 is further configured to:

[0168] determining an image perception loss between the first image block and the second image block when the loss is less than a loss threshold;

[0169] When the image perceptual loss is less than a perceptual loss threshold, using the reconstructed image as a target super-resolution image; or

[0170] When the image perceptual loss is greater than or equal to the perceptual loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the image perceptual loss between the first image block and the second image block is less than the perceptual loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

[0171] In one embodiment, the reconstruction module 1304 is further configured to:

[0172] determining an overlapping portion between the reconstructed image blocks according to overlapping portions between adjacent first image blocks;

[0173] The reconstructed image blocks are spliced together according to overlapping portions between the reconstructed image blocks to obtain a reconstructed image.

[0174] In one embodiment, the reconstruction module 1304 is further configured to:

[0175] Identifying image content in the first image block to obtain a content classification corresponding to the first image block;

[0176] determining an image reconstruction network corresponding to the first image block according to the content classification corresponding to the first image block;

[0177] The image reconstruction network is used to reconstruct the first image block to obtain a reconstructed image block.

[0178] In one embodiment, the ratio of the size of the reconstructed image block to the size of the first image block is the same as the ratio of the size of the target super-resolution image to the size of the image to be processed.

[0179] Each module in the above-mentioned image reconstruction device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0180] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 14 As shown. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements an image reconstruction method.

[0181] Those skilled in the art will understand that Figure 14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0182] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0183] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0184] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0185] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0186] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0187] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0188] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An image reconstruction method, characterized in that: The method comprises: In the i-th round of image reconstruction, a target image segmentation method is used to segment the image to be processed to obtain a plurality of first image blocks. The target image segmentation method is used to indicate the size and image content overlap of the first image blocks. The number and / or size of the first image blocks obtained by segmentation using the target image segmentation method are different from the number and / or size of the image blocks obtained by segmentation using the image segmentation method in the previous i-1 rounds of image reconstruction. There is overlap between adjacent first image blocks. The image content overlap is used to characterize the degree of overlap between adjacent first image blocks. Wherein, i is a positive integer. Performing reconstruction processing on each of the first image blocks to obtain a reconstructed image block corresponding to each of the first image blocks, and merging the reconstructed image blocks to obtain a reconstructed image; Downsampling the reconstructed image to obtain a downsampled image, and segmenting the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks, wherein the size of the downsampled image is the same as the size of the image to be processed; determining a loss between the first image block and the second image block; When the loss is less than a loss threshold, the reconstructed image is used as a target super-resolution image; or When the loss is greater than or equal to the loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

2. The method according to claim 1, characterized in that In the i-th round of image reconstruction, before segmenting the image to be processed using the target image segmentation method, the method further includes: When i is greater than 1, the size of the image block and / or the image content overlap corresponding to the image segmentation method used in the (i-1)th round of image reconstruction is adjusted to obtain the target image segmentation method.

3. The method according to claim 1, characterized in that In the i-th round of image reconstruction, before segmenting the image to be processed using the target image segmentation method, the method further includes: When i is equal to 1, determining the complexity of the image to be processed; determining, according to the complexity of the image to be processed, an image content overlap between image blocks, wherein the image content overlap is positively correlated with the complexity of the image to be processed; The target image segmentation method is obtained according to the image content overlap and the preset size of the image block.

4. The method according to claim 1, wherein When the loss is less than a loss threshold, using the reconstructed image as a target super-resolution image comprises: determining an image perception loss between the first image block and the second image block when the loss is less than a loss threshold; When the image perceptual loss is less than a perceptual loss threshold, using the reconstructed image as a target super-resolution image; or When the image perceptual loss is greater than or equal to the perceptual loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the image perceptual loss between the first image block and the second image block is less than the perceptual loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

5. The method according to claim 1, wherein The merging of the reconstructed image blocks to obtain a reconstructed image includes: determining an overlapping portion between the reconstructed image blocks according to overlapping portions between adjacent first image blocks; The reconstructed image blocks are spliced together according to overlapping portions between the reconstructed image blocks to obtain a reconstructed image.

6. The method according to claim 1, characterized in that The performing reconstruction processing on each of the first image blocks to obtain a reconstructed image block corresponding to each of the first image blocks includes: Identifying image content in the first image block to obtain a content classification corresponding to the first image block; determining an image reconstruction network corresponding to the first image block according to the content classification corresponding to the first image block; The image reconstruction network is used to reconstruct the first image block to obtain a reconstructed image block.

7. The method according to claim 6, characterized in that The ratio of the size of the reconstructed image block to the size of the first image block is the same as the ratio of the size of the target super-resolution image to the size of the image to be processed.

8. An image reconstruction device, characterized in that: The device comprises: a segmentation module configured to segment the image to be processed using a target image segmentation method in an i-th round of image reconstruction to obtain a plurality of first image blocks, where the target image segmentation method is used to indicate the size and image content overlap of the first image blocks, the number and / or size of the first image blocks obtained by segmentation using the target image segmentation method being different from the number and / or size of the image blocks obtained by segmentation using the image segmentation method in the previous i-1 rounds of image reconstruction, there being overlap between adjacent first image blocks, and the image content overlap being used to characterize the degree of overlap between adjacent first image blocks, where i is a positive integer; a reconstruction module, configured to perform reconstruction processing on each of the first image blocks respectively to obtain a reconstructed image block corresponding to each of the first image blocks, and to merge the reconstructed image blocks to obtain a reconstructed image; a downsampling module, configured to perform downsampling processing on the reconstructed image to obtain a downsampled image, and segment the downsampled image according to the target image segmentation method to obtain a plurality of second image blocks, wherein the size of the downsampled image is the same as the size of the image to be processed; Get modules for: determining a loss between the first image block and the second image block; When the loss is less than a loss threshold, the reconstructed image is used as a target super-resolution image; or When the loss is greater than or equal to the loss threshold, the downsampled image is used as a new image to be processed and enters the i+1th round of image reconstruction until the loss between the first image block and the second image block is less than the loss threshold, and the reconstructed image obtained in the current round of image reconstruction is used as the target super-resolution image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images