Image reconstruction method and device

Abstract

The embodiment of the present invention provides an image reconstruction method and device, which relates to the field of image processing technology. The above method includes: using different segmentation methods to segment the image to be processed to obtain different sub-image sets, for each sub-image set, using the image reconstruction processing method corresponding to the target segmentation method to perform image reconstruction processing on each sub-image contained in the sub-image set, and obtain the sub-image reconstruction result of each sub-image, for each sub-image set, merging the sub-image reconstruction results of each sub-image contained in the sub-image set to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method, and based on each preset attention weight, fusing each reconstructed image to obtain the final image reconstruction result. The application of the technical solution provided by the embodiment of the present invention can improve the clarity of image reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image reconstruction method and device. Background Art

[0002] In real life, images are compressed when they are saved, or the lens shakes when they are taken, which can cause the image to be blurred, lost, distorted, and so on. In this case, the image clarity is low. In order to improve the clarity of the image, image reconstruction technology can be used to process the image.

[0003] When image reconstruction technology is currently used to process images, the overall features of the image are first extracted, and the image is reconstructed based on a single overall feature. However, the clarity of the reconstructed image obtained based only on a single overall feature is often still low. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide an image reconstruction method and device to improve the clarity of the image obtained after image reconstruction. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides an image reconstruction method, the method comprising:

[0006] Different segmentation methods are used to segment the image to be processed to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed using one segmentation method;

[0007] For each sub-image set, image reconstruction processing is performed on each sub-image included in the sub-image set using an image reconstruction processing method corresponding to the target segmentation method to obtain a sub-image reconstruction result of each sub-image, wherein the target segmentation method is: a segmentation method used when obtaining the sub-images included in the sub-image set;

[0008] For each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set are merged to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method;

[0009] Based on each preset attention weight, each reconstructed image is fused to obtain the final image reconstruction result, wherein each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

[0010] In one embodiment of the present invention, the image reconstruction processing method corresponding to the target segmentation method is used to perform image reconstruction processing on each sub-image included in the sub-image set, including:

[0011] For each sub-image in the sub-image set, image reconstruction processing is performed on the sub-image in the following manner:

[0012] Divide the sub-image into channels to obtain different channel images;

[0013] Performing full connection processing corresponding to the target segmentation method on each channel image respectively to obtain a first result;

[0014] Fusing the first results to obtain a second result corresponding to the sub-image;

[0015] The second result is reconstructed in accordance with the target segmentation method to obtain a sub-image reconstruction result of the sub-image.

[0016] In one embodiment of the present invention, performing full connection processing corresponding to the target segmentation method on each channel image to obtain the first result includes:

[0017] Input each channel image into the pre-trained fully connected layer corresponding to the target segmentation method, and use the output result as the first result;

[0018] The step of performing reconstruction processing on the second result corresponding to the target segmentation mode to obtain a sub-image reconstruction result of the sub-image includes:

[0019] The second result is input into a pre-trained reconstruction model corresponding to the target segmentation method, and the output result is used as the sub-image reconstruction result of the sub-image.

[0020] In one embodiment of the present invention, for each segmentation method, the preset fully connected layer is trained in the following manner to obtain the fully connected layer corresponding to the segmentation method, and the preset model is trained to obtain the reconstruction model corresponding to the segmentation method:

[0021] The sample image is segmented by using the segmentation method to obtain sample sub-images;

[0022] Perform image reconstruction processing on each sample sub-image to obtain a sample comparison result;

[0023] Perform channel division on each sample sub-image respectively to obtain a sample channel image of each sample sub-image;

[0024] Input each sample channel image into the preset fully connected layer, and use the output result as the first sample result;

[0025] Input each sample channel image into the preset fully connected layer, and use the output result as the first sample result;

[0026] For each sample sub-image, each first sample result generated based on the sample sub-image is merged to obtain a second sample result corresponding to the sample sub-image;

[0027] For each sample sub-image, input the second sample result corresponding to the sample sub-image into the preset model, and use the output result as the sample reconstruction result of the sample sub-image;

[0028] Based on the sample comparison results and sample reconstruction results of each sample sub-image, the sub-loss corresponding to the segmentation method is calculated;

[0029] Calculate the total loss based on the sub-losses corresponding to each segmentation method;

[0030] Adjusting parameters of the preset fully connected layer based on the sub-loss, and adjusting parameters of the preset model based on the total loss;

[0031] Return to executing the step of segmenting the sample image using the segmentation method to obtain sample sub-images until the preset training termination condition is reached, determine the preset fully connected layer after parameter adjustment as the fully connected layer corresponding to the segmentation method, and determine the preset model after parameter adjustment as the reconstruction model corresponding to the segmentation method.

[0032] In one embodiment of the present invention, the total loss is calculated based on the sub-losses corresponding to each segmentation mode, including:

[0033] Based on each sub-loss, calculate the attention weight corresponding to each segmentation method;

[0034] Based on each sub-loss and each attention weight, the total loss is calculated.

[0035] In one embodiment of the present invention, adjusting the parameters of the preset fully connected layer based on the sub-loss includes:

[0036] Calculate an update coefficient based on the sub-loss corresponding to the segmentation method, wherein the update coefficient represents: an adjustment amplitude when adjusting the parameters of the preset fully connected layer;

[0037] Based on the update coefficient and the sub-loss, parameters of the preset fully connected layer are adjusted.

[0038] In one embodiment of the present invention, the image to be processed is segmented using different segmentation methods to obtain different sub-image sets, including:

[0039] Dividing the image to be processed according to different sizes to obtain different sub-image sets;

[0040] and / or

[0041] The image to be processed is segmented to obtain different sub-image sets based on whether the sub-images obtained by segmentation contain the same image area.

[0042] In a second aspect, an embodiment of the present invention provides an image reconstruction device, the device comprising:

[0043] An image segmentation module is used to segment the image to be processed by using different segmentation methods to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed by using one segmentation method;

[0044] An image reconstruction module is used to perform image reconstruction processing on each sub-image set respectively by using an image reconstruction processing method corresponding to a target segmentation method to obtain a sub-image reconstruction result of each sub-image, wherein the target segmentation method is a segmentation method used when obtaining the sub-images contained in the sub-image set;

[0045] An image merging module is used to merge, for each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method;

[0046] The image fusion module is used to fuse the reconstructed images based on the preset attention weights to obtain the final image reconstruction result, wherein each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

[0047] In one embodiment of the present invention, the image reconstruction module is used to perform image reconstruction processing on each sub-image set and each sub-image in the sub-image set through the following sub-modules:

[0048] A channel division submodule is used to divide the sub-image into channels to obtain different channel images;

[0049] A full connection processing submodule, used for performing full connection processing corresponding to the target segmentation mode on each channel image to obtain a first result;

[0050] A result fusion submodule, used for fusing the first results to obtain a second result corresponding to the sub-image;

[0051] The reconstruction processing submodule is used to perform reconstruction processing corresponding to the target segmentation method on the second result to obtain a sub-image reconstruction result of the sub-image.

[0052] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0053] Memory, used to store computer programs;

[0054] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0055] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0056] In a fifth aspect, an embodiment of the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect.

[0057] Beneficial effects of the embodiments of the present invention:

[0058] The image reconstruction method provided by the embodiment of the present invention adopts different segmentation methods to segment the image to be processed to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed using a segmentation method, for each sub-image set, image reconstruction processing is performed on each sub-image contained in the sub-image set to obtain a sub-image reconstruction result of each sub-image, wherein the image reconstruction processing methods for the sub-images contained in different sub-image sets are different, for each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set are merged to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method, and based on each preset attention weight, each reconstructed image is fused to obtain a final image reconstruction result.

[0059] As can be seen from the above, by segmenting the image to be processed in different segmentation methods, the small-sized sub-images obtained are local images of the image to be processed, so the sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed, and the obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained in the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0060] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0062] Figure 1 A schematic flow chart of a first image reconstruction method provided by an embodiment of the present invention;

[0063] Figure 2 A schematic flow chart of a second image reconstruction method provided by an embodiment of the present invention;

[0064] Figure 3 A flowchart of a model training method provided by an embodiment of the present invention;

[0065] Figure 4 A schematic structural diagram of a first image reconstruction device provided by an embodiment of the present invention;

[0066] Figure 5 A schematic structural diagram of a second image reconstruction device provided by an embodiment of the present invention;

[0067] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0069] When image reconstruction technology is currently used to process an image, the overall features of the image are first extracted, and the image is reconstructed based on a single overall feature. However, the clarity of the reconstructed image based only on a single overall feature is often still low. In order to improve the clarity of the image obtained after image reconstruction, an embodiment of the present invention provides an image reconstruction method and device.

[0070] An embodiment of the present invention provides an image reconstruction method, the method comprising:

[0071] Different segmentation methods are used to segment the image to be processed to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed using one segmentation method;

[0072] For each sub-image set, image reconstruction processing is performed on each sub-image included in the sub-image set using an image reconstruction processing method corresponding to the target segmentation method to obtain a sub-image reconstruction result of each sub-image, wherein the target segmentation method is: a segmentation method used when obtaining the sub-images included in the sub-image set;

[0073] For each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set are merged to obtain a reconstructed image with the same size as the above-mentioned image to be processed and corresponding to the target segmentation method;

[0074] Based on each preset attention weight, each reconstructed image is fused to obtain the final image reconstruction result, wherein each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

[0075] As can be seen from the above, by segmenting the image to be processed in different segmentation methods, the small-sized sub-images obtained are local images of the image to be processed, so the sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed, and the obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained in the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0076] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity.

[0077] See also Figure 1 , is a flow chart of a first image reconstruction method provided in an embodiment of the present invention, the method includes the following steps S101-S104.

[0078] Step S101: Segment the image to be processed using different segmentation methods to obtain different sub-image sets.

[0079] Each sub-image set includes: all sub-images obtained by segmenting the image to be processed using a segmentation method.

[0080] In one embodiment of the present invention, the image to be processed may be segmented through the following step A or step B to obtain different sub-image sets.

[0081] Step A: Segment the above-mentioned image to be processed according to different sizes to obtain different sub-image sets.

[0082] Specifically, the above size may be 3 pixels×3 pixels, 5 pixels×5 pixels, 8 pixels×8 pixels, etc.

[0083] Step B: Segment the image to be processed based on whether the segmented sub-images contain the same image area to obtain different sub-image sets.

[0084] Specifically, two adjacent sub-images obtained by segmentation may include the same image area, and two adjacent sub-images obtained by segmentation may also include the same image area. The same image area may be referred to as an overlapping area, and a segmentation method in which the sub-images obtained by segmentation include overlapping areas may be referred to as an overlapping segmentation method, and a segmentation method in which the sub-images obtained by segmentation include overlapping areas may be referred to as a non-overlapping segmentation method.

[0085] In addition, the width of the overlapping area contained in the sub-image is smaller than the width of the sub-image. For example, if the size of the sub-image is 3 pixels × 3 pixels, the width of the above-mentioned same image area is 1 pixel. If the size of the sub-image is 5 pixels × 5 pixels, the width of the above-mentioned same image area is 2 pixels, and so on.

[0086] Furthermore, the width of the same image region can be preset, and the image to be processed can be segmented based on the width of the same image region to obtain different sub-image sets.

[0087] In one embodiment of the present invention, different sub-image sets can be obtained only through step A, that is, the sub-images contained in the obtained different sub-image sets have different sizes, and the sub-images in each sub-image set may contain overlapping areas, or may not contain overlapping areas.

[0088] In addition, different sub-image sets may be obtained only through step B, that is, the sub-images contained in the different sub-image sets obtained have the same size, some sub-image sets are obtained based on overlapping segmentation, and other sub-image sets are obtained based on non-overlapping segmentation.

[0089] In another embodiment of the present invention, step A can be combined with step B. For each size, the image to be processed can be segmented by overlapping segmentation and non-overlapping segmentation to obtain different sub-image sets, and the sizes of the sub-images contained in the obtained sub-image sets are all of the sizes. For example, according to the size of 3 pixels × 3 pixels, the image to be processed is segmented by overlapping segmentation and non-overlapping segmentation to obtain two sub-image sets, one of which contains overlapping areas in the sub-images, and the other does not contain overlapping areas in the sub-images. Then, the image to be processed is segmented by 5 pixels × 5 pixels to obtain two sub-image sets, one of which contains overlapping areas in the sub-images, and the other does not contain overlapping areas in the sub-images, and finally 4 different sub-image sets can be obtained.

[0090] In addition, when the image to be processed is segmented, if the sub-image obtained by segmentation and located at the edge of the image to be processed is incomplete, that is, the image to be processed cannot meet the requirements of complete segmentation, then the image to be processed can be expanded, that is, the image to be processed is filled with blank pixels so that the image to be processed can meet the requirements of complete segmentation.

[0091] Step S102: for each sub-image set, image reconstruction processing is performed on each sub-image included in the sub-image set using an image reconstruction processing method corresponding to the target segmentation method to obtain a sub-image reconstruction result of each sub-image.

[0092] The target segmentation method is a segmentation method used when obtaining the sub-images included in the sub-image set.

[0093] Specifically, different sub-image sets are obtained by segmenting the image to be processed using different segmentation methods. The sub-images in different sub-image sets contain different local features in the image to be processed. In order to better process different local features, different image reconstruction processing methods are used in the embodiments of the present invention to perform image reconstruction processing on each sub-image contained in different sub-image sets, so as to obtain different sub-image reconstruction results.

[0094] Specifically, for each sub-image set, the image reconstruction processing method corresponding to the target segmentation method is used to perform image reconstruction processing on each sub-image contained in the sub-image set, which can be achieved by the following steps S102A-102D, which will not be described in detail here. In addition, the image reconstruction processing process can also be completed by any method in the prior art, which will not be repeated here.

[0095] Step S103: for each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set are merged to obtain a reconstructed image with the same size as the above-mentioned image to be processed and corresponding to the target segmentation method.

[0096] Specifically, since the same sub-image set includes all sub-images obtained by segmenting the same image to be processed using the same segmentation method, all sub-images included in the same sub-image set can be merged to obtain the original image to be processed. Performing image reconstruction processing on each sub-image included in the same sub-image set is equivalent to performing image reconstruction processing on different regions in the image to be processed respectively, and merging the sub-image reconstruction results of each sub-image is equivalent to merging the image reconstruction results of different regions in the image to be processed, and the obtained result is equivalent to the result obtained by performing image reconstruction processing on the image to be processed.

[0097] In one embodiment of the present invention, the sub-image reconstruction results may be merged according to the positions of the sub-images in the image to be processed, that is, the position of the sub-image reconstruction results in the reconstructed image is the same as the position of the original sub-image in the image to be processed.

[0098] Step S104: Based on the preset attention weights, the reconstructed images are fused to obtain the final image reconstruction result.

[0099] Among them, after processing the sub-images contained in different sub-image sets respectively, the reconstructed images obtained are all of the same size as the image to be processed, so the various reconstructed images can be fused. And although the sub-images contained in different sub-image sets are different, and the image reconstruction processing methods for the sub-images contained in different sub-image sets are different, since all sub-images come from the image to be processed, the obtained reconstructed images can be regarded as an image reconstruction result of the image to be processed, and the result obtained after fusing the various reconstructed images can be used as the final image reconstruction result of the image to be processed.

[0100] In addition, each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

[0101] Based on the preset attention weights, each reconstructed image is fused to obtain the final image reconstruction result, which can be seen in the following formula:

[0102] M=m1×n1+m2×n2

[0103] Among them, M represents the final image reconstruction result, m1 and m2 represent the reconstructed images after different segmentation methods, and n1 and n2 represent the attention weights corresponding to different segmentation methods.

[0104] Furthermore, the above attention weights can be set manually or calculated based on step E below, which will not be described in detail here.

[0105] As can be seen from the above, by segmenting the image to be processed in different segmentation methods, the small-sized sub-images obtained are local images of the image to be processed, so the sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed, and the obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained in the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0106] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity.

[0107] See also Figure 2 , is a flow chart of a second image reconstruction method provided by an embodiment of the present invention, which is similar to the above Figure 1 Compared with the embodiment shown, for each sub-image set, for each sub-image in the sub-image set, the following steps S102A-S102D may be used to perform image reconstruction processing on the sub-image.

[0108] Step S102A: Divide the sub-image into channels to obtain different channel images.

[0109] Specifically, the sub-images may be divided into channels according to the three primary color channels to obtain an R channel image, a G channel image, and a B channel image, which correspond to red, green, and blue, respectively.

[0110] Step S102B: Perform full connection processing corresponding to the target segmentation method on each channel image to obtain a first result.

[0111] Specifically, the fully connected processing can be performed on each channel image in a manner known in the prior art.

[0112] In one embodiment of the present invention, the above step S102B can be implemented by the following step C.

[0113] Step C: Input each channel image into a pre-trained fully connected layer corresponding to the target segmentation method, and use the output result as the first result.

[0114] Specifically, different segmentation methods correspond to different parameters of the fully connected layer, and inputting different channel images into the fully connected layer with different parameters is equivalent to performing full connection processing on different channel images in different ways.

[0115] In addition, before performing full connection processing on the channel image, a tensor expansion calculation may be performed on the channel image, and a result obtained by the tensor expansion calculation may be fully connected to obtain a first result.

[0116] Taking the case where the size of the sub-image obtained by segmentation is 3 pixels × 3 pixels by using a non-overlapping segmentation method to segment the image to be processed as an example, for the G channel image X = (NHW / (3×3), G, 3, 3), after tensor expansion calculation, we can get Y = (NHW / (3×3), G×3×3), where X represents the set of G channel images, N represents the number of images to be processed, H represents the image height of the image to be processed, and W represents the image width of the image to be processed. Then NHW / (3×3) is the total number of G channel images, G represents that the channel image is a G channel image, and the last two parameters 3 represent that the height and width of the G channel image are both 3 pixels. Y represents the calculation result obtained after tensor expansion calculation of the G channel image, and G×3×3 represents the number of G channels in the second dimension of the calculation result Y.

[0117] Here, only the G channel image is taken as an example. The processing of the R channel image and the B channel image is similar to the above and will not be repeated here.

[0118] Specifically, since the channel image is originally two-dimensional data, the tensor obtained after converting the channel image into a tensor is one-dimensional data. Compared with directly processing the two-dimensional channel image, the processing efficiency of fully connecting the one-dimensional tensor is faster. And the first result obtained by fully connecting the one-dimensional tensor is also a one-dimensional tensor. Continuing to perform subsequent processing on the one-dimensional first result can also improve the processing efficiency of subsequent processing.

[0119] Step S102C: Fusing the first results to obtain a second result corresponding to the sub-image.

[0120] In one embodiment of the present invention, channel division of a sub-image can obtain multiple channel images, each channel image is a part of the sub-image, and multiple first results can be obtained after full connection processing is performed on the multiple channel images of the sub-image respectively. Then, the first results are fused, and the second result obtained is equivalent to the result obtained after the sub-image is fully connected. Each second result corresponds to a sub-image.

[0121] Step S102D: Perform reconstruction processing corresponding to the target segmentation method on the second result to obtain a sub-image reconstruction result of the sub-image.

[0122] In one embodiment of the present invention, the second result may be reconstructed using a method in the prior art.

[0123] Specifically, the above step S102D can be implemented by the following step D.

[0124] Step D: input the second result into a pre-trained reconstruction model corresponding to the target segmentation method, and use the output result as the sub-image reconstruction result of the sub-image.

[0125] Specifically, the above reconstruction model can be any model used for image reconstruction in the prior art.

[0126] The reconstruction model can be a basic module in an MLP (Multilayer Perceptron) model, and the second result is input into the trained basic module, and the output result of the basic module is used as the sub-image reconstruction result. The basic module includes three parts: a fully connected layer, an activation function, and a BN (Batch Normalization) process. Specifically, the second result input into the basic module can be first input into the fully connected layer to obtain a first processing result, and the processing result is processed using an activation function to obtain a second processing result, and then the second processing result is subjected to BN processing to obtain a processing result of the basic module as the sub-image reconstruction result.

[0127] Among them, the above-mentioned fully connected layer, activation function and BN processing are commonly used methods in the prior art and will not be repeated here.

[0128] It can be seen from the above that in the process of reconstructing the sub-image, the segmented sub-image is first divided into channels, and each channel image is extracted through channel division. Subsequently, each single channel image is processed to eliminate the mutual influence caused by overlap between different channels, so that the results of subsequent image reconstruction are more accurate.

[0129] See also Figure 3 , is a flow chart of a model training method provided by an embodiment of the present invention. For each segmentation method, the preset fully connected layer is trained in the following manner to obtain the fully connected layer corresponding to the segmentation method shown above, and the preset model is trained to obtain the reconstruction model corresponding to the segmentation method shown above. The above method includes the following steps S301-S309

[0130] Step S301: Segment the sample image using the segmentation method to obtain sample sub-images.

[0131] The above segmentation method is consistent with the segmentation method of the image to be processed in step S101 above, and will not be described in detail here.

[0132] Step S302: Perform image reconstruction processing on each sample sub-image to obtain a sample comparison result.

[0133] Specifically, the above sample comparison result is the expected value of the image reconstruction processing result in the embodiment of the present invention.

[0134] The above-mentioned image reconstruction processing for each sample sub-image may be inputting the sample sub-image into an MLP model. The MLP model here is trained, and the result output by the MLP model can be determined to be an accurate result, and the output result can be used as an expected value of image reconstruction.

[0135] Step S303: performing channel division on each sample sub-image respectively to obtain a sample channel image of each sample sub-image.

[0136] Specifically, the above step S303 is similar to the above step S102A, and will not be repeated here.

[0137] Step S304: input each sample channel image into the above-mentioned preset fully connected layer, and use the output result as the first sample result.

[0138] Specifically, before the sample image channel is input into the preset fully connected layer, the sample image can be subjected to tensor expansion calculation to obtain a one-dimensional calculation result, which is then input into the fully connected layer. The output result is also a one-dimensional first sample result. The one-dimensional first sample result is subjected to subsequent image reconstruction processing, and the obtained sample reconstruction result is also one-dimensional data. Therefore, in order to ensure the consistency of the data formats of the two when comparing the sample reconstruction result with the sample comparison result, the sample comparison result can also be subjected to tensor expansion calculation to obtain one-dimensional data for calculating the loss.

[0139] In addition, the above tensor expansion calculation method is similar to the aforementioned step S102B, which will not be repeated here.

[0140] Step S305: for each sample sub-image, each first sample result generated based on the sample sub-image is merged to obtain a second sample result corresponding to the sample sub-image.

[0141] Step S306: for each sample sub-image, input the second sample result corresponding to the sample sub-image into the preset model, and use the output result as the sample reconstruction result of the sample sub-image.

[0142] Specifically, the above steps S305-S306 are similar to the above steps S102A-S102B, and will not be repeated here.

[0143] Step S307: Based on the sample comparison results and sample reconstruction results of each sample sub-image, the sub-loss corresponding to the segmentation method is calculated.

[0144] Specifically, if the sample comparison result and the sample reconstruction result are both one-dimensional tensors, the sub-loss can be calculated by using the method of calculating the loss of one-dimensional data in the prior art. If the sample comparison result and the sample reconstruction result are both two-dimensional images, the sub-loss can be calculated by using the method of calculating the loss of two-dimensional images in the prior art. The embodiment of the present invention does not limit the method of calculating the sub-loss.

[0145] Step S308: Calculate the total loss based on the sub-losses corresponding to each segmentation method.

[0146] Specifically, the above total loss can be obtained by summing up the sub-losses, or the total loss can be calculated based on the following formula:

[0147] K=∑α i L i

[0148] Among them, i is the number of the segmentation method, K represents the total loss, α i represents the attention weight corresponding to the i-th segmentation method, L i It shows the sub-loss corresponding to the i-th segmentation method.

[0149] Step S309: adjusting the parameters of the preset fully connected layer based on the sub-loss, and adjusting the parameters of the preset model based on the total loss.

[0150] Specifically, in the embodiments of the present invention, the parameters of the preset fully connected layer and the parameters of the preset model can be adjusted based on the algorithm in the prior art.

[0151] The above algorithm may be an Adam (Adaptive Momentum Estimation) algorithm.

[0152] If the current model training process does not reach the preset training termination condition, the process returns to step S301 and uses the new image as a sample image to continue the model training process.

[0153] If the current model training process reaches the preset training termination condition, the preset fully connected layer after parameter adjustment is determined as the fully connected layer corresponding to the segmentation method, and the preset model after parameter adjustment is determined as the reconstruction model corresponding to the segmentation method.

[0154] Specifically, the preset convergence condition may be that the total loss is lower than a preset loss, or the number of times steps S301 - S309 are executed in a loop reaches a preset number.

[0155] From the above, it can be seen that in the embodiment of the present invention, different segmentation methods are used to segment the sample image respectively. After obtaining different sample sub-images, the preset fully connected layer and the preset model corresponding to the segmentation method are trained using the sample sub-images, so that the trained fully connected layer and the reconstruction model match the segmentation method, and the best processing effect can be obtained when the sub-image segmented by the segmentation method is subsequently processed. In addition, in the embodiment of the present invention, the total loss is obtained by uniformly calculating the sub-losses corresponding to different segmentation methods, and then the preset model is adjusted based on the total loss until the preset training termination condition is reached. It can ensure that the preset fully connected layers and preset models corresponding to different segmentation methods achieve good training results, so that better processing results can be obtained when the image to be processed is reconstructed based on the fully connected layers and reconstruction models corresponding to each segmentation method.

[0156] In one embodiment of the present invention, the above step S308 can be implemented by the following steps E-F.

[0157] Step E: Based on each sub-loss, calculate the attention weight corresponding to each segmentation method.

[0158] Specifically, the larger the sub-loss corresponding to the segmentation method, the less accurate the sample reconstruction result currently obtained by the preset fully connected layer and the preset model corresponding to the segmentation method. If the sample reconstruction results corresponding to each segmentation method are fused, the sample reconstruction result obtained by the segmentation method has the greatest impact on the fusion result. Therefore, when training the model, you can focus on the preset model and the preset fully connected layer corresponding to the segmentation method. That is, the larger the sub-loss, the greater the attention weight of the corresponding segmentation method.

[0159] In one embodiment of the present invention, the calculation of the above attention weight can refer to the following formula:

[0160]

[0161] Among them, exp() represents the exponential function with the natural constant e as the base, α i represents the attention weight corresponding to the i-th segmentation method, k and i represent the number of the segmentation method, and L i represents the sub-loss of the i-th segmentation method, L K Represents the sub-loss of the k-th segmentation method.

[0162] Step F: Based on each sub-loss and each attention weight, calculate the total loss.

[0163] The calculation of total loss is as mentioned above and will not be repeated here.

[0164] In addition, after each training, the sub-loss corresponding to each segmentation method changes, and the above-mentioned attention weight will also change accordingly. Therefore, the above-mentioned attention parameters can be calculated once during each training process. And the attention parameters obtained from the last training can be used as the attention parameters used for image reconstruction after the model is trained. Specifically, the method of image reconstruction based on the attention parameters can be the same as the previous step S104, which will not be repeated here.

[0165] In another embodiment of the present invention, the parameters of the preset fully connected layer can be adjusted through the following steps E-F.

[0166] Step E: Calculate the update coefficient based on the sub-loss corresponding to the segmentation method.

[0167] The update coefficient indicates the adjustment range when adjusting the parameters of the preset fully connected layer.

[0168] Specifically, the larger the sub-loss corresponding to the segmentation method, the greater the gap between the sample reconstruction result corresponding to the segmentation method and the sample control result, and the less accurate the sample reconstruction result. In order to obtain accurate sample reconstruction results as soon as possible, that is, to complete the model training process as soon as possible, the amplitude of adjusting the parameters of the preset fully connected layer corresponding to the segmentation method can be increased, so that the preset fully connected layer can achieve better training results as soon as possible.

[0169] In one embodiment of the present invention, the update coefficient may be calculated based on the following formula:

[0170]

[0171] Among them, β represents the update coefficient, i represents the number of the segmentation method, and L i Represents the sub-loss of the i-th segmentation method.

[0172] Step F: Based on the update coefficient and the sub-loss, adjust the parameters of the preset fully connected layer.

[0173] Specifically, the adjustment amplitude of the preset fully connected layer parameters can be calculated based on the above-mentioned sub-loss by the method in the prior art, and the calculated amplitude is multiplied by the above-mentioned update coefficient to obtain a calculation result, and the calculation result is used as the final adjustment amplitude of the preset fully connected layer parameters, and the parameters of the preset fully connected layer are adjusted based on the final adjustment amplitude.

[0174] Corresponding to the aforementioned image reconstruction method, an embodiment of the present invention further provides an image reconstruction device.

[0175] See also Figure 4 , is a schematic structural diagram of a first image reconstruction device provided by an embodiment of the present invention, the device comprising:

[0176] The image segmentation module 401 is used to segment the image to be processed by using different segmentation methods to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed by using one segmentation method;

[0177] The image reconstruction module 402 is used to perform image reconstruction processing on each sub-image set respectively by using an image reconstruction processing method corresponding to a target segmentation method to obtain a sub-image reconstruction result of each sub-image, wherein the target segmentation method is a segmentation method used when obtaining the sub-images contained in the sub-image set;

[0178] An image merging module 403, for each sub-image set, merges the sub-image reconstruction results of each sub-image contained in the sub-image set to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method;

[0179] The image fusion module 404 is used to fuse the reconstructed images based on the preset attention weights to obtain the final image reconstruction result, wherein each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

[0180] As can be seen from the above, by segmenting the image to be processed in different segmentation methods, the small-sized sub-images obtained are local images of the image to be processed, so the sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed, and the obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained in the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0181] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity.

[0182] See also Figure 5, is a schematic diagram of the structure of a second image reconstruction device provided by an embodiment of the present invention, wherein the image reconstruction module 402 is used to, for each sub-image set and for each sub-image in the sub-image set, perform image reconstruction processing on the sub-image through the following sub-modules:

[0183] The channel division submodule 402A is used to divide the sub-image into channels to obtain different channel images;

[0184] A full connection processing submodule 402B is used to perform full connection processing corresponding to the target segmentation method on each channel image to obtain a first result;

[0185] A result fusion submodule 402C, configured to fuse the first results to obtain a second result corresponding to the sub-image;

[0186] The reconstruction processing submodule 402D is used to perform reconstruction processing on the second result corresponding to the target segmentation method to obtain a sub-image reconstruction result of the sub-image.

[0187] It can be seen from the above that in the process of reconstructing the sub-image, the segmented sub-image is first divided into channels, and each channel image is extracted through channel division. Subsequently, each single channel image is processed to eliminate the mutual influence caused by overlap between different channels, so that the results of subsequent image reconstruction are more accurate.

[0188] In one embodiment of the present invention, the full connection processing submodule 402B is specifically used for:

[0189] Each channel image is input into a pre-trained fully connected layer corresponding to the target segmentation method, and the output result is taken as the first result.

[0190] In one embodiment of the present invention, the reconstruction processing submodule 402D is specifically used for:

[0191] The second result is input into a pre-trained reconstruction model corresponding to the target segmentation method, and the output result is used as the sub-image reconstruction result of the sub-image.

[0192] In one embodiment of the present invention, the device further includes the following modules. For each segmentation method, the preset fully connected layer is trained through the following modules to obtain the fully connected layer corresponding to the segmentation method, and the preset model is trained to obtain the reconstruction model corresponding to the segmentation method:

[0193] A sample segmentation module is used to segment the sample image using the segmentation method to obtain sample sub-images;

[0194] A sample reconstruction module is used to perform image reconstruction processing on each sample sub-image to obtain a sample comparison result;

[0195] An image division module is used to divide each sample sub-image into channels to obtain a sample channel image of each sample sub-image;

[0196] A fully connected module, used to input each sample channel image into the preset fully connected layer, and use the output result as the first sample result;

[0197] A result fusion module, used for fusing, for each sample sub-image, first sample results generated based on the sample sub-image to obtain a second sample result corresponding to the sample sub-image;

[0198] A sample reconstruction module, for each sample sub-image, inputting the second sample result corresponding to the sample sub-image into the preset model, and using the output result as the sample reconstruction result of the sample sub-image;

[0199] A sub-loss calculation module, used to calculate the sub-loss corresponding to the segmentation method based on the sample comparison results and sample reconstruction results of each sample sub-image;

[0200] The total loss calculation module is used to calculate the total loss based on the sub-losses corresponding to each segmentation method;

[0201] A parameter adjustment module is used to adjust the parameters of the preset fully connected layer based on the sub-loss, and adjust the parameters of the preset model based on the total loss, return to trigger the execution of the sample segmentation module until the preset training termination condition is reached, and determine the preset fully connected layer after the parameter adjustment as the fully connected layer corresponding to the segmentation method, and determine the preset model after the parameter adjustment as the reconstruction model corresponding to the segmentation method.

[0202] From the above, it can be seen that in the embodiment of the present invention, different segmentation methods are used to segment the sample image respectively. After obtaining different sample sub-images, the preset fully connected layer and the preset model corresponding to the segmentation method are trained using the sample sub-images, so that the trained fully connected layer and the reconstruction model match the segmentation method, and the best processing effect can be obtained when the sub-image segmented by the segmentation method is subsequently processed. In addition, in the embodiment of the present invention, the total loss is obtained by uniformly calculating the sub-losses corresponding to different segmentation methods, and then the preset model is adjusted based on the total loss until the preset training termination condition is reached. It can ensure that the preset fully connected layers and preset models corresponding to different segmentation methods achieve good training results, so that better processing results can be obtained when the image to be processed is reconstructed based on the fully connected layers and reconstruction models corresponding to each segmentation method.

[0203] In one embodiment of the present invention, the total loss calculation module is specifically used to:

[0204] Based on each sub-loss, calculate the attention weight corresponding to each segmentation method;

[0205] Based on each sub-loss and each attention weight, the total loss is calculated.

[0206] In one embodiment of the present invention, the parameter adjustment module is specifically used to:

[0207] Calculate an update coefficient based on the sub-loss corresponding to the segmentation method, wherein the update coefficient represents: an adjustment amplitude when adjusting the parameters of the preset fully connected layer;

[0208] Adjusting parameters of the preset fully connected layer based on the update coefficient and the sub-loss;

[0209] Adjust the parameters of the preset model based on the total loss, return to execute the sample segmentation module until the preset training termination condition is reached, determine the preset fully connected layer after the parameter adjustment as the fully connected layer corresponding to the segmentation method, and determine the preset model after the parameter adjustment as the reconstruction model corresponding to the segmentation method.

[0210] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0211] Memory 603, used for storing computer programs;

[0212] The processor 601 is used to implement any method step of the image reconstruction method when executing the program stored in the memory 603.

[0213] When the electronic device provided by the embodiment of the present invention is used to reconstruct an image, the image to be processed is segmented by different segmentation methods, and the obtained small-sized sub-images are local images of the image to be processed. Therefore, the above sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed. The obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained by the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0214] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity.

[0215] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0216] The communication interface is used for communication between the above electronic device and other devices.

[0217] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0218] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0219] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the image reconstruction methods are implemented.

[0220] When the computer program stored in the computer-readable storage medium provided by the present embodiment is applied to perform image reconstruction, the image to be processed is segmented by different segmentation methods, and the obtained small-sized sub-images are local images of the image to be processed. Therefore, the above sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed. The obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained in the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0221] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity.

[0222] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any one of the steps of the image reconstruction method in the above embodiments.

[0223] When executing the computer program product provided by this embodiment to reconstruct an image, the image to be processed is segmented by different segmentation methods, and the obtained small-sized sub-images are local images of the image to be processed. Therefore, the above sub-images contain local features of the image to be processed. Performing image reconstruction processing on each sub-image separately is equivalent to performing image reconstruction processing on each image area based on the local features of different image areas of the image to be processed. The obtained sub-image reconstruction results are reconstruction results based on the local features of the image to be processed. Therefore, the reconstructed image formed by merging the sub-image reconstruction results is an image generated based on a large number of local features of the image to be processed. Compared with the result obtained by reconstructing the image to be processed based on a single overall feature in the prior art, the reconstructed image obtained in the embodiment of the present invention has more reference features and the clarity of the reconstructed image is higher.

[0224] Moreover, in the embodiment of the present invention, the image to be processed is segmented based on different segmentation methods, and the local features contained in the sub-images in the obtained different sub-image sets are different, so the different reconstructed images generated based on the different sub-image sets can reflect different local features. The final image reconstruction result obtained by fusing the reconstructed images in the embodiment of the present invention is obtained based on a large number of different local features of the image to be processed, and the image reconstruction result has higher clarity.

[0225] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0226] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0227] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0228] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An image reconstruction method, characterized in that: The method comprises: Different segmentation methods are used to segment the image to be processed to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed using one segmentation method; For each sub-image set, image reconstruction processing is performed on each sub-image included in the sub-image set using an image reconstruction processing method corresponding to the target segmentation method to obtain a sub-image reconstruction result of each sub-image, wherein the target segmentation method is: a segmentation method used when obtaining the sub-images included in the sub-image set; For each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set are merged to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method; Based on each preset attention weight, each reconstructed image is fused to obtain the final image reconstruction result, wherein each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

2. The method according to claim 1, characterized in that The image reconstruction processing method corresponding to the target segmentation method is used to perform image reconstruction processing on each sub-image included in the sub-image set, including: For each sub-image in the sub-image set, image reconstruction processing is performed on the sub-image in the following manner: Divide the sub-image into channels to obtain different channel images; Performing full connection processing corresponding to the target segmentation method on each channel image respectively to obtain a first result; Fusing the first results to obtain a second result corresponding to the sub-image; The second result is reconstructed in accordance with the target segmentation method to obtain a sub-image reconstruction result of the sub-image.

3. The method according to claim 2, characterized in that The step of performing full connection processing corresponding to the target segmentation method on each channel image to obtain a first result includes: Input each channel image into the pre-trained fully connected layer corresponding to the target segmentation method, and use the output result as the first result; The step of performing reconstruction processing on the second result corresponding to the target segmentation mode to obtain a sub-image reconstruction result of the sub-image includes: The second result is input into a pre-trained reconstruction model corresponding to the target segmentation method, and the output result is used as the sub-image reconstruction result of the sub-image.

4. The method according to claim 3, characterized in that: For each segmentation method, the preset fully connected layer is trained in the following way to obtain the fully connected layer corresponding to the segmentation method, and the preset model is trained to obtain the reconstruction model corresponding to the segmentation method: The sample image is segmented by using the segmentation method to obtain sample sub-images; Perform image reconstruction processing on each sample sub-image to obtain a sample comparison result; Perform channel division on each sample sub-image respectively to obtain a sample channel image of each sample sub-image; Input each sample channel image into the preset fully connected layer, and use the output result as the first sample result; For each sample sub-image, each first sample result generated based on the sample sub-image is merged to obtain a second sample result corresponding to the sample sub-image; For each sample sub-image, input the second sample result corresponding to the sample sub-image into the preset model, and use the output result as the sample reconstruction result of the sample sub-image; Based on the sample comparison results and sample reconstruction results of each sample sub-image, the sub-loss corresponding to the segmentation method is calculated; Calculate the total loss based on the sub-losses corresponding to each segmentation method; Adjusting parameters of the preset fully connected layer based on the sub-loss, and adjusting parameters of the preset model based on the total loss; Return to executing the step of segmenting the sample image using the segmentation method to obtain sample sub-images until the preset training termination condition is reached, determine the preset fully connected layer after parameter adjustment as the fully connected layer corresponding to the segmentation method, and determine the preset model after parameter adjustment as the reconstruction model corresponding to the segmentation method.

5. The method according to claim 4, characterized in that The total loss is calculated based on the sub-losses corresponding to each segmentation method, including: Based on each sub-loss, calculate the attention weight corresponding to each segmentation method; Based on each sub-loss and each attention weight, the total loss is calculated.

6. The method according to claim 4, characterized in that The adjusting the parameters of the preset fully connected layer based on the sub-loss includes: Calculate an update coefficient based on the sub-loss corresponding to the segmentation method, wherein the update coefficient represents: an adjustment amplitude when adjusting the parameters of the preset fully connected layer; Based on the update coefficient and the sub-loss, parameters of the preset fully connected layer are adjusted.

7. The method according to any one of claims 1 to 6, characterized in that: The image to be processed is segmented by using different segmentation methods to obtain different sub-image sets, including: Dividing the image to be processed according to different sizes to obtain different sub-image sets; and / or The image to be processed is segmented to obtain different sub-image sets based on whether the sub-images obtained by segmentation contain the same image area.

8. An image reconstruction device, characterized in that: The device comprises: An image segmentation module is used to segment the image to be processed by using different segmentation methods to obtain different sub-image sets, wherein each sub-image set includes: all sub-images obtained by segmenting the image to be processed by using one segmentation method; An image reconstruction module is used to perform image reconstruction processing on each sub-image set respectively by using an image reconstruction processing method corresponding to a target segmentation method to obtain a sub-image reconstruction result of each sub-image, wherein the target segmentation method is a segmentation method used when obtaining the sub-images contained in the sub-image set; An image merging module is used to merge, for each sub-image set, the sub-image reconstruction results of each sub-image contained in the sub-image set to obtain a reconstructed image with the same size as the image to be processed and corresponding to the target segmentation method; The image fusion module is used to fuse the reconstructed images based on the preset attention weights to obtain the final image reconstruction result, wherein each attention weight corresponds to each segmentation method one by one, and each attention weight represents: the predicted clarity of the reconstructed image corresponding to the segmentation method corresponding to the attention weight.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images