Image restoration method and model, electronic equipment and storage medium

Abstract

The invention relates to an image restoration method and model, electronic equipment and a storage medium. The method comprises the following steps: randomly shielding input image data to obtain low-quality image data; training an image restoration model through the input image data and the low-quality image data to obtain a trained image restoration model; and inputting to-be-restored image data into the trained image restoration model to obtain restored image data corresponding to the to-be-restored image data. According to the scheme provided by the invention, the image restoration requirement of a high-precision map application scene can be met.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an image restoration method, model, electronic device and storage medium. Background Art

[0002] Image restoration (IR) is a study on restoring image degradation such as blur, noise, distortion, etc. through computer software and algorithms based on mathematical theory and computer technology.

[0003] Image restoration using filters, linear interpolation, etc. has poor image restoration effects and is difficult to use. With the development of deep learning, many image restoration technologies based on deep learning have emerged. For example: image restoration technology using feature enhancement and regularization term constraint angles, image restoration technology using potential features of the image to be restored, and image restoration technology using additional prior and external knowledge. However, image restoration technology based on deep learning is difficult to restore irregular missing images, resulting in missing restored images, large deformation problems, and low correlation between restored images and original images. It cannot meet the image restoration requirements of application scenarios such as high-precision maps. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related technology, the present application provides an image restoration method, model, electronic device and storage medium, which can meet the image restoration requirements of high-precision map application scenarios.

[0005] The first aspect of the present application provides an image restoration method, the method comprising: Randomly mask the input image data to obtain low-quality image data; Training an image restoration model using the input image data and the low-quality image data to obtain a trained image restoration model; The image data to be restored is input into a trained image restoration model to obtain restored image data corresponding to the image data to be restored.

[0006] Preferably, the randomly masking the input image data to obtain low-quality image data includes: Randomly grouping a data set consisting of input image data into training image data and test image data; Performing image data enhancement on the training image data to obtain enhanced training image data; The training image data and the enhanced training image data are randomly masked to obtain the low-quality image data.

[0007] Preferably, the randomly masking the training image data and the enhanced training image data to obtain the low-quality image data includes: According to a set number of random positions, the training image data and the enhanced training image data are randomly masked by using masking regions to obtain the low-quality image data corresponding to the training image data and the enhanced training image data; wherein, The occluded area occupies a preset proportion of the image of the training image data, or the occluded area occupies a preset proportion of the image of the enhanced training image data.

[0008] Preferably, the step of training the image restoration model using the input image data and the low-quality image data to obtain the trained image restoration model comprises: Construct an image restoration model based on the dynamic Transformer diffusion model; The image restoration model based on the dynamic Transformer diffusion model is trained by using the training image data, the test image data and the low-quality image data to obtain a trained image restoration model based on the dynamic Transformer diffusion model.

[0009] Preferably, the step of training the image restoration model based on the dynamic Transformer diffusion model by using the training image data, the test image data, and the low-quality image data to obtain the trained image restoration model based on the dynamic Transformer diffusion model comprises: Learning the low-quality image data through a compact image restoration prior extraction network to obtain an image restoration prior representation; The image restoration model based on the dynamic Transformer diffusion model is trained by using the training image data, the test image data, and according to the image restoration prior representation, so as to obtain a trained image restoration model based on the dynamic Transformer diffusion model.

[0010] Preferably, the step of training the image restoration model based on the dynamic Transformer diffusion model by using the training image data, the test image data, and the image restoration prior representation to obtain the trained image restoration model based on the dynamic Transformer diffusion model comprises: The image restoration model based on the dynamic Transformer diffusion model is trained by using the training image data and according to the image restoration prior representation, and when the loss function value of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set loss function value, the training of the image restoration model based on the dynamic Transformer diffusion model is terminated; The image restoration model based on the dynamic Transformer diffusion model that has completed training is tested using the test image data. When the restoration accuracy of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set restoration accuracy, a trained image restoration model based on the dynamic Transformer diffusion model is obtained.

[0011] A second aspect of the present application provides an image restoration model, the image restoration model comprising: A random occlusion module is used to randomly occlude input image data to obtain low-quality image data; A model training module, used to train an image restoration model using the input image data and the low-quality image data obtained by the random occlusion module to obtain a trained image restoration model; The image restoration module is used to input the image data to be restored into the image restoration model trained by the model training module to obtain restored image data corresponding to the image data to be restored.

[0012] Preferably, the image restoration model further includes: A data augmentation module is used to randomly group a data set consisting of input image data into training image data and test image data; and perform image data augmentation on the training image data to obtain enhanced training image data; The random masking module is further used to randomly mask the training image data and the enhanced training image data obtained by the data augmentation module to obtain the low-quality image data.

[0013] A third aspect of the present application provides an electronic device, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0015] The technical solution provided by this application may have the following beneficial effects: The technical solution of the present application obtains distorted low-quality image data through random occlusion, trains an image restoration model based on a dynamic Transformer diffusion model through the low-quality image data, fully learns the correlation between image pixel regions of image data containing road traffic signs, obtains an image restoration priori representation of the image restored by the image restoration model based on the dynamic Transformer diffusion model, and trains the image restoration model based on the dynamic Transformer diffusion model according to the image restoration priori representation, which can accelerate the convergence speed of the loss function value when training the image restoration model based on the dynamic Transformer diffusion model, and improve the training efficiency of the image restoration model based on the dynamic Transformer diffusion model; restores the image to be restored through the image restoration model based on the dynamic Transformer diffusion model, obtains the restored image corresponding to the image to be restored, and can restore the irregular missing of the image to be restored, solves the problems of missing restored images, large deformation problems, low correlation between restored images and original images, etc., can improve the image restoration effect in different scenarios, and can meet the image restoration requirements of high-precision map application scenarios.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0018] Figure 1 is a flowchart of an image restoration method shown in an embodiment of the present application; Figure 2 is another flowchart of the image restoration method shown in an embodiment of the present application; Figure 3 is an image schematic diagram of an image to be restored in the image restoration method shown in an embodiment of the present application; Figure 4 yes Figure 3 An image schematic diagram of the occluded area of ​​the image to be restored; Figure 5 yes Figure 3 An image schematic diagram of a restored image of the image to be restored; Figure 6 yes Figure 4 The occluded area and Figure 5 A schematic diagram of an image of an image of an effect image superimposed with a restored image; Figure 7is a structural schematic diagram of an image restoration model shown in an embodiment of the present application; Figure 8 is another structural schematic diagram of the image restoration model shown in the embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0020] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0022] The embodiment of the present application provides an image restoration method that can meet the image restoration requirements of high-precision map application scenarios.

[0023] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0024] Figure 1 It is a flowchart of the image restoration method shown in an embodiment of the present application.

[0025] See also Figure 1 , an image restoration method, comprising: Step 101: randomly mask the input image data to obtain low-quality image data.

[0026] In one embodiment, the input image data containing the road traffic signs may be randomly occluded, and pre-trained low-quality image data corresponding to the image data containing the road traffic signs in the missing area may be generated through random occlusion.

[0027] Step 102: Train the image restoration model by inputting image data and low-quality image data to obtain a trained image restoration model.

[0028] In one embodiment, input image data including road traffic signs and low-quality image data corresponding to the input image data can be input into the image restoration model to be trained; the loss function value of the image restoration model to be trained is calculated according to the output of the image restoration model to be trained based on the input image data including road traffic signs and the low-quality image data corresponding to the input image data; when the loss function value of the image restoration model to be trained converges to the set loss function value, the training of the image restoration model to be trained is terminated to obtain a trained image restoration model.

[0029] Step 103: input the image data to be restored into the trained image restoration model to obtain restored image data corresponding to the image data to be restored.

[0030] In one embodiment, the image data to be restored containing road traffic signs can be input into a trained image restoration model; the trained image restoration model restores the input image data to be restored containing road traffic signs, and outputs restored image data corresponding to the image data to be restored containing road traffic signs.

[0031] The image restoration method of the embodiment of the present application obtains distorted low-quality image data through random occlusion, trains an image restoration model through the low-quality image data to fully learn the correlation between image pixel areas of image data containing road traffic signs, obtains an image restoration prior representation of the image restored by the image restoration model, and obtains a trained image restoration model. The trained image restoration model can be used to restore irregular missing parts of the image to be restored, solves the problems of missing and large deformation of the restored image, and low correlation between the restored image and the original image, and can improve the effect of image restoration in different scenarios, and can meet the image restoration requirements of high-precision map application scenarios.

[0032] Figure 2 It is another flowchart of the image restoration method shown in an embodiment of the present application. Figure 2 Relative to Figure 1 The technical solution of the present application is described in more detail.

[0033] See also Figure 2 , an image restoration method, comprising: Step 201, screen the image data to obtain input image data.

[0034] In one embodiment, the image data may be screened to remove image data that does not contain road traffic signs, and obtain N frames of input image data that contain road traffic signs; wherein N is a positive integer greater than 1.

[0035] In one embodiment, road traffic signs may include, but are not limited to, various lines, arrows, texts, elevation marks, raised road signs, and contour marks drawn on the road surface.

[0036] Step 202: randomly group the data set consisting of the input image data into training image data and test image data.

[0037] In one embodiment, N frames of input image data may be randomly grouped, and the input image data with a first ratio P1 may be grouped as a training image data set, and the input image data with a second ratio P2 may be grouped as a testing image data set.

[0038] For example, N frames of input image data containing road traffic signs may be randomly grouped, with 80% of the input image data containing road traffic signs being grouped as a training image data set and 20% of the input image data containing road traffic signs being grouped as a test image data set.

[0039] Step 203: perform image data enhancement on the training image data to obtain enhanced training image data.

[0040] In one embodiment, an image augmentation technique may be used to perform image data augmentation on each frame of training image data in a training image data set to obtain an enhanced training image data set corresponding to the training image data set, and each frame of enhanced training image data in the enhanced training image data set corresponds to each frame of training image data in the training image data set. Performing image data augmentation on the training image data set can increase the diversity of the training image data, prevent overfitting, and improve the generalization ability of the image restoration model.

[0041] In one embodiment, the image augmentation technology may include but is not limited to random illumination and random cropping.

[0042] Step 204: randomly mask the training image data and the enhanced training image data to obtain Low quality image data.

[0043] In one embodiment, the training image data and the enhanced training image data may be randomly masked using masking regions according to a set number of random positions to obtain low-quality image data corresponding to the training image data and the enhanced training image data.

[0044] In one embodiment, the occluded area occupies a preset proportion of the image of the training image data, or the occluded area occupies a preset proportion of the image of the enhanced training image data. The size of the occluded area may be 20%-40% of the image pixels of the training image data, or the size of the occluded area may be 20%-40% of the image pixels of the enhanced training image data.

[0045] In one embodiment, each frame of training image data in the training image data set and each frame of enhanced training image data in the enhanced training image data set can be randomly occluded using randomly positioned occlusion areas to generate a pre-trained low-quality image data set of missing areas, wherein each frame of low-quality image data in the low-quality image data set corresponds one-to-one to each frame of training image data in the training image data set and each frame of enhanced training image data in the enhanced training image data set.

[0046] In one embodiment, a set number of random positions can be set; according to the set number of random positions, each frame of training image data of the training image data set and each frame of enhanced training image data of the enhanced training image data set are randomly occluded using occlusion areas to obtain low-quality image data that correspond one-to-one to each frame of training image data of the training image data set and each frame of enhanced training image data of the enhanced training image data set, and form a low-quality image data set.

[0047] For example, according to the image pixel P of the training image data in the training image dataset X , or the image pixel P of the enhanced training image data in the enhanced training image dataset Z , set the size of the occluded area to 30% *P X , or 30% *P Z ; Use size 30% *P X , or 30% *P Z The occlusion area and the set number of random positions are randomly occluded for each frame of training image data of the training image data set and each frame of enhanced training image data of the enhanced training image data set, respectively, to obtain low-quality image data corresponding to each frame of training image data of the training image data set and each frame of enhanced training image data of the enhanced training image data set, and form a low-quality image data set.

[0048] Step 205 , training the image restoration model using the training image data, the test image data, and the low-quality image data to obtain a trained image restoration model.

[0049] In one embodiment, an image restoration model based on a dynamic Transformer diffusion model can be constructed; the image restoration model based on the dynamic Transformer diffusion model is trained by training image data, test image data, and low-quality image data to obtain a trained image restoration model based on the dynamic Transformer diffusion model.

[0050] In one embodiment, low-quality image data can be learned through a compact image restoration prior extraction network to obtain an image restoration prior representation; an image restoration model based on a dynamic Transformer diffusion model is trained through training image data, test image data, and according to the image restoration prior representation to obtain a trained image restoration model based on a dynamic Transformer diffusion model.

[0051] In one embodiment, the image restoration model based on the dynamic Transformer diffusion model can be trained by training image data and according to the image restoration prior representation. When the loss function value of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set loss function value, the training of the image restoration model based on the dynamic Transformer diffusion model is terminated. The image restoration model based on the dynamic Transformer diffusion model that has been trained is tested by testing image data. When the restoration accuracy of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set restoration accuracy, a trained image restoration model based on the dynamic Transformer diffusion model is obtained.

[0052] In one embodiment, each frame of training image data in the training image data set, each frame of enhanced training image data in the enhanced training image data set, each frame of test image data in the test image data set, and each frame of low-quality image data in the low-quality image data set can be preprocessed, and each frame of image data in the training image data set, the enhanced training image data set, the test image data set, and the low-quality image data set can be converted into a matrix of 1080×1920×3.

[0053] In one embodiment, an image restoration model based on a dynamic Transformer diffusion model (DM) can be constructed. The image restoration model based on the dynamic Transformer diffusion model (DM) can use residual blocks and linearly stacked CPEN (Compact IR Prior Extraction Network) modules and dynamic Transformer to learn features such as pixel-to-pixel correlation, pixel-to-region correlation, and region-to-region correlation of each frame of 1080×1920×3 image data of a low-quality image dataset to obtain an image restoration prior representation (IPR).

[0054] In one embodiment, the image restoration model based on the dynamic Transformer diffusion model can adopt a dynamic image restoration transformer (DIRformer), extract and aggregate multi-level features of 1080×1920×3 image data per frame in a low-quality image dataset through a Unet shape stacked transformer block, and learn features such as pixel-to-pixel correlation, pixel-to-region correlation, and region-to-region correlation of 1080×1920×3 image data per frame in the low-quality image dataset.

[0055] In one embodiment, an image restoration model based on a dynamic Transformer diffusion model may adopt a denoising network. Based on the features of each frame of 1080×1920×3 image data in a low-quality image data set, an occluded area of ​​each frame of 1080×1920×3 image data in the low-quality image data set is used as a noise point. All denoising iterations are run at a time step t, and each frame of 1080×1920×3 image data in the low-quality image data set is iteratively restored to obtain restored image data that corresponds one-to-one to each frame of 1080×1920×3 image data in the low-quality image data.

[0056] In one embodiment, based on the restored image data corresponding one-to-one to each 1080×1920×3 image data of each frame of the low-quality image data set, as well as the 1080×1920×3 training image data, 1080×1920×3 enhanced training image data, and 1080×1920×3 test image data corresponding one-to-one to each frame of image data in the low-quality image data set, a loss function value of an image restoration model based on a dynamic Transformer diffusion model for iteratively restoring each frame of image data in the low-quality image data set is calculated; the loss function value is back-propagated, and the network parameters of the image restoration model based on the dynamic Transformer diffusion model are updated so that the loss function value converges to the set loss function value, thereby terminating the training of the image restoration model based on the dynamic Transformer diffusion model.

[0057] In one embodiment, a test image data set with test image data of 1080×1920×3 per frame can be used to test the image restoration model based on the dynamic Transformer diffusion model that has completed training. When the restoration accuracy of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set restoration accuracy, a trained image restoration model based on the dynamic Transformer diffusion model is obtained.

[0058] In one embodiment, an image restoration model based on a dynamic Transformer diffusion model can screen input image data, eliminate image data that does not meet the specifications, and obtain input image data containing road traffic signs; randomly group the processed input image data into training image data and test image data, and save the path to a txt (text file); read the training image data and test image data from the txt, and parse the training image data and test image data into a 1080×1920×3 matrix to train the image restoration model to obtain a trained image restoration model; use the trained image restoration model to restore the image to be restored to obtain a restored image, compare the restored image with the real image corresponding to the image to be restored, and calculate the loss function value of the trained image restoration model; if the loss function value converges to the set loss function value, a trained image restoration model based on a dynamic Transformer diffusion model is obtained.

[0059] Step 206: input the image data to be restored into the trained image restoration model to obtain restored image data corresponding to the image data to be restored.

[0060] In one embodiment, the image data to be restored containing road traffic signs can be input into a trained image restoration model based on a dynamic Transformer diffusion model; the trained image restoration model based on a dynamic Transformer diffusion model restores the input image data to be restored containing road traffic signs, and outputs restored image data corresponding to the image data to be restored containing road traffic signs.

[0061] like Figure 3 According to the image to be restored, the following is obtained: Figure 4 The occluded area 401 (white area) of the image to be restored is shown; the trained image restoration model is based on Figure 4 The occluded area 401 shown in FIG. 1 is randomly occluded for the image to be restored, and an image restoration model is used to restore an image restoration priori representation of the image to be restored. The image to be restored is restored according to the image restoration priori representation to obtain the following: Figure 5 The restored image shown. Figure 4 The occluded area shown is Figure 5 The restored images shown in the figure are superimposed to obtain Figure 6 The effect image is shown.

[0062] The image restoration method of the embodiment of the present application obtains distorted low-quality image data through random occlusion, trains an image restoration model based on a dynamic Transformer diffusion model through the low-quality image data, so as to fully learn the correlation between image pixel regions of image data containing road traffic signs, obtain an image restoration priori representation of the image restored by the image restoration model based on the dynamic Transformer diffusion model, and trains the image restoration model based on the dynamic Transformer diffusion model according to the image restoration priori representation, so as to accelerate the convergence speed of the loss function value when training the image restoration model based on the dynamic Transformer diffusion model, and improve the training efficiency of the image restoration model based on the dynamic Transformer diffusion model; restores the image to be restored through the image restoration model based on the dynamic Transformer diffusion model, obtains the restored image corresponding to the image to be restored, and can restore the irregular missing of the image to be restored, solves the problems of missing restored images, large deformation problems, low correlation between restored images and original images, etc., can improve the image restoration effect in different scenarios, and can meet the image restoration requirements of high-precision map application scenarios.

[0063] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides an image restoration model, an electronic device and corresponding embodiments.

[0064] Figure 7 It is a structural schematic diagram of the image restoration model shown in the embodiment of the present application.

[0065] See also Figure 7 , an image restoration model, including a random occlusion module 701, a model training module 702, and an image restoration module 703.

[0066] The random masking module 701 is used to randomly mask the input image data to obtain low-quality image data.

[0067] In one embodiment, the random occlusion module 701 may randomly occlude the input image data containing road traffic signs, and generate pre-trained low-quality image data of the missing area corresponding to the input image data through random occlusion.

[0068] The model training module 702 is used to train the image restoration model by inputting image data and the low-quality image data obtained by the random occlusion module 701 to obtain a trained image restoration model.

[0069] In one embodiment, the model training module 702 can input input image data including road traffic signs and low-quality image data corresponding to the input image data obtained by the random occlusion module 701 into the image restoration model to be trained; according to the output of the image restoration model to be trained based on the input image data including road traffic signs and the low-quality image data corresponding to the input image data, the loss function value of the image restoration model to be trained is calculated; when the loss function value of the image restoration model to be trained converges to the set loss function value, the training of the image restoration model to be trained is terminated to obtain a trained image restoration model.

[0070] The image restoration module 703 is used to input the image data to be restored into the image restoration model trained by the model training module 702 to obtain restored image data corresponding to the image data to be restored.

[0071] In one embodiment, the image restoration module 703 can input the image data to be restored containing road traffic signs into the image restoration model trained by the model training module 702; the trained image restoration model restores the input image data to be restored containing road traffic signs, and outputs restored image data corresponding to the image data to be restored containing road traffic signs.

[0072] The technical solution of the embodiment of the present application obtains distorted low-quality image data through random occlusion, trains an image restoration model through the low-quality image data to fully learn the correlation between image pixel areas of image data containing road traffic signs, obtains an image restoration prior representation of the image restored by the image restoration model, and obtains a trained image restoration model. The trained image restoration model can restore irregular missing parts of the image to be restored, solves the problems of missing and large deformation of the restored image, and low correlation between the restored image and the original image, and can improve the effect of image restoration in different scenarios, and can meet the image restoration requirements of high-precision map application scenarios.

[0073] Figure 8 It is another structural schematic diagram of the image restoration model shown in the embodiment of the present application.

[0074] See also Figure 8 , an image restoration model includes a random occlusion module 701, a model training module 702, an image restoration module 703, and a data augmentation module 801.

[0075] The data augmentation module 801 is used to randomly group a data set consisting of input image data into training image data and test image data; and perform image data enhancement on the training image data to obtain enhanced training image data.

[0076] In one embodiment, the image restoration model can randomly group a data set consisting of input image data containing road traffic signs into training image data and test image data; the data augmentation module 801 performs image data enhancement on the training image data to obtain enhanced training image data.

[0077] In one embodiment, the random occlusion module 701 performs random occlusion on the training image data and the enhanced training image data to obtain low-quality image data.

[0078] In one embodiment, the model training module 702 constructs an image restoration model based on a dynamic Transformer diffusion model; the image restoration model based on the dynamic Transformer diffusion model is trained by training image data, test image data, and low-quality image data to obtain a trained image restoration model based on the dynamic Transformer diffusion model.

[0079] In one embodiment, the image restoration module 703 inputs the image data to be restored including the road traffic signs into the image restoration model based on the dynamic Transformer diffusion model trained by the model training module 702 to obtain restored image data corresponding to the image data to be restored.

[0080] Regarding the model in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0081] Fig. 9 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0082] See also Fig. 9 , the electronic device 1000 includes a memory 1010 and a processor 1020 .

[0083] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0084] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0085] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.

[0086] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0087] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0088] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image restoration method, characterized in that: include: Randomly mask the input image data to obtain low-quality image data; Training an image restoration model using the input image data and the low-quality image data to obtain a trained image restoration model; The image data to be restored is input into a trained image restoration model to obtain restored image data corresponding to the image data to be restored.

2. The method according to claim 1, characterized in that: The step of randomly blocking the input image data to obtain low-quality image data includes: Randomly grouping a data set consisting of input image data into training image data and test image data; Performing image data enhancement on the training image data to obtain enhanced training image data; The training image data and the enhanced training image data are randomly masked to obtain the low-quality image data.

3. The method according to claim 2, characterized in that The randomly masking the training image data and the enhanced training image data to obtain the low-quality image data includes: According to a set number of random positions, the training image data and the enhanced training image data are randomly masked by using masking regions to obtain the low-quality image data corresponding to the training image data and the enhanced training image data; wherein, The occluded area occupies a preset proportion of the image of the training image data, or the occluded area occupies a preset proportion of the image of the enhanced training image data.

4. The method according to claim 2, characterized in that: The step of training the image restoration model by using the input image data and the low-quality image data to obtain a trained image restoration model comprises: Construct an image restoration model based on the dynamic Transformer diffusion model; The image restoration model based on the dynamic Transformer diffusion model is trained by using the training image data, the test image data and the low-quality image data to obtain a trained image restoration model based on the dynamic Transformer diffusion model.

5. The method according to claim 4, characterized in that The step of training the image restoration model based on the dynamic Transformer diffusion model by using the training image data, the test image data, and the low-quality image data to obtain a trained image restoration model based on the dynamic Transformer diffusion model comprises: Learning the low-quality image data through a compact image restoration prior extraction network to obtain an image restoration prior representation; The image restoration model based on the dynamic Transformer diffusion model is trained by using the training image data, the test image data, and according to the image restoration prior representation, so as to obtain a trained image restoration model based on the dynamic Transformer diffusion model.

6. The method according to claim 5, characterized in that The step of training the image restoration model based on the dynamic Transformer diffusion model by using the training image data, the test image data, and the image restoration prior representation to obtain a trained image restoration model based on the dynamic Transformer diffusion model includes: The image restoration model based on the dynamic Transformer diffusion model is trained by using the training image data and according to the image restoration prior representation, and when the loss function value of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set loss function value, the training of the image restoration model based on the dynamic Transformer diffusion model is terminated; The image restoration model based on the dynamic Transformer diffusion model that has completed training is tested using the test image data. When the restoration accuracy of the image restoration model based on the dynamic Transformer diffusion model is less than or equal to the set restoration accuracy, a trained image restoration model based on the dynamic Transformer diffusion model is obtained.

7. An image restoration model, characterized in that: include: A random occlusion module is used to randomly occlude input image data to obtain low-quality image data; A model training module, used to train an image restoration model using the input image data and the low-quality image data obtained by the random occlusion module to obtain a trained image restoration model; The image restoration module is used to input the image data to be restored into the image restoration model trained by the model training module to obtain restored image data corresponding to the image data to be restored.

8. The image restoration model according to claim 7, characterized in that: The image restoration model also includes: A data augmentation module is used to randomly group a data set consisting of input image data into training image data and test image data; and perform image data augmentation on the training image data to obtain enhanced training image data; The random occlusion module is further used to randomly occlude the training image data and the enhanced training image data obtained by the data augmentation module to obtain the low-quality image data.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 6.

Images