Image restoration method and device, storage medium and electronic equipment

The image restoration method addresses accuracy issues in traditional methods by extracting edge and global features through neural networks and super-resolution reconstruction, achieving high-precision repairs of cultural relics.

CN119067902APending Publication Date: 2024-12-03INNER MONGOLIA UNIVERSITY
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202310623266.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Traditional image restoration methods, including super-resolution reconstruction technology and deep learning, suffer from accuracy issues due to reliance on prior knowledge and complexity, leading to incomplete feature extraction and low restoration quality, especially in cultural relic restoration where professional expertise is scarce.

Method used

An image restoration method that extracts edge and global feature information using grayscale processing, edge detection, and neural networks like ResNet and TransUNet, followed by super-resolution reconstruction with multi-scale residual attention layers to enhance feature fusion and reconstruction accuracy.

Benefits of technology

The method improves image restoration accuracy by comprehensively incorporating edge and global features, resulting in clearer, higher-resolution images with repaired defects, enhancing the efficiency and precision of cultural relic restoration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067902A_ABST
    Figure CN119067902A_ABST
Patent Text Reader

Abstract

The invention provides an image restoration method and device, a storage medium and electronic equipment. The image restoration method comprises the steps of obtaining a to-be-restored image; extracting edge feature information and global feature information of the to-be-restored image; determining a first image based on the edge feature information and the global feature information; super-resolution reconstruction is carried out on the first image to obtain a second image, and the second image is an image obtained by repairing the to-be-repaired image. According to the method, the first image is determined by utilizing the edge feature information and the global feature information of the to-be-restored image, the first image comprises relatively comprehensive features in the to-be-restored image, and then the first image is subjected to super-resolution reconstruction; according to the method and the device, the image to be repaired can be obtained through the first image and the second image, the second image is clear, high in resolution and capable of repairing abrasion / cracks / color contamination / small-part incompleteness and the like, the image to be repaired is repaired, and the image repairing accuracy is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image restoration method, device, storage medium and electronic device. Background Art

[0002] At present, the damage rate of cultural relics collections is relatively high due to various reasons, but there is a far shortage of professional staff engaged in cultural relics restoration. In addition, the traditional method based on manual restoration is highly dependent on personal experience and relevant professional knowledge, which can easily cause secondary damage. This makes cultural relics restoration work time-consuming and low-return.

[0003] Traditional restoration methods include super-resolution reconstruction technology and deep learning, but the restoration effect of traditional super-resolution reconstruction technology is limited by factors such as the restorer's prior knowledge and the complexity of the scene. That is, this restoration method cannot guarantee the accuracy of image restoration; and the super-resolution reconstruction method based on deep learning mainly improves performance by continuously deepening the network. Although it can extract some feature information, there is still omission of features, resulting in low accuracy of image restoration. Summary of the invention

[0004] In view of this, an object of the embodiments of the present application is to provide an image restoration method, device, storage medium and electronic device, which can improve the accuracy of image restoration.

[0005] In a first aspect, an embodiment of the present application provides an image restoration method, which includes:

[0006] Obtain the image to be repaired;

[0007] Extracting edge feature information and global feature information of the image to be repaired;

[0008] Determine a first image based on the edge feature information and the global feature information;

[0009] The first image is super-resolution reconstructed to obtain a second image, wherein the second image is an image obtained by completing restoration of the image to be restored.

[0010] In a possible implementation, extracting edge feature information and global feature information of the image to be restored includes:

[0011] Gray-scaling the image to be repaired to obtain a single-channel image;

[0012] Based on the single-channel image, the edge feature information is extracted, and based on the image to be repaired, the global feature information is extracted.

[0013] In a possible implementation, extracting the edge feature information based on the single-channel image includes:

[0014] Performing edge detection and masking on the single-channel image to obtain an edge mask image;

[0015] The edge feature information is obtained by performing feature extraction on the edge mask image through a codec.

[0016] In a possible implementation, extracting the global feature information based on the image to be repaired includes:

[0017] Adding a mask to the image to be repaired to obtain a global mask image;

[0018] A first algorithm is used to perform feature extraction on the global mask image to obtain the global feature information.

[0019] In a possible implementation manner, determining the first image based on the edge feature information and the global feature information includes:

[0020] Feature fusion is performed on the edge feature information and the global feature information to obtain the first image.

[0021] In a possible implementation manner, performing super-resolution reconstruction on the first image to obtain the second image includes:

[0022] Extracting low-level feature information and high-level feature information of the first image;

[0023] Fusing the low-level feature information with the high-level feature information to obtain a feature image;

[0024] Amplifying the characteristic image to obtain an amplified characteristic image;

[0025] The resolution of the enlarged feature image is reconstructed using the first algorithm to obtain the second image.

[0026] In a possible implementation manner, extracting low-level feature information and high-level feature information of the first image includes:

[0027] extracting the low-level feature information of the first image using a second algorithm;

[0028] A third algorithm is used to perform multi-dimensional calculation on the low-level feature information to obtain the high-level feature information.

[0029] In a second aspect, the embodiment of the present application further provides an image restoration device, which includes:

[0030] An acquisition module configured to acquire an image to be restored;

[0031] An extraction module configured to extract edge feature information and global feature information of the image to be repaired;

[0032] A determination module configured to determine a first image based on the edge feature information and the global feature information;

[0033] A reconstruction module is configured to perform super-resolution reconstruction on the first image to obtain a second image, wherein the second image is an image obtained by completing the restoration of the image to be restored.

[0034] In a third aspect, an embodiment of the present application further provides a storage medium, wherein the computer program, when executed by a processor, executes the steps of any one of the image restoration methods described above.

[0035] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through a bus, and when the machine-readable instructions are executed by the processor, the steps of the image restoration method as described in any one of the above are performed.

[0036] The embodiment of the present application utilizes edge feature information and global feature information of the image to be repaired to determine a first image, which includes relatively comprehensive features of the image to be repaired. The first image is then super-resolution reconstructed to obtain a second image that is clearer, has a higher resolution, and has repaired wear / cracks / color stains / small defects, thereby completing the repair of the image to be repaired, and the accuracy of the image repair is relatively high.

[0037] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 A flow chart of an image restoration method provided by the present application is shown;

[0040] Figure 2A flowchart of an image restoration method provided by the present application for extracting edge feature information and global feature information of an image to be restored is shown;

[0041] Figure 3 A schematic diagram of an image obtained after edge detection is performed on a single-channel image provided by the present application is shown;

[0042] Figure 4 A schematic diagram of an edge mask image provided by the present application is shown;

[0043] Figure 5 A flowchart of performing super-resolution reconstruction on a first image to obtain a second image in an image restoration method provided by the present application is shown;

[0044] Figure 6 A schematic diagram of the structure of a multi-scale residual module provided by the present application is shown;

[0045] Figure 7 A schematic diagram of the structure of a multi-scale residual attention layer provided in the present application is shown;

[0046] Figure 8 A schematic diagram of the structure of an image restoration device provided by the present application is shown;

[0047] Fig. 9 A schematic structural diagram of an electronic device provided by the present application is shown. DETAILED DESCRIPTION

[0048] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0049] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0050] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0051] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0052] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0053] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0054] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0055] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0056] In the first aspect, in order to facilitate the understanding of the present application, an image restoration method provided by the present application is first described in detail. Figure 1 As shown, it is a flowchart of the image restoration method provided by the embodiment of the present application, according to Figure 1 The method steps shown can improve the accuracy of image restoration. The execution subject of the embodiment of the present application can be a server, or a processor, etc., wherein the specific steps include S101-S104.

[0057] S101, obtaining an image to be repaired.

[0058] Here, the image to be repaired includes a target object, and the target object has certain defects, wear and tear, cracks, etc.

[0059] The image to be repaired may be an image contained in a PDF file, or may be a photographed image, etc. Moreover, the format of the image to be repaired may be JPEG, GIF, PNG, etc. The embodiments of the present application do not specifically limit the above.

[0060] S102: extract edge feature information and global feature information of the image to be repaired.

[0061] After obtaining the image to be repaired, based on the strategy of generative adversarial network, a dual-branch network located in the generator is designed to extract edge feature information and global feature information of the image to be repaired, respectively, thereby preventing the omission of feature information in the image to be repaired. Among them, the embodiment of the present application introduces the ResNet network as the discriminator in the generative adversarial network, and completes the network update based on the idea of ​​the generative adversarial network, thereby improving the accuracy of image repair.

[0062] As one example, Figure 2 A flow chart of a method for extracting edge feature information and global feature information of an image to be repaired is shown, wherein specific steps include S201 and S202.

[0063] S201, grayscale processing is performed on the image to be repaired to obtain a single-channel image.

[0064] S202: extract edge feature information based on the single-channel image, and extract global feature information based on the image to be repaired.

[0065] In a specific implementation, the image to be repaired is a color image of 256×256×3, where 256×256 represents the resolution of the image to be repaired, and 3 represents the pixel channel of the image to be repaired. Optionally, when extracting edge feature information of the image to be repaired, the image to be repaired is first grayscaled, such as calling the cvtColor() function in opencv to convert the image to be repaired into a single-channel image, which is a grayscale image of 256×256×1.

[0066] After obtaining the single-channel image, edge detection and masking are performed on the single-channel image to obtain an edge mask image. Optionally, the edge of the single-channel image is screened by the Canny edge algorithm to obtain pixel points with drastic brightness changes in the single-channel image, wherein the screened pixel points represent the contour of the target object in the image to be repaired. Figure 3 The image obtained after edge detection on a single channel image is shown. Figure 3 The image in is masked, that is, the defective area in the image to be repaired is simulated to obtain an edge mask image. Figure 4 An edge mask image is shown.

[0067] After obtaining a single-channel image, the edge mask image is subjected to feature extraction through the codec to obtain edge feature information, wherein the edge feature information includes the position, size, and color of edge pixels, etc. Thus, the edge area missing or damaged in the image to be repaired can be restored more accurately through the extracted edge feature information, thereby improving the accuracy of image repair.

[0068] Furthermore, when the global feature information is extracted based on the image to be repaired, a mask is directly added to the image to be repaired to obtain a global mask image; then, the feature of the global mask image is extracted by the first algorithm to obtain the global feature information. Here, the first algorithm includes a neural network model TransUNet network, specifically using a Transformer encoder to extract features in the global mask image, thereby obtaining the global feature information.

[0069] The global feature information includes the overall structure, content, and correlation between regions of the image to be repaired, such as the category to which the target object in the image to be repaired belongs, the global texture, and the like.

[0070] S103: Determine a first image based on the edge feature information and the global feature information.

[0071] After obtaining the edge feature information and global feature information of the image to be repaired, feature fusion is performed on the edge feature information and the global feature information to obtain a first image. Optionally, the edge feature information and the global feature information are feature fused through a bidirectional gated feature fusion (Bi-directional Gated Feature Fusion, Bi-GFF) module, that is, the ratio of the edge feature information and the global feature information in the feature fusion process is controlled to achieve the exchange and combination of the structure and texture in the edge feature information and the global feature information, and enhance the consistency of the edge feature information and the global feature information in the first image. In addition, it is possible to retain important feature information while suppressing redundant information irrelevant to the repaired image, thereby improving the accuracy of image repair.

[0072] Here, after the feature fusion is completed, the corresponding image after the feature fusion is adjusted to the same resolution and number of channels as the image to be repaired to ensure that the resolution, color, size, etc. of the first image obtained are consistent with the resolution, color, size, etc. of the image to be repaired.

[0073] It is worth noting that S101-S103 in the above steps can be completed based on a pre-built model. At this time, the model needs to be continuously updated to ensure the accuracy of the model. For example, in the process of determining the first image based on edge feature information and global feature information, Resnet-18 is used as a discriminator, which can extract important feature information in the image to determine whether the restored first image is real based on the important feature information. Among them, ResNet-18 can also be set to use residual connection technology, thereby effectively alleviating the gradient vanishing and gradient exploding problems, so that the model has better robustness.

[0074] S104, super-resolution reconstruction is performed on the first image to obtain a second image, wherein the second image is an image obtained by completing restoration of the image to be restored.

[0075] After obtaining the first image, super-resolution reconstruction is performed on the first image to obtain a second image. Figure 5 The method flowchart shown is used to perform super-resolution reconstruction on the first image to obtain the second image, wherein the specific steps include S501-S504.

[0076] S501, extracting low-level feature information and high-level feature information of a first image.

[0077] S502: Fusing the low-level feature information and the high-level feature information to obtain a feature image.

[0078] S503, enlarging the feature image to obtain an enlarged feature image.

[0079] S504: Reconstruct the resolution of the enlarged feature image using the first algorithm to obtain a second image.

[0080] When super-resolution reconstruction is performed on the first image, a super-resolution reconstruction model constructed based on SRGAN can be used to super-resolution reconstruct the first image. Optionally, in order to enhance the network, i.e., the super-resolution reconstruction model's ability to extract features and learn low-resolution images, while avoiding the large amount of computation caused by increasing the network.

[0081] In the specific implementation, the residual block in the original network is replaced by a multi-scale residual block. Figure 6 The figure shows a schematic diagram of the structure of a multi-scale residual module, which introduces convolution kernels of different sizes to increase the width of the network, thereby enabling the super-resolution reconstruction model to adaptively detect the features of images of different scales and realize feature fusion of different scales. n-1 represents the input features, P1, Q1, and L1 represent the features after passing through the convolutional layers with kernel sizes of 3×3, 5×5, and 7×7 in the first layer, respectively, P2, Q2, and L2 represent the features after passing through the convolutional layers with kernel sizes of 3×3, 5×5, and 7×7 in the second layer, respectively, M represents the features obtained after all the features of the second layer are connected and sent to a 1×1 convolutional layer, W n Represents the output features after residual learning.

[0082] In another embodiment, an attention mechanism can be introduced into the multi-scale residual block, and the improved multi-scale residual block and attention mechanism module are used to form a multi-scale residual attention layer. The structure of the multi-scale residual attention layer can refer to Figure 7 , where CA-MSRBT is a multi-scale residual attention layer, MSRBT is a multi-scale residual block, CA is an attention mechanism (Channel attention), and CONV is a convolutional layer with a convolution kernel size of 3×3, 64 channels, and a stride of 1.

[0083] Through skip connections within the layer, residual learning is performed on the low-level features and the processed features to discover feature information, reduce computational redundancy, reduce the amount of computation, and avoid problems such as gradient dissipation and gradient explosion.

[0084] Specifically, when extracting low-level feature information and high-level feature information of the first image, the second algorithm is used to extract the low-level feature information of the first image, and the low-level feature information of the first image includes relatively intuitive information in the first image. The second algorithm belongs to the above-mentioned super-resolution reconstruction model, such as the algorithm in the calculation layer such as the Conv convolution layer and the ReLU layer included in the super-resolution reconstruction model, and the super-resolution reconstruction model is constructed and trained based on deep learning. Low-level: intuitive; high-level: abstract, complex, and semantic

[0085] Furthermore, a third algorithm is used to perform multi-dimensional calculations on the low-level feature information to obtain high-level feature information. For example, the third algorithm includes multiple multi-scale residual attention layers. In the embodiment of the present application, 16 multi-scale residual attention layers are set to obtain multi-scale feature information in layers, and then the output layered feature information is combined through a bottleneck layer. Finally, a convolution layer is used to achieve global multi-scale feature fusion to obtain high-level feature information. Among them, the high-level feature information of the first image includes more abstract and complex information in the first image, and may also include semantic information, etc.

[0086] Furthermore, a residual block is combined to fuse the low-level feature information with the high-level feature information to obtain a feature image, so that the feature image can not only retain and transmit the low-level feature information, but also make the generated high-level feature information more natural and realistic.

[0087] After obtaining the feature image, the feature image is enlarged by the sub-pixel convolution layer set in the super-resolution reconstruction model to obtain an enlarged feature image. Afterwards, the enlarged feature image is reconstructed by the first algorithm to obtain a second image, wherein the first algorithm also belongs to the above-mentioned super-resolution reconstruction model, such as the algorithm in the Conv layer, to obtain a high-resolution image, i.e., the second image, which is the image obtained by completing the restoration of the image to be restored, and is more complete, clearer, and has better image quality than the image to be restored.

[0088] Of course, the super-resolution reconstruction model also needs to be trained, such as real-time or periodic training based on deep learning, to ensure its accuracy.

[0089] The image restoration method of the embodiment of the present application can be applied to images to be restored in different types and scenarios. For example, it can be used to restore damaged cultural relics of various categories in museum collections, thereby assisting cultural relic restoration workers to quickly and efficiently restore cultural relics. In other words, a clearer image corresponding to the cultural relic is obtained, so that the restoration personnel can refer to the restoration details and restore more realistic cultural relic works.

[0090] Of course, after completing the restoration of the image to be restored using the above-mentioned image restoration method, that is, after obtaining the second image corresponding to the image to be restored, the image to be restored and the second image can also be displayed on the same display device at the same time, so that the restoration personnel can compare the difference between the image to be restored and the second image, and / or, so that the restoration personnel can confirm whether the second image has completed the complete restoration of the image to be restored, and / or, so that the restoration personnel can confirm whether the second image meets the restoration requirements, etc., thereby realizing the visualization requirements and making the restoration effect more intuitive.

[0091] The embodiment of the present application uses the edge feature information and global feature information of the image to be repaired to determine a first image, which includes relatively comprehensive features in the image to be repaired, and then performs super-resolution reconstruction on the first image to obtain a second image that is relatively clear, has a higher resolution, and has repaired wear / cracks / color stains / small parts of the image, thereby completing the repair of the image to be repaired, and the image repair has a high accuracy. At the same time, the image repair method can be equipped with a general cultural relic restoration system, with a simple interface, visualization function, easy operation, and convenient for users to use, which greatly improves the efficiency of image repair.

[0092] Based on the same inventive concept, the second aspect of the present application also provides an image repair device corresponding to an image repair method. Since the principle of solving the problem by the image repair device in the present application is similar to that of the above-mentioned image repair method of the present application, the implementation of the image repair device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0093] Figure 8 A schematic diagram of an image restoration device provided in an embodiment of the present application is shown, which specifically includes:

[0094] An acquisition module 801 is configured to acquire an image to be restored;

[0095] An extraction module 802 is configured to extract edge feature information and global feature information of the image to be repaired;

[0096] A determination module 803, configured to determine a first image based on the edge feature information and the global feature information;

[0097] The reconstruction module 804 is configured to perform super-resolution reconstruction on the first image to obtain a second image, wherein the second image is an image obtained by completing the restoration of the image to be restored.

[0098] In yet another embodiment, the extraction module 802 is specifically configured as follows:

[0099] Gray-scaling the image to be repaired to obtain a single-channel image;

[0100] Based on the single-channel image, the edge feature information is extracted, and based on the image to be repaired, the global feature information is extracted.

[0101] In yet another embodiment, the extraction module 802 is further configured to:

[0102] Performing edge detection and masking on the single-channel image to obtain an edge mask image;

[0103] The edge feature information is obtained by performing feature extraction on the edge mask image through a codec.

[0104] In yet another embodiment, the extraction module 802 is further configured to:

[0105] Adding a mask to the image to be repaired to obtain a global mask image;

[0106] A first algorithm is used to perform feature extraction on the global mask image to obtain the global feature information.

[0107] In yet another embodiment, the determination module 803 is specifically configured as follows:

[0108] Feature fusion is performed on the edge feature information and the global feature information to obtain the first image.

[0109] In yet another embodiment, the reconstruction module 804 is specifically configured as follows:

[0110] Extracting low-level feature information and high-level feature information of the first image;

[0111] Fusing the low-level feature information with the high-level feature information to obtain a feature image;

[0112] Amplifying the characteristic image to obtain an amplified characteristic image;

[0113] The resolution of the enlarged feature image is reconstructed using the first algorithm to obtain the second image.

[0114] In yet another embodiment, the reconstruction module 804 is further configured to:

[0115] extracting the low-level feature information of the first image using a second algorithm;

[0116] A third algorithm is used to perform multi-dimensional calculation on the low-level feature information to obtain the high-level feature information.

[0117] The embodiment of the present application utilizes edge feature information and global feature information of the image to be repaired to determine a first image, which includes relatively comprehensive features of the image to be repaired. The first image is then super-resolution reconstructed to obtain a second image that is clearer, has a higher resolution, and has repaired wear / cracks / color stains / small defects, thereby completing the repair of the image to be repaired, and the accuracy of the image repair is relatively high.

[0118] An embodiment of the present application provides a storage medium, which is a computer-readable medium and stores a computer program. When the computer program is executed by a processor, the method provided by any embodiment of the present application is implemented, including the following steps S11 to S14:

[0119] S11, obtaining the image to be repaired;

[0120] S12, extracting edge feature information and global feature information of the image to be repaired;

[0121] S13, determining a first image based on the edge feature information and the global feature information;

[0122] S14, performing super-resolution reconstruction on the first image to obtain a second image, wherein the second image is an image obtained by completing restoration of the image to be restored.

[0123] The embodiment of the present application utilizes edge feature information and global feature information of the image to be repaired to determine a first image, which includes relatively comprehensive features of the image to be repaired. The first image is then super-resolution reconstructed to obtain a second image that is clearer, has a higher resolution, and has repaired wear / cracks / color stains / small defects, thereby completing the repair of the image to be repaired, and the accuracy of the image repair is relatively high.

[0124] The embodiment of the present application also provides an electronic device. The structural diagram of the electronic device can be as follows: Fig. 9 As shown, at least a memory 901 and a processor 902 are included, the memory 901 stores a computer program, and the processor 902 implements the method provided by any embodiment of the present application when executing the computer program on the memory 901. Exemplarily, the steps of the electronic device computer program are as follows S21 to S24:

[0125] S21, obtaining the image to be repaired;

[0126] S22, extracting edge feature information and global feature information of the image to be repaired;

[0127] S23, determining a first image based on the edge feature information and the global feature information;

[0128] S24, performing super-resolution reconstruction on the first image to obtain a second image, wherein the second image is an image obtained by completing restoration of the image to be restored.

[0129] The embodiment of the present application utilizes edge feature information and global feature information of the image to be repaired to determine a first image, which includes relatively comprehensive features of the image to be repaired. The first image is then super-resolution reconstructed to obtain a second image that is clearer, has a higher resolution, and has repaired wear / cracks / color stains / small defects, thereby completing the repair of the image to be repaired, and the accuracy of the image repair is relatively high.

[0130] Optionally, in this embodiment, the above-mentioned storage medium may include, but is not limited to, various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk. Optionally, in this embodiment, the processor executes the method steps recorded in the above-mentioned embodiment according to the program code stored in the storage medium. Optionally, the specific examples in this embodiment can refer to the examples described in the above-mentioned embodiment and the optional implementation mode, and this embodiment is not repeated here. Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented with a general computing device, which can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Optionally, they can be implemented with executable program codes of computing devices, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from here, or they can be made into individual integrated circuit modules respectively, or multiple modules or steps in them can be made into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.

[0131] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0132] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0133] Multiple embodiments of the present application are described in detail above, but the present application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications to the embodiments based on the concept of the present application, and these variations and modifications should all fall within the scope of protection required by the present application.

Claims

1. An image restoration method, characterized in that: include: Obtain the image to be repaired; Extracting edge feature information and global feature information of the image to be repaired; Determine a first image based on the edge feature information and the global feature information; The first image is super-resolution reconstructed to obtain a second image, wherein the second image is an image obtained by completing restoration of the image to be restored.

2. The image restoration method according to claim 1, characterized in that: The step of extracting edge feature information and global feature information of the image to be repaired includes: Gray-scaling the image to be repaired to obtain a single-channel image; Based on the single-channel image, the edge feature information is extracted, and based on the image to be repaired, the global feature information is extracted.

3. The image restoration method according to claim 2, characterized in that: The step of extracting the edge feature information based on the single-channel image includes: Performing edge detection and masking on the single-channel image to obtain an edge mask image; The edge feature information is obtained by performing feature extraction on the edge mask image through a codec.

4. The image restoration method according to claim 2, characterized in that: The extracting the global feature information based on the image to be repaired includes: Adding a mask to the image to be repaired to obtain a global mask image; A first algorithm is used to perform feature extraction on the global mask image to obtain the global feature information.

5. The image restoration method according to any one of claims 1 to 4, characterized in that: The determining of the first image based on the edge feature information and the global feature information includes: Feature fusion is performed on the edge feature information and the global feature information to obtain the first image.

6. The image restoration method according to claim 1, characterized in that: The step of performing super-resolution reconstruction on the first image to obtain a second image comprises: Extracting low-level feature information and high-level feature information of the first image; Fusing the low-level feature information with the high-level feature information to obtain a feature image; Amplifying the characteristic image to obtain an amplified characteristic image; The resolution of the enlarged feature image is reconstructed using the first algorithm to obtain the second image.

7. The image restoration method according to claim 6, characterized in that: The extracting low-level feature information and high-level feature information of the first image includes: extracting the low-level feature information of the first image using a second algorithm; A third algorithm is used to perform multi-dimensional calculation on the low-level feature information to obtain the high-level feature information.

8. An image restoration device, characterized in that: include: An acquisition module configured to acquire an image to be restored; An extraction module configured to extract edge feature information and global feature information of the image to be repaired; A determination module configured to determine a first image based on the edge feature information and the global feature information; A reconstruction module is configured to perform super-resolution reconstruction on the first image to obtain a second image, wherein the second image is an image obtained by completing the restoration of the image to be restored.

9. A storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image restoration method according to any one of claims 1 to 7 are executed.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via a bus, and when the machine-readable instructions are executed by the processor, the steps of the image restoration method as described in any one of claims 1 to 7 are performed.

Citation Information

Patent Citations

  • Image restoration method based on edge generation

    CN111047522A

  • Image completion method and system based on semantic edge fusion

    CN112184585A

  • Image restoration method based on edge information reconstruction

    CN113240613A

  • Image super-resolution reconstruction method and device

    CN114581300A

  • Image restoration method

    CN116051407A