Image restoration method and device, electronic equipment and storage medium
By chunking and hierarchical classification of images, selecting suitable image repair algorithms for repair, and splicing the repaired image blocks together, the problem of slow image repair speed in the prior art is solved, and more efficient and high-quality image repair effect is achieved.
Patent Information
- Application Number
- CN202411600411.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-03-07
AI Technical Summary
Existing image repair techniques use a single algorithm to process high-resolution images, resulting in problems such as large calculations and slow speed.
By chunking the images to be processed, the complexity level is determined for each image block, and the corresponding image repair algorithm is selected according to the complex level for repair, and the repaired image blocks are finally stitched into a complete repair image.
This method reduces the computational complexity through block processing and differentiated image repair algorithm processing, improves the efficiency and quality of image repair, and solves the problem of slow processing speed in the prior art.
Smart Images
Figure CN119579467A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for image restoration. Background Art
[0002] Image restoration technology is widely used in the fields of face recognition, beauty cameras, artificial intelligence, and many other fields, as well as in many other fields such as medical care, security, and entertainment. Especially with the rapid development of social networks today, people need to use image restoration technology to improve the beauty of selfies or portraits.
[0003] Currently, the most common technology in the field of image restoration is to use deep learning, such as convolutional neural networks, to repair facial images. Although this method has achieved good results, it requires processing a large number of high-resolution facial images, so the calculation is large and the speed is slow. Summary of the invention
[0004] In view of this, in order to solve the problem of slow processing speed caused by using a single image restoration algorithm, an embodiment of the present invention provides a method, device, electronic device and storage medium for image restoration.
[0005] In a first aspect, an embodiment of the present invention provides an image restoration method, the method comprising:
[0006] The image to be processed is processed into blocks to obtain multiple image blocks;
[0007] For each image block, the following steps are performed: performing a grade classification process on the image block to determine a complexity level of the image block, determining an image restoration algorithm for restoring the image block according to the complexity level, and performing a restoration process on the image block based on the image restoration algorithm to obtain a restored image block; wherein, the higher the complexity level of the image block, the more complex the model of the corresponding image restoration algorithm;
[0008] The repaired image blocks are stitched together to obtain the repaired image.
[0009] In a possible implementation, performing a level classification process on the image block to determine the complexity level of the image block includes:
[0010] The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
[0011] In one possible implementation, the training process of the complex hierarchical classification model includes:
[0012] Acquire multiple training samples, wherein each training sample includes a training image block and a complexity level of annotated training image block;
[0013] The training image blocks in each training sample are input into the classification model for training to obtain the predicted complexity level;
[0014] Determine a convergence function according to the predicted complexity level and the complexity level corresponding to each training sample;
[0015] By minimizing the convergence function, the network parameters of the classification model are determined and a complex hierarchical classification model is obtained.
[0016] In a possible implementation, determining an image restoration algorithm for restoring an image block according to a complexity level includes:
[0017] An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
[0018] In a possible implementation manner, each image block is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed;
[0019] Each repaired image block is stitched together to obtain a repaired image, including:
[0020] Each repaired image block is stitched according to the position information to obtain a repaired image.
[0021] In a second aspect, an embodiment of the present invention provides an image restoration device, the device comprising:
[0022] A block processing module is used to process the image to be processed in blocks to obtain multiple image blocks;
[0023] The restoration module is used to perform, for each image block, the following steps: classify the image blocks to determine the complexity level of the image blocks, determine the image restoration algorithm for restoring the image blocks according to the complexity level, and perform restoration processing on the image blocks based on the image restoration algorithm to obtain the restored image blocks; wherein the higher the complexity level of the image blocks, the more complex the model of the corresponding image restoration algorithm;
[0024] The stitching module is used to stitch the various repaired image blocks to obtain a repaired image.
[0025] In a possible implementation, performing a level classification process on the image block to determine the complexity level of the image block includes:
[0026] The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
[0027] In one possible implementation, the training process of the complex hierarchical classification model includes:
[0028] Acquire multiple training samples, wherein each training sample includes a training image block and a complexity level of annotated training image block;
[0029] The training image blocks in each training sample are input into the classification model for training to obtain the predicted complexity level;
[0030] Determine a convergence function according to the predicted complexity level and the complexity level corresponding to each training sample;
[0031] By minimizing the convergence function, the network parameters of the classification model are determined and a complex hierarchical classification model is obtained.
[0032] In a possible implementation, the repair module is further configured to:
[0033] An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
[0034] In a possible implementation manner, each image block is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed;
[0035] The splicing module is also used for:
[0036] Each repaired image block is stitched according to the position information to obtain a repaired image.
[0037] In a third aspect, an embodiment of the present invention provides an electronic device, which includes: a processor and a memory, the processor is used to execute an image restoration program stored in the memory to implement the above-mentioned image restoration method.
[0038] In a fourth aspect, an embodiment of the present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned image restoration method.
[0039] The method, device, electronic device and storage medium for image restoration provided by the embodiments of the present invention include: dividing the image to be processed into blocks to obtain multiple image blocks; for each image block, performing the following steps: classifying the image blocks to determine the complexity level of the image blocks, determining the image restoration algorithm for restoring the image blocks according to the complexity level, restoring the image blocks based on the image restoration algorithm to obtain the restored image blocks; wherein, the higher the complexity level of the image blocks, the more complex the model of the corresponding image restoration algorithm; and splicing the restored image blocks to obtain the restored image. The above technical solution can divide the image to be processed into blocks to reduce the computational complexity, and determine the complexity level of each divided image block so that the image restoration algorithm corresponding to the complexity level can perform image restoration. Compared with the method of using a single image restoration algorithm to restore the entire image in the prior art, this method can better perform differentiated processing on complex and simple image parts, which not only ensures the restoration quality, but also improves the restoration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A flowchart of an embodiment of an image restoration method provided by an embodiment of the present invention;
[0041] Figure 2 A training flow chart of a complex hierarchical classification model provided by an embodiment of the present invention;
[0042] Figure 3 A block diagram of an embodiment of an image restoration device provided by an embodiment of the present invention;
[0043] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0045] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.
[0046] The present invention provides a method for image restoration. Figure 1 , Figure 1 A flowchart of an embodiment of an image restoration method provided by an embodiment of the present invention. Figure 1 The process shown may include the following steps:
[0047] Step 101, performing block processing on the image to be processed to obtain multiple image blocks;
[0048] In one implementation, the image to be processed may be processed in blocks according to a block ratio, for example, the block ratio is 2*2, 3*3 or 4*4, etc., and the image to be processed is evenly divided into multiple image blocks according to the block ratio.
[0049] In another embodiment, the image to be processed can be divided into blocks according to specific areas. Specifically, if the image to be processed is a face image, the face image can be divided into blocks according to various parts of the face to obtain image blocks of eyes, cheeks, noses, mouths, foreheads, etc.; if the image to be processed is a scene image, such as a mountain, a tree, or a person, it can be divided into blocks according to different objects to obtain image blocks of mountains, trees, and people.
[0050] In order to achieve the best effect, the appropriate division method and image block size can be selected according to the actual situation. Specifically, a dynamic method can be considered to dynamically perform block processing to obtain image blocks. The block processing method is not limited here, and all methods of dividing the image into blocks are applicable in this embodiment.
[0051] Step 102, for each image block, the following are performed: performing a grade classification process on the image block to determine a complexity level of the image block, determining an image restoration algorithm for restoring the image block according to the complexity level, and performing a restoration process on the image block based on the image restoration algorithm to obtain a restored image block;
[0052] The complexity level is used to characterize the complexity of the image features included in the image block. It can be understood that the higher the complexity level, the more complex the image features, and vice versa. For example, the texture complexity of different parts of a face image is different. For example, the texture features of the eyes, nose and other parts may be more complex, while the texture features of the cheeks and other parts may be simpler. In order to distinguish the complexity of the image features included in each image block, it is necessary to classify each image block to determine the complexity level of each image block, so that the complexity level can be used to intuitively represent the complexity of the image features included in each image block.
[0053] Among them, the higher the complexity level of the image block, the more complex the model of the corresponding image restoration algorithm; it can be understood that the higher the complexity of the image block, the stronger the restoration ability of the image restoration algorithm is required to restore it, and vice versa. Specifically, for image blocks with simple image features, a simple image restoration algorithm is used to restore them, thereby reducing the amount of calculation. For image blocks with complex image features, a complex image restoration algorithm is used to restore them, truly achieving differentiated processing of complex and simple image parts, thereby improving the restoration effect. This method not only ensures the quality of restoration, but also helps to save computing resources and improve restoration efficiency.
[0054] Step 103, stitching the repaired image blocks to obtain a repaired image.
[0055] After the above step 102, each image block is repaired in a targeted manner to obtain a repaired image block. In order to obtain a complete repaired image, each repaired image block needs to be spliced so that each repaired image block is restored to the original image position to obtain the final repaired image. The repaired image is displayed on the user's terminal, such as a mobile phone, computer, tablet, or other electronic device that can display images, for the user's use. The above image to be processed can also be saved on the user's terminal, so that the user can choose different block processing methods to re-perform personalized repair on the image to be processed.
[0056] The method for image restoration provided by the embodiment of the present invention comprises: dividing the image to be processed into blocks to obtain multiple image blocks; for each image block, performing the following steps: classifying the image blocks to determine the complexity level of the image blocks, determining the image restoration algorithm for restoring the image blocks according to the complexity level, restoring the image blocks based on the image restoration algorithm to obtain the restored image blocks; wherein, the higher the complexity level of the image blocks, the more complex the model of the corresponding image restoration algorithm; and splicing the restored image blocks to obtain the restored image. The above technical solution can divide the image to be processed into blocks to reduce the computational complexity, and determine the complexity level of each divided image block, so that the image restoration algorithm corresponding to the complexity level can perform image restoration. Compared with the method of using a single image restoration algorithm to restore the entire image in the prior art, this method can better perform differentiated processing on complex and simple image parts, which not only ensures the restoration quality, but also improves the restoration efficiency.
[0057] The step 102 of classifying the image block to determine the complexity level of the image block is specifically implemented in the following manner: the image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
[0058] The training process of complex hierarchical classification models can be achieved by Figure 2 Steps 201 to 204 shown implement:
[0059] Step 201, obtaining a plurality of training samples, wherein each training sample includes a training image block and a complexity level annotated on the training image block;
[0060] The complexity level of annotation of each training image block is used as the data label of the training sample. In the field of machine learning and deep learning, data annotation refers to the process of manually labeling training samples, giving each training sample a clear attribute information so that the classification model can understand the meaning and category of the data. Through data annotation, the classification model can learn the mapping relationship between input data and correct output labels during the training process, so that it has the ability to accurately identify and classify new data that it has never seen. Therefore, accurate annotation labels are the key to improving model accuracy.
[0061] Step 202, inputting the training image blocks in each training sample into the classification model for training to obtain a predicted complexity level;
[0062] The classification model can be a deep learning model, such as a deep convolutional neural network, or it can be an algorithm model that can achieve classification, such as a random forest, decision tree, naive Bayes, etc., which is not limited here. All algorithm models that can achieve classification are applicable in this embodiment.
[0063] Step 203, determining a convergence function according to the predicted complexity level and the complexity level corresponding to each training sample;
[0064] The convergence function can be obtained by using the cross entropy method as shown in the formula:
[0065]
[0066] Among them, i represents the image block, N represents the total number of image blocks during training, and y i represents the complexity level of the i-th image block, p(y i ) represents the prediction complexity level of the i-th image block.
[0067] Step 204, by minimizing the convergence function, the network parameters of the classification model are determined to obtain a complex hierarchical classification model.
[0068] During the training process, the gradient descent algorithm is used to gradually minimize the convergence function and continuously adjust the network parameters. The gradual decrease of the convergence function is a sign that the model has learned the data patterns and features. When the convergence function no longer decreases significantly, it can be considered that the classification model has converged, thereby ultimately training a complex hierarchical classification model that can correctly classify the complexity levels of image blocks.
[0069] In fact, when meeting certain specific requirements, it may be necessary to select other types of convergence functions, which will not be described here one by one.
[0070] In the specific implementation, in addition to using the convergence function to ensure the completion of the training of the classification model, the accuracy rate, learning rate or number of training times can also be used to ensure the convergence training of the classification model to obtain a complex hierarchical classification model, which will not be described in detail here.
[0071] The image restoration algorithm for the image block to be restored according to the complexity level in step 102 is specifically implemented in the following manner:
[0072] An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
[0073] To facilitate understanding of the image modification algorithm library, as shown in Table 1, the complexity level is represented by numbers, and the larger the number, the higher the complexity level:
[0074] Complexity level Image Inpainting Algorithms 1 Image restoration algorithm 1 2 Image Inpainting Algorithm 2 3 Image restoration algorithm 3 4 Image restoration algorithm 4
[0075] It should be noted that Table 1 only shows an example of the correspondence between the complexity level and the image restoration algorithm. The specific correspondence between the complexity level and the image restoration algorithm can be set according to actual needs and is not limited here.
[0076] In one embodiment, image restoration algorithm 1, image restoration algorithm 2, image restoration algorithm 3, and image restoration algorithm 4 in the above table may be the same image restoration algorithm, such as a deep learning algorithm, but the network structure layers and / or network parameters of each deep learning algorithm are different to obtain the image modification algorithms with different image restoration capabilities; in another embodiment, image restoration algorithm 1, image restoration algorithm 2, image restoration algorithm 3, and image restoration algorithm 4 in the above table may be multiple different image restoration algorithms with different image restoration capabilities.
[0077] For example, the complexity level of image block 1 is 2. By querying Table 1, it can be known that the image restoration algorithm corresponding to complexity level 2 is image restoration algorithm 2. Then image restoration algorithm 2 can be used to perform image restoration on image block 1 using the corresponding algorithm. Similarly, the process of determining the image restoration algorithm corresponding to each image block is the same as above and will not be elaborated here.
[0078] In practical applications, each image block after block processing is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed; the specific position information can be the upper left corner position coordinates and the lower right corner position coordinates in the image block to represent the specific position of the image block in the image to be processed. The repaired image blocks can be re-stitched into a complete repaired image through the position information of each image block. The above step 103 stitches each repaired image block to obtain the repaired image. Specifically, the process is: stitching each repaired image block according to the position information to obtain the repaired image.
[0079] See also Figure 3 , is a block diagram of an embodiment of an image restoration device provided by an embodiment of the present invention. Figure 3 As shown, the device comprises:
[0080] A block processing module 301 is used to perform block processing on the image to be processed to obtain multiple image blocks;
[0081] The restoration module 302 is used to perform, for each image block, the following steps: performing a grade classification process on the image block to determine a complexity level of the image block, determining an image restoration algorithm for restoring the image block according to the complexity level, and performing a restoration process on the image block based on the image restoration algorithm to obtain a restored image block; wherein, the higher the complexity level of the image block, the more complex the model of the corresponding image restoration algorithm;
[0082] The stitching module 303 is used to stitch the repaired image blocks to obtain a repaired image.
[0083] The image restoration device provided by the embodiment of the present invention comprises: dividing the image to be processed into blocks to obtain multiple image blocks; for each image block, performing the following steps: classifying the image blocks to determine the complexity level of the image blocks, determining the image restoration algorithm for restoring the image blocks according to the complexity level, restoring the image blocks based on the image restoration algorithm to obtain the restored image blocks; wherein, the higher the complexity level of the image blocks, the more complex the model of the corresponding image restoration algorithm; and splicing the restored image blocks to obtain the restored image. The above technical solution can divide the image to be processed into blocks to reduce the computational complexity, and determine the complexity level of each divided image block, so that the image restoration algorithm corresponding to the complexity level can perform image restoration. Compared with the method of using a single image restoration algorithm to restore the entire image in the prior art, this method can better differentiate the complex and simple image parts, which not only ensures the restoration quality, but also improves the restoration efficiency.
[0084] In a possible implementation, performing a level classification process on the image block to determine the complexity level of the image block includes:
[0085] The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
[0086] In one possible implementation, the training process of the complex hierarchical classification model includes:
[0087] Acquire multiple training samples, wherein each training sample includes a training image block and a complexity level of annotated training image block;
[0088] The training image blocks in each training sample are input into the classification model for training to obtain the predicted complexity level;
[0089] Determine a convergence function according to the predicted complexity level and the complexity level corresponding to each training sample;
[0090] By minimizing the convergence function, the network parameters of the classification model are determined and a complex hierarchical classification model is obtained.
[0091] In a possible implementation, the repair module 302 is further configured to:
[0092] An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
[0093] In a possible implementation manner, each image block is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed;
[0094] The splicing module 303 is also used for:
[0095] Each repaired image block is stitched according to the position information to obtain a repaired image.
[0096] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, Figure 4 The electronic device 500 shown includes: at least one processor 501, a memory 502, at least one network interface 504 and other user interfaces 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 505 is not shown in FIG. Figure 4 Various buses are labeled as bus system 505 .
[0097] The user interface 503 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).
[0098] It can be understood that the memory 502 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0099] In some implementations, the memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 5021 and an application program 5022 .
[0100] The operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 5022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 5022.
[0101] In the embodiment of the present invention, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application 5022, the processor 501 is used to execute the method steps provided by each method embodiment, for example, including:
[0102] The image to be processed is processed into blocks to obtain multiple image blocks;
[0103] For each image block, the following steps are performed: performing a grade classification process on the image block to determine a complexity level of the image block, determining an image restoration algorithm for restoring the image block according to the complexity level, and performing a restoration process on the image block based on the image restoration algorithm to obtain a restored image block; wherein, the higher the complexity level of the image block, the more complex the model of the corresponding image restoration algorithm;
[0104] The repaired image blocks are stitched together to obtain the repaired image.
[0105] In a possible implementation, performing a level classification process on the image block to determine the complexity level of the image block includes:
[0106] The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
[0107] In one possible implementation, the training process of the complex hierarchical classification model includes:
[0108] Acquire multiple training samples, wherein each training sample includes a training image block and a complexity level of annotated training image block;
[0109] The training image blocks in each training sample are input into the classification model for training to obtain the predicted complexity level;
[0110] Determine a convergence function according to the predicted complexity level and the complexity level corresponding to each training sample;
[0111] By minimizing the convergence function, the network parameters of the classification model are determined and a complex hierarchical classification model is obtained.
[0112] In a possible implementation, determining an image restoration algorithm for restoring an image block according to a complexity level includes:
[0113] An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
[0114] In a possible implementation manner, each image block is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed;
[0115] Each repaired image block is stitched together to obtain a repaired image, including:
[0116] Each repaired image block is stitched according to the position information to obtain a repaired image.
[0117] The method disclosed in the above embodiment of the present invention can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 501 or the instruction in the form of software. The above processor 501 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the steps of the above method in combination with its hardware.
[0118] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.
[0119] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0120] The electronic device provided in this embodiment may be Figure 4 The electronic device shown in FIG. 1 may perform the following steps: Figure 1-2 All steps of the image restoration method in Figure 1-2 For details, please refer to Figure 1-2 For the sake of brevity, the relevant description is not repeated here.
[0121] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.
[0122] When one or more programs in the storage medium can be executed by one or more processors, the above-mentioned image restoration method can be implemented.
[0123] The processor is used to execute the image restoration program stored in the memory to implement the following steps of the image restoration method:
[0124] The image to be processed is processed into blocks to obtain multiple image blocks;
[0125] For each image block, the following steps are performed: performing a grade classification process on the image block to determine a complexity level of the image block, determining an image restoration algorithm for restoring the image block according to the complexity level, and performing a restoration process on the image block based on the image restoration algorithm to obtain a restored image block; wherein, the higher the complexity level of the image block, the more complex the model of the corresponding image restoration algorithm;
[0126] The repaired image blocks are stitched together to obtain the repaired image.
[0127] In a possible implementation, performing a level classification process on the image block to determine the complexity level of the image block includes:
[0128] The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
[0129] In one possible implementation, the training process of the complex hierarchical classification model includes:
[0130] Acquire multiple training samples, wherein each training sample includes a training image block and a complexity level of annotated training image block;
[0131] The training image blocks in each training sample are input into the classification model for training to obtain the predicted complexity level;
[0132] Determine a convergence function according to the predicted complexity level and the complexity level corresponding to each training sample;
[0133] By minimizing the convergence function, the network parameters of the classification model are determined and a complex hierarchical classification model is obtained.
[0134] In a possible implementation, determining an image restoration algorithm for restoring an image block according to a complexity level includes:
[0135] An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
[0136] In a possible implementation manner, each image block is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed;
[0137] Each repaired image block is stitched together to obtain a repaired image, including:
[0138] Each repaired image block is stitched according to the position information to obtain a repaired image.
[0139] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0140] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0141] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for image restoration, characterized in that: The method comprises: The image to be processed is processed into blocks to obtain multiple image blocks; For each of the image blocks, the following steps are performed: performing a grade classification process on the image block to determine a complexity level of the image block, determining an image restoration algorithm for restoring the image block according to the complexity level, and performing a restoration process on the image block based on the image restoration algorithm to obtain a restored image block; wherein, the higher the complexity level of the image block, the more complex the model of the corresponding image restoration algorithm; The repaired image blocks are stitched together to obtain a repaired image.
2. The method according to claim 1, characterized in that The step of performing a level classification process on the image block to determine the complexity level of the image block includes: The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
3. The method according to claim 2, characterized in that The training process of the complex hierarchical classification model includes: Acquire a plurality of training samples, wherein each of the training samples includes a training image block and a complexity level annotated for the training image block; Inputting the training image blocks in each training sample into the classification model for training to obtain a predicted complexity level; Determining a convergence function according to the predicted complexity level and the complexity level corresponding to each of the training samples; The network parameters of the classification model are determined by minimizing the convergence function to obtain the complex hierarchical classification model.
4. The method according to claim 1, characterized in that: The step of determining an image restoration algorithm for restoring the image block according to the complexity level comprises: An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
5. The method according to claim 1, characterized in that Each of the image blocks is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed; The step of performing image stitching on the repaired image blocks to obtain the repaired image comprises: The repaired image blocks are stitched together according to the position information to obtain a repaired image.
6. An image restoration device, characterized in that: The device comprises: A block processing module, used for performing block processing on the image to be processed to obtain multiple image blocks; A restoration module is used to perform, for each of the image blocks, the following steps: performing a grade classification process on the image blocks to determine a complexity level of the image blocks, determining an image restoration algorithm for restoring the image blocks according to the complexity level, and performing a restoration process on the image blocks based on the image restoration algorithm to obtain a restored image block; wherein, the higher the complexity level of the image blocks, the more complex the model of the corresponding image restoration algorithm; The stitching module is used to stitch the repaired image blocks to obtain a repaired image.
7. The device according to claim 6, characterized in that The step of performing a level classification process on the image block to determine the complexity level of the image block includes: The image block is input into a pre-trained complexity level classification model, and the complexity level classification model outputs the complexity level of the image block.
8. The device according to claim 7, characterized in that The training process of the complex hierarchical classification model includes: Acquire a plurality of training samples, wherein each of the training samples includes a training image block and a complexity level annotated for the training image block; Inputting the training image blocks in each training sample into the classification model for training to obtain a predicted complexity level; Determining a convergence function according to the predicted complexity level and the complexity level corresponding to each of the training samples; The network parameters of the classification model are determined by minimizing the convergence function to obtain the complex hierarchical classification model.
9. The device according to claim 6, characterized in that The repair module is further used for: An image restoration algorithm corresponding to the complexity level is searched from an image restoration algorithm library; wherein the image restoration algorithm library stores a correspondence between the complexity level and the image restoration algorithm.
10. The device according to claim 6, characterized in that Each of the image blocks is marked with position information, and the position information is used to indicate the position of the image block in the image to be processed; The splicing module is also used for: The repaired image blocks are stitched together according to the position information to obtain a repaired image.
11. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is used to execute the image restoration program stored in the memory to implement the image restoration method according to any one of claims 1 to 5.
12. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the image restoration method according to any one of claims 1 to 5.