Restoration methods and devices based on series models

Abstract

This application provides a restoration method, apparatus, electronic device, and storage medium based on a concatenated model, relating to the field of computer technology. The method includes: acquiring a low-quality image; inputting the low-quality image into a pre-trained image quality evaluation large language model to obtain degradation information and semantic information corresponding to the low-quality image, wherein the semantic information includes detailed semantic information and simplified semantic information; retrieving a pre-trained low-frequency restoration model and a high-frequency restoration model; inputting the low-quality image, degradation information, and simplified semantic information into the low-frequency restoration model to obtain a corresponding low-frequency restored image; and inputting the low-frequency restored image and the detailed semantic information into the high-frequency restoration model to determine the corresponding target restored image. This application solves the problem of low image restoration quality in related technologies.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a restoration method, apparatus, electronic device, and storage medium based on a serial model. Background Technology

[0002] Image restoration technology is mainly proposed to address the "degradation" in the imaging process. The "degradation" phenomenon in the imaging process mainly refers to the influence of various factors on the imaging system, such as defocusing of the imaging system, relative motion between the device and the object, or inherent defects of the equipment, which result in the image quality not meeting the ideal requirements.

[0003] In the industry, there are two main solutions. One is based on manual restoration, where professional image restorers manually edit low-quality images into high-quality ones using post-editing software. The second is to use traditional convolutional neural networks for restoration, but this often results in over-smoothing or localized artifacts, thus affecting the quality of image restoration.

[0004] As can be seen from the above, how to improve the quality of image restoration still needs to be addressed. Summary of the Invention

[0005] This application provides a restoration method, apparatus, electronic device, and storage medium based on a series model, which can solve the problem of low image restoration quality in related technologies. The technical solutions are as follows:

[0006] According to one aspect of this application, a restoration method based on a concatenated model includes: acquiring a low-quality image; inputting the low-quality image into a pre-trained image quality evaluation large language model to obtain degradation information and semantic information corresponding to the low-quality image, wherein the semantic information includes detailed semantic information and simplified semantic information; retrieving a pre-trained low-frequency restoration model and a high-frequency restoration model; inputting the low-quality image, degradation information, and simplified semantic information into the low-frequency restoration model to obtain a corresponding low-frequency restored image; and inputting the low-frequency restored image and the detailed semantic information into the high-frequency restoration model to determine a corresponding target restored image.

[0007] According to one aspect of this application, a restoration device based on a series model is provided, the device including but not limited to:

[0008] The information acquisition module acquires low-quality images and inputs them into a pre-trained image quality evaluation language model to obtain degradation information and semantic information corresponding to the low-quality images. The semantic information includes detailed semantic information and simplified semantic information.

[0009] The low-frequency restored image acquisition module retrieves the pre-trained low-frequency restoration model and high-frequency restoration model, and inputs the low-quality image, degradation information and simplified semantic information into the low-frequency restoration model to obtain the corresponding low-frequency restored image.

[0010] The target restored image determination module inputs the low-frequency restored image and the detailed semantic information into the high-frequency restoration model to determine the corresponding target restored image.

[0011] In one exemplary embodiment, the apparatus includes, but is not limited to:

[0012] The low-frequency degradation module acquires high-quality training images, degrades the high-quality training images, and acquires corresponding low-quality training images and training degradation information.

[0013] The low-frequency training restoration image determination module retrieves the low-frequency training model and uses the low-quality training image and the training degradation information to input the low-frequency training model to determine the corresponding low-frequency training restoration image.

[0014] The low-frequency training module is used to determine the corresponding training loss function between the high-quality restored image and the low-frequency training restored image, modify the parameters in the low-frequency training model based on the training loss function, and repeatedly obtain the training process of the low-frequency training restored image until the low-frequency training model training converges, and determine the corresponding low-frequency restoration model.

[0015] In one exemplary embodiment, the apparatus includes, but is not limited to: a high-frequency degradation model, for acquiring high-quality training images, and for degrading the high-quality training images to acquire corresponding low-quality training images;

[0016] The training detailed semantic information acquisition module is used to input the low-quality training image into a pre-trained image quality evaluation large language model to obtain the corresponding training detailed semantic information.

[0017] The high-frequency training restored image determination module retrieves the high-frequency training model and inputs the low-quality training image and the detailed semantic information of the training into the high-frequency training model to determine the corresponding high-frequency training restored image.

[0018] The high-frequency restoration model determination module determines the corresponding training loss function between the high-quality restored image and the high-frequency restored image, modifies the parameters in the high-frequency training model based on the training loss function, and repeatedly obtains the training process of the high-frequency restored image until the high-frequency training model training converges, which is used to determine the corresponding high-frequency restoration model.

[0019] In one exemplary embodiment, the apparatus includes, but is not limited to, the gradient descent method used for training both the low-frequency restoration model and the high-frequency restoration model.

[0020] In one exemplary embodiment, the apparatus includes, but is not limited to:

[0021] The interaction module is used to interact the degradation information and semantic information with the user and obtain the corresponding prior data, wherein the prior data is the degradation information and semantic information modified by the user.

[0022] In an exemplary embodiment, during the process of retrieving the pre-trained low-frequency restoration model and high-frequency restoration model, inputting the low-quality image, degradation information, and simplified semantic information into the low-frequency restoration model, and obtaining the corresponding low-frequency restored image, the apparatus includes, but is not limited to:

[0023] The evaluation information acquisition module is used to input the low-frequency restored image into a pre-trained image quality evaluation big language model to obtain the image quality evaluation information of the corresponding low-frequency restored image, wherein the image quality evaluation information includes preliminary semantic information;

[0024] The detailed semantic information determination module interacts with the user with the preliminary semantic information and determines the corresponding detailed semantic information based on whether the user makes modifications.

[0025] According to one aspect of this application, an electronic device includes at least one processor and at least one memory, wherein computer-readable instructions are stored on the memory; the computer-readable instructions are executed by one or more of the processors to cause the electronic device to implement the serial model-based restoration method as described above.

[0026] According to one aspect of this application, a storage medium stores computer-readable instructions thereon, which are executed by one or more processors to implement the serial model-based restoration method as described above.

[0027] According to one aspect of this application, a computer program product includes computer-readable instructions stored in a storage medium, wherein one or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, causing the electronic device to implement the serial model-based restoration method as described above.

[0028] The beneficial effects of the technical solution provided in this application are as follows: In the above technical solution, a low-quality image is acquired, and the low-quality image is input into a pre-trained image quality evaluation large language model to obtain the degradation information and semantic information corresponding to the low-quality image. A pre-trained low-frequency restoration model and a high-frequency restoration model are retrieved, and the low-quality image, degradation information, and simplified semantic information are input into the low-frequency restoration model to obtain the corresponding low-frequency restored image. The low-frequency restored image and the detailed semantic information are input into the high-frequency restoration model to determine the corresponding target restored image. In the above process, by obtaining degradation information and semantic information through the image quality evaluation large language model and inputting them into the corresponding low-frequency restoration model, the low-quality image can be restored in the low-frequency dimension first, and then the low-frequency restored image can be input into the high-frequency restoration model for restoration. This achieves restoration of the low-frequency restoration model in the high-frequency dimension, restoring low-frequency and high-frequency information respectively, and achieving high-fidelity image restoration. This effectively solves the problem of low image restoration quality in related technologies. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram based on the implementation environment involved in this application;

[0031] Figure 2 This is a flowchart illustrating a restoration method based on a serial model according to an exemplary embodiment;

[0032] Figure 3 This is a flowchart of steps S111 to S113 in a restoration method based on a serial model, according to an exemplary embodiment.

[0033] Figure 4 This is a flowchart of S121 to S124 in a restoration method based on a serial model, according to an exemplary embodiment.

[0034] Figure 5 This is a structural block diagram of a restoration device based on a series model, according to an exemplary embodiment.

[0035] Figure 6 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0036] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0037] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this disclosure means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0038] As can be seen from the above, the related technologies still suffer from the drawback of low accuracy in restoration based on the serial model.

[0039] Therefore, the cascade model-based restoration method provided in this application can effectively improve the accuracy of cascade model-based restoration. Accordingly, the cascade model-based restoration method is applicable to cascade model-based restoration devices, which can be deployed in electronic devices.

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0041] Figure 1 This is a schematic diagram of the implementation environment involved in a restoration method based on a serial model. The implementation environment includes a data acquisition terminal and a server terminal.

[0042] Please see Figure 2 This application provides a restoration method based on a series model, applicable to electronic devices, which can be... Figure 1 The server in the implementation environment is shown.

[0043] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0044] like Figure 2 As shown, the method may include the following steps:

[0045] S100: Obtain low-quality images and input them into a pre-trained image quality evaluation language model to obtain the degradation information and semantic information corresponding to the low-quality images.

[0046] Semantic information includes detailed semantic information and simplified semantic information.

[0047] It's important to note that multimodal large-scale models are large neural network models capable of processing and understanding various types of data. Traditional large-scale language models primarily handle text data, but multimodal large-scale models can simultaneously process multiple data types such as text, images, and audio, thus better understanding and generating complex cross-modal information. These models are typically based on deep learning techniques, combining technologies from fields such as Natural Language Processing (NLP) and Computer Vision (CV). By training these models, they can learn rich semantic and contextual information from different types of data, enabling broader applications such as natural language description of image content, image understanding, and speech recognition. Image quality assessment large-scale language models are a type of multimodal large-scale model, specifically designed for image quality assessment. Inputting an image into an image quality assessment large-scale language model allows it to recognize the image and output corresponding degradation and semantic information. The information output by the image quality assessment large-scale language model varies depending on the specific problem being addressed.

[0048] S110: Retrieve the pre-trained low-frequency restoration model and high-frequency restoration model, input the low-quality image, degradation information and simplified semantic information into the low-frequency restoration model, and obtain the corresponding low-frequency restored image.

[0049] After obtaining the corresponding degradation information and voice information, the degradation information and semantic information can be exchanged with the user. After receiving the corresponding degradation information and semantic information, the user can modify the degradation information and semantic information, and then upload the modified degradation information and semantic information to the system. The system can then obtain the corresponding prior data, which is the user-modified degradation information and semantic information.

[0050] By acquiring prior data, the accuracy of degradation and semantic information can be improved after user modifications during the low-frequency restoration of low-quality images, thereby enhancing the accuracy of low-quality image restoration in the low-frequency restoration stage.

[0051] Here, it should be noted that low-frequency restoration models require prior training and the collection of corresponding training data. Furthermore, low-frequency restoration models primarily focus on restoring low-frequency elements of an image. Conversely, high-frequency restoration models perform high-frequency restoration on the image restored by the low-frequency restoration model. For example, Figure 3 As shown, the specific process for the low-frequency restoration model in this application is as follows:

[0052] S111, acquire high-quality training images, degrade the high-quality training images and acquire corresponding low-quality training images and training degradation information.

[0053] It should be noted that high-quality images can be found on some copyrighted open-source photography websites by filtering out high-quality photographs. When degrading high-quality images, existing degradation models can be used. The high-quality training images can be input into the degradation model, which will randomly apply one or more of the following to the high-quality training images: noise reduction, blurring, sharpening, color quantization, compression, and resolution adjustment, thereby obtaining the corresponding low-quality training images.

[0054] In addition, it is important to emphasize that the training degradation information here refers to the specific operations performed by the degradation model on high-quality training images at low frequencies, such as noise reduction, blurring, sharpening, color quantization, compression, and resolution adjustment, or one or more of these operations. The training degradation information records the specific content of the degradation.

[0055] S112, retrieve the low-frequency training model, input the low-quality training image and training degradation information into the low-frequency training model to determine the corresponding low-frequency training restoration image.

[0056] In the training process of the low-frequency training model, low-quality training images and training degradation information are input into the low-frequency training images together. At this time, the low-frequency training model can output the corresponding low-frequency training restoration image. However, the obtained low-frequency training restoration image may not meet the required restoration quality, and the parameters of the low-frequency training model need to be continuously adjusted.

[0057] S113, determine the corresponding training loss function for the high-quality restored image and the low-frequency training restored image, modify the parameters in the low-frequency training model based on the training loss function, and repeatedly obtain the training process of the low-frequency training restored image until the low-frequency training model training converges, and determine the corresponding low-frequency restoration model.

[0058] During the training process, gradient descent is used. Once the training loss function of the low-frequency training model meets the training requirements, it can be retrieved and applied.

[0059] S120: Input the low-frequency restored image and detailed semantic information into the high-frequency restored model to determine the corresponding target restored image.

[0060] First, similar to the low-frequency restoration model, a high-frequency restoration model needs to be trained first, such as... Figure 4 As shown, the specific process is as follows:

[0061] S121, Obtain high-quality training images, degrade the high-quality training images and obtain the corresponding low-quality training images;

[0062] The low-quality training images used for training the high-frequency restoration model are different from those used for the low-frequency restoration model. Here, the low-quality training images are the low-quality training images that have been degraded from the high-quality training images. After being restored by the low-frequency restoration model, the images are used as low-quality training images for training the high-frequency restoration model. Although they are both named as low-quality training images, the images used as training sets are different, and the corresponding acquisition methods are also different.

[0063] S122, Input the low-quality training image into the pre-trained image quality evaluation large language model to obtain the corresponding detailed semantic information of the training;

[0064] Among them, the detailed semantic information of training records more detailed degradation data than the simplified semantic information. As mentioned earlier, the low-quality training image here is the low-frequency restored image, and the detailed semantic information of training is the content after evaluating the image quality of the low-frequency restored model.

[0065] S123, retrieve the high-frequency training model, input the low-quality training image and detailed semantic information of training into the high-frequency training model to determine the corresponding high-frequency training restoration image;

[0066] S124, determine the corresponding training loss function for the high-quality restored image and the high-frequency restored image, modify the parameters in the high-frequency training model based on the training loss function, and repeat the training process of obtaining the high-frequency restored image until the high-frequency training model training converges, and determine the corresponding high-frequency restoration model.

[0067] Similar to the low-frequency restoration model, the high-frequency restoration model also uses gradient descent during training. Correspondingly, when applying the high-frequency restoration model, it is necessary to obtain the low-frequency restored image and its corresponding detailed semantic information. The specific process for obtaining the detailed semantic information is as follows:

[0068] S125, input the low-frequency restored image into the pre-trained image quality evaluation large language model to obtain the image quality evaluation information of the corresponding low-frequency restored image.

[0069] Among them, the image quality evaluation information includes preliminary semantic information, that is, the image quality evaluation of low-frequency restored images by the image quality evaluation big language model.

[0070] S126, Initial semantic information is exchanged with the user, and the corresponding detailed semantic information is determined based on whether the user makes modifications.

[0071] By exchanging preliminary semantic information with the user, on the one hand, the user needs to judge whether there are any errors in the preliminary semantic information and make timely corrections; on the other hand, according to the user's specific needs, a specific aspect of the low-frequency restored image can be repeatedly restored, such as noise, blurring, sharpening, color quantization, compression, resolution adjustment or one or more of these.

[0072] The following are embodiments of the apparatus described in this application, which can be used to execute the restoration method based on the serial model involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the restoration method based on the serial model involved in this application.

[0073] Please see Figure 5 This application provides a restoration device based on a series model, which includes, but is not limited to:

[0074] The information acquisition module 200 acquires low-quality images and inputs the low-quality images into a pre-trained image quality evaluation large language model to obtain the degradation information and semantic information corresponding to the low-quality images. The semantic information includes detailed semantic information and simplified semantic information.

[0075] The low-frequency restored image acquisition module 210 retrieves the pre-trained low-frequency restoration model and high-frequency restoration model, and inputs the low-quality image, degradation information and simplified semantic information into the low-frequency restoration model to acquire the corresponding low-frequency restored image.

[0076] The target restored image determination module 220 inputs the low-frequency restored image and the detailed semantic information into the high-frequency restoration model to determine the corresponding target restored image.

[0077] In one exemplary embodiment, the apparatus includes, but is not limited to:

[0078] The low-frequency degradation module 300 acquires high-quality training images and degrades the high-quality training images to acquire corresponding low-quality training images and training degradation information.

[0079] The low-frequency training restoration image determination module 310 retrieves the low-frequency training model and uses it to input the low-quality training image and the training degradation information into the low-frequency training model to determine the corresponding low-frequency training restoration image.

[0080] The low-frequency training module 320 is used to determine the corresponding training loss function between the high-quality restored image and the low-frequency training restored image, modify the parameters in the low-frequency training model based on the training loss function, and repeatedly obtain the training process of the low-frequency training restored image until the low-frequency training model training converges, and determine the corresponding low-frequency restoration model.

[0081] In one exemplary embodiment, the apparatus includes, but is not limited to:

[0082] The high-frequency degradation model 400 obtains high-quality training images, which are then used to degrade the high-quality training images and obtain corresponding low-quality training images.

[0083] The training detailed semantic information acquisition module 410 is used to input the low-quality training image into a pre-trained image quality evaluation large language model to obtain the corresponding training detailed semantic information.

[0084] The high-frequency training restored image determination module 420 retrieves the high-frequency training model and inputs the low-quality training image and the detailed semantic information of the training into the high-frequency training model to determine the corresponding high-frequency training restored image.

[0085] The high-frequency restoration model determination module 430 determines the corresponding training loss function between the high-quality restored image and the high-frequency restored image, modifies the parameters in the high-frequency training model based on the training loss function, and repeatedly obtains the training process of the high-frequency restored image until the high-frequency training model training converges, thereby determining the corresponding high-frequency restoration model.

[0086] In one exemplary embodiment, the apparatus includes, but is not limited to, the gradient descent method used for training both the low-frequency restoration model and the high-frequency restoration model.

[0087] In one exemplary embodiment, the apparatus includes, but is not limited to:

[0088] The interaction module 500 is used to interact the degradation information and semantic information with the user and obtain the corresponding prior data, wherein the prior data is the degradation information and semantic information modified by the user.

[0089] In an exemplary embodiment, during the process of retrieving the pre-trained low-frequency restoration model and high-frequency restoration model, inputting the low-quality image, degradation information, and simplified semantic information into the low-frequency restoration model, and obtaining the corresponding low-frequency restored image, the apparatus includes, but is not limited to:

[0090] The evaluation information acquisition module 600 is used to input the low-frequency restored image into a pre-trained image quality evaluation big language model to obtain the image quality evaluation information of the corresponding low-frequency restored image, wherein the image quality evaluation information includes preliminary semantic information;

[0091] The detailed semantic information determination module 610 interacts with the user to exchange the preliminary semantic information, and determines the corresponding detailed semantic information based on whether the user makes modifications.

[0092] It should be noted that the above embodiments of the restoration device based on the serial model are only illustrated by the division of the above functional modules when performing restoration based on the serial model. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the restoration device based on the serial model will be divided into different functional modules to complete all or part of the functions described above.

[0093] Furthermore, the restoration device based on the series model and the restoration method based on the series model provided in the above embodiments belong to the same concept. The specific way in which each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0094] Please see Figure 6 This application provides an electronic device 4000, which may include: a desktop computer, a laptop computer, a server, etc.

[0095] exist Figure 6 In this context, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0096] The data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0097] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0098] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0099] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 400, but not limited thereto.

[0100] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002.

[0101] The computer-readable instructions are executed by one or more processors 4001 to implement the serial model-based restoration method in the above embodiments.

[0102] Furthermore, this application provides a storage medium storing computer-readable instructions that are executed by one or more processors to implement the serial model-based restoration method described above.

[0103] This application provides a computer program product including computer-readable instructions stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the restoration method based on the serial model as described above.

[0104] Compared with related technologies, the high-frequency restoration model and low-frequency restoration model used in this application, during the restoration process, obtain semantic and degradation information. After the semantic and degradation information is interacted with the user, the user judges whether there is an error in the evaluation information. If there is no error, the semantic and degradation information can be directly input into the high-frequency restoration model and the low-frequency restoration model. Otherwise, the evaluation information with errors can be modified, and the modified semantic information can be input into the restoration model to further guide the restoration model in the restoration process of low-quality images. In addition, by restoring low-frequency and high-frequency information separately, high-fidelity image restoration can be achieved, thereby effectively solving the problem of low image restoration quality in related technologies.

[0105] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0106] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A restoration method based on a series model, characterized in that, include: A low-quality image is acquired, and the low-quality image is input into a pre-trained image quality evaluation large language model to obtain the degradation information and semantic information corresponding to the low-quality image. The semantic information includes detailed semantic information and simplified semantic information. Retrieve the pre-trained low-frequency restoration model and high-frequency restoration model, input the low-quality image, degradation information and simplified semantic information into the low-frequency restoration model, and obtain the corresponding low-frequency restored image; The low-frequency restored image and the detailed semantic information are input into the high-frequency restoration model to determine the corresponding target restored image; The training of the low-frequency restoration model also includes: Obtain high-quality training images, degrade the high-quality training images, and obtain the corresponding low-quality training images and training degradation information; Retrieve the low-frequency training model, and input the low-quality training image and the training degradation information into the low-frequency training model to determine the corresponding low-frequency training restoration image; The training loss function is determined by comparing the high-quality restored image with the low-frequency training restored image. The parameters in the low-frequency training model are modified based on the training loss function. The training process of obtaining the low-frequency training restored image is repeated until the low-frequency training model training converges, and the corresponding low-frequency restoration model is determined. The method further includes: Obtain high-quality training images, degrade the high-quality training images, and obtain corresponding low-quality training images; The low-quality training images are input into a pre-trained image quality evaluation large language model to obtain the corresponding detailed semantic information of the training. Retrieve the high-frequency training model, and input the low-quality training image and the detailed semantic information of the training into the high-frequency training model to determine the corresponding high-frequency training restoration image; The high-quality restored image and the high-frequency training restored image are used to determine the corresponding training loss function. Based on the training loss function, the parameters in the high-frequency training model are modified. The training process of obtaining the high-frequency restored image is repeated until the high-frequency training model training converges, and the corresponding high-frequency restoration model is determined.

2. The method as described in claim 1, characterized in that, The method further includes: Both the low-frequency restoration model and the high-frequency restoration model are trained using the gradient descent method.

3. The method as described in claim 1, characterized in that, The method further includes: The degradation information and semantic information are interacted with the user to obtain the corresponding prior data, wherein the prior data is the degradation information and semantic information modified by the user.

4. The method as described in claim 1, characterized in that, After retrieving the pre-trained low-frequency restoration model and high-frequency restoration model, inputting the low-quality image, degradation information, and simplified semantic information into the low-frequency restoration model to obtain the corresponding low-frequency restored image, the process further includes: The low-frequency restored image is input into a pre-trained image quality evaluation big language model to obtain the image quality evaluation information of the corresponding low-frequency restored image, wherein the image quality evaluation information includes preliminary semantic information; The preliminary semantic information is presented to the user, and the corresponding detailed semantic information is determined based on whether the user makes any modifications.

5. A restoration device based on a series model, characterized in that, include: The information acquisition module acquires low-quality images and inputs them into a pre-trained image quality evaluation language model to obtain degradation information and semantic information corresponding to the low-quality images. The semantic information includes detailed semantic information and simplified semantic information. The low-frequency restored image acquisition module retrieves the pre-trained low-frequency restoration model and high-frequency restoration model, and inputs the low-quality image, degradation information and simplified semantic information into the low-frequency restoration model to obtain the corresponding low-frequency restored image. The target restored image determination module inputs the low-frequency restored image and the detailed semantic information into the high-frequency restoration model to determine the corresponding target restored image; The low-frequency degradation module acquires high-quality training images, degrades the high-quality training images, and acquires the corresponding low-quality training images and training degradation information. The low-frequency training restoration image determination module retrieves the low-frequency training model and uses the low-quality training image and the training degradation information to input the low-frequency training model to determine the corresponding low-frequency training restoration image. The low-frequency training module is used to determine the corresponding training loss function between the high-quality restored image and the low-frequency training restored image, modify the parameters in the low-frequency training model based on the training loss function, and repeatedly obtain the training process of the low-frequency training restored image until the low-frequency training model training converges, and determine the corresponding low-frequency restoration model. A high-frequency degradation model is used to obtain high-quality training images, which are then used to degrade the high-quality training images and obtain corresponding low-quality training images. The training detailed semantic information acquisition module is used to input the low-quality training image into a pre-trained image quality evaluation large language model to obtain the corresponding training detailed semantic information. The high-frequency training restored image determination module retrieves the high-frequency training model and inputs the low-quality training image and the detailed semantic information of the training into the high-frequency training model to determine the corresponding high-frequency training restored image. The high-frequency restoration model determination module determines the corresponding training loss function between the high-quality restored image and the high-frequency restored image, modifies the parameters in the high-frequency training model based on the training loss function, and repeatedly obtains the training process of the high-frequency restored image until the high-frequency training model training converges, which is used to determine the corresponding high-frequency restoration model.

6. The apparatus as claimed in claim 5, characterized in that, The device further includes: The low-frequency degradation module acquires high-quality training images, degrades the high-quality training images, and acquires corresponding low-quality training images and training degradation information. The low-frequency training restoration image determination module retrieves the low-frequency training model and uses the low-quality training image and the training degradation information to input the low-frequency training model to determine the corresponding low-frequency training restoration image. The low-frequency training module is used to determine the corresponding training loss function between the high-quality restored image and the low-frequency training restored image, modify the parameters in the low-frequency training model based on the training loss function, and repeatedly obtain the training process of the low-frequency training restored image until the low-frequency training model training converges, and determine the corresponding low-frequency restoration model.

7. An electronic device, characterized in that, include: At least one processor and at least one memory, wherein, The memory stores computer-readable instructions; The computer-readable instructions are executed by one or more of the processors, causing the electronic device to implement the restoration method based on the serial model as described in any one of claims 1 to 4.

8. A storage medium having computer-readable instructions stored thereon, characterized in that, The computer-readable instructions are executed by one or more processors to implement the restoration method based on the serial model as described in any one of claims 1 to 4.

Images