Visual agent self-adaptive recovery method and system based on physical degradation prior

Abstract

This invention discloses an adaptive restoration method and system based on physical degradation priors using visual intelligent agents, relating to the fields of image processing and computer vision. The method includes: using a multimodal large model to perceive degradation in an input degraded image, identifying degradation types and assessing the degree of degradation; classifying degradation hierarchically according to the physical degradation patterns of real images, and generating a phased restoration plan with prior constraints in reverse order of degradation generation; determining the execution order of degradation expert agents through a scheduling agent for overall planning, with the degradation expert agents matching restoration tools to complete round-by-round image restoration; adaptively updating the restoration strategy based on feedback from no-reference image quality evaluation, and outputting the final restored image, degradation perception results, and restoration plan after iteration termination, thus completing the image restoration task. This invention achieves hybrid degradation recognition through a structured multi-degradation perception mechanism, solving the problem of low accuracy in degradation diagnosis in complex scenes.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to an adaptive restoration method and system for visual intelligent agents based on physical degradation priors. Background Technology

[0002] Visual image restoration is a core fundamental task in the field of computer vision. Its core technical goal is to effectively restore clear, complete, and high-fidelity high-quality original images from degraded images affected by various factors. In real-world and complex application scenarios, images commonly suffer from multiple mixed degradation problems, rather than being affected by a single degradation factor. Specifically, images are prone to simultaneously superimposed with multiple degradation types such as noise, motion blur, low-light distortion, resolution loss, rain and fog interference, and JPEG compression artifacts. These various degradations are coupled and interfere with each other, forming a complex composite degradation distribution, which significantly increases the difficulty of image feature extraction, degradation identification, and accurate restoration, making traditional fixed-pattern restoration methods unsuitable for the complex restoration needs of real-world scenarios.

[0003] Currently, existing image restoration technologies are mainly divided into two categories: single-task dedicated model methods and all-in-one model methods that unify all tasks. Both types of technologies have inherent technical defects, failing to balance restoration accuracy and adaptability in complex scenes. Dedicated restoration models are designed and optimized for a single degradation type, exhibiting good restoration performance and adaptability in single degradation scenarios. However, they have significant limitations when facing multiple mixed degradations in real-world scenes. These methods rely on human experience to determine degradation types, select restoration models, and combine models in the correct order. The entire process involves high levels of human intervention and low automation, lacking dynamic adaptive adjustment capabilities and making it difficult to adapt to complex scenes where degradation types are unknown and degradation levels dynamically change.

[0004] The All-in-One unified restoration model achieves unified processing of multiple degradation types by constructing a single general network structure, effectively improving the model's scenario versatility and avoiding the tedious operation of manually combining multiple models. However, this technical solution still has core technical shortcomings. Due to the significant differences in optimization objectives, feature constraints, and repair logic among different degradation types, problems such as target coupling, gradient conflicts, and feature interference are prone to occur during the joint optimization of multiple degradation types. This makes it difficult for the model to take into account the repair accuracy of various degradation types and balance generalization performance and restoration effect. At the same time, this type of model is highly dependent on the degradation distribution of the training dataset, and has poor ability to match unseen degradations and complex mixed degradation combinations outside the training distribution, resulting in insufficient fault tolerance and robustness.

[0005] From an overall technical paradigm perspective, existing mainstream image restoration methods all employ static processing mechanisms without intelligent agents. They perform single-pass forward inference restoration of the input degraded image using a single network model or a pre-set fixed process, lacking the ability to dynamically perceive, globally plan, and control the process for complex mixed degradation. In complex scenarios with multiple coupled and overlapping degradations, these static processing mechanisms cannot accurately decompose and quantify the degradation composition and intensity of the image, and lack the ability to dynamically adjust the restoration strategy based on intermediate restoration states. The selection of restoration tools and the sequence of restoration processes rely on manual presets or fixed rules, resulting in a rigid restoration path and extremely poor flexibility. When facing real-world scenarios with complex degradation combinations and significant deviations between degradation distribution and training samples, technical problems such as incomplete restoration, loss of image details, amplification of existing artifacts, and new distortions are highly likely to occur, severely limiting the practical application of image restoration technology in complex real-world scenarios.

[0006] Meanwhile, image degradation in real-world scenarios is not a random and disordered superposition of various degradations, but rather follows fixed physical imaging laws, generated step-by-step in a fixed temporal order of scene degradation, imaging degradation, and compression degradation. Existing restoration techniques generally ignore the objective physical prior mechanisms of image degradation, blindly searching and repairing within a vast tool combination space without physical constraints. This not only generates a large number of invalid repair operations and increases computational redundancy, but also leads to problems where the restoration order violates the degradation generation mechanism, resulting in logical disorder in the restoration process, incomplete degradation removal, and a significant reduction in the final image restoration quality.

[0007] In recent years, multimodal large-scale models and intelligent agent technologies have developed rapidly. Based on their powerful cross-modal feature understanding, logical reasoning, and dynamic decision-making capabilities, they have provided a new approach to breaking through the technical bottlenecks of the traditional static restoration paradigm. Introducing visual intelligent agents with independent perception, planning, execution, and iterative decision-making capabilities into image restoration tasks can transform the traditional static, single-inference restoration mode into a dynamic perception, step-by-step planning, and iterative optimization intelligent decision-making restoration mode, which can update the restoration strategy in real time according to the image degradation status and restoration progress.

[0008] In summary, to address the technical problems of existing technologies, such as low accuracy of hybrid degradation perception, unconstrained restoration paths, rigid repair strategies, poor adaptability, weak interpretability, and unsatisfactory restoration results, there is an urgent need for a visual intelligent agent image restoration method that integrates physical degradation priors and has dynamic adaptive decision-making capabilities, so as to achieve accurate, efficient, and interpretable intelligent image restoration in complex hybrid degradation scenarios. Summary of the Invention

[0009] The main objective of this invention is to provide an adaptive restoration method for visual intelligent agents based on physical degradation priors.

[0010] Another objective of this invention is to propose an adaptive restoration system for visual intelligent agents based on physical degradation priors.

[0011] The third objective of this invention is to provide an electronic device.

[0012] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0013] To achieve the above objectives, a first aspect of the present invention proposes an adaptive restoration method for visual intelligent agents based on physical degradation priors, comprising:

[0014] A multimodal large model is used to perform degradation perception on the input degraded image. By identifying each degradation and assessing its intensity, the degradation type and its corresponding degree of degradation information are obtained, and degradation perception results are generated. Based on the real-world image degradation formation process, the degradation types in the degradation perception results are mapped to scene degradation, imaging degradation, and compression degradation, and a phased restoration plan with prior constraints is generated according to the reverse order of the physical sequence of the degradation generation process. Input the degraded image, user instructions, degradation perception results, phased restoration plan, and experience information into the scheduling agent, which generates the overall restoration plan and determines the execution order of the degradation expert agents; According to the overall restoration plan, the corresponding degradation expert agents are activated sequentially. The degradation expert agents combine the current image state, degradation degree and tool registration information to select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results. After each round of restoration is completed, the intermediate restoration results are evaluated using no-reference image quality assessment. The subsequent restoration operation strategy is adaptively updated based on the changes in image quality. The process is iterated until the restoration termination condition is met, and the final restored image, degradation perception result, and overall restoration plan are output to complete the image restoration task.

[0015] Optionally, a multimodal large model is used to perform degradation perception on the input degraded image. Through item-by-item degradation identification and intensity evaluation, degradation type and its corresponding degree information are obtained, and degradation perception results are generated, including: The multimodal large model is used to analyze the input degraded image, and the hybrid degradation perception task is decomposed into multiple single degradation recognition sub-tasks. The existence of each type of degradation is detected in a pre-defined order to determine whether the corresponding degradation type exists in the degraded image. For the types of degradation identified by the detection, further quantitative assessment of the degradation intensity will be conducted to obtain information on the degree of degradation corresponding to each type of degradation; Integrate the identification results and intensity data of all degradation types to generate complete degradation perception results.

[0016] Optionally, the degradation types in the degradation perception results are mapped to scene degradation, imaging degradation, and compression degradation, and a phased restoration plan with prior constraints is generated according to the reverse order of the physical sequence of the degradation generation process, including: Based on the physical formation order of real-world image degradation, the degradation types in the degradation perception results are classified into three physical levels: scene degradation, imaging degradation, and compression degradation. The restoration order is constrained according to the order of the degradation generation process, so that the restoration process is executed in reverse order of compression degradation, imaging degradation, and scene degradation. The unconstrained restoration path search is transformed into a constrained search process under physical prior constraints, generating a phased restoration plan with prior constraints.

[0017] Optionally, the degraded image, user instructions, degradation perception results, phased restoration plan, and experience information are input into the scheduling agent, which generates an overall restoration plan and determines the execution order of the degradation expert agents, including: The input degraded images, user commands, degradation perception results, phased restoration plans, and historical experience information are all uniformly input into the scheduling agent; By utilizing a scheduling agent that combines physical degradation priors with various input information, the entire image restoration process can be comprehensively planned and task decomposed. Based on the phased division of the phased restoration plan, the execution order of different degradation expert agents is arranged. Integrate the overall planning content with the execution order of the intelligent agent to output an overall restoration plan adapted to the current image restoration task.

[0018] Optionally, according to the overall restoration plan, the corresponding degradation expert agents are activated sequentially. These agents, combining the current image state, degradation level, and tool registration information, select candidate restoration tools for progressive restoration processing and output intermediate restoration results, including: According to the execution order determined in the overall restoration plan, the degradation expert agents corresponding to the current degradation type are activated sequentially. The activated degradation expert agent reads the current image state, the corresponding degree of degradation, and tool registration information in real time; By combining the current state of the image with its degradation features, candidate restoration tools that match the restoration needs are selected from the toolset; The selected candidate restoration tool is invoked to perform round-by-round restoration processing on the image, and the corresponding intermediate restoration result is output after each round of processing.

[0019] Optionally, the intermediate restoration results are evaluated using a no-reference image quality assessment, and the subsequent restoration strategy is adaptively updated based on changes in image quality, including: After each round of image restoration processing is completed, multiple no-reference image quality evaluation indicators are used to quantitatively evaluate the intermediate restoration results generated in this round. Based on a greedy strategy for quality improvement, the image quality changes brought about by different candidate restoration operations are compared, and the restoration operation that can maximize the improvement of image quality is selected as the next execution strategy, thereby achieving adaptive updating of the restoration operation strategy. The process involves iteratively performing recovery processing, quality assessment, and strategy updates, while continuously verifying whether the recovery termination conditions are met.

[0020] Optionally, the restoration termination condition includes whether all degradation in the degradation perception result has been repaired or whether multiple consecutive restoration operations have failed to improve image quality. When any of the termination conditions is triggered, all restoration iteration operations will be stopped immediately.

[0021] To achieve the above objectives, a second aspect of the present invention proposes a visual intelligent agent adaptive restoration system based on physical degradation priors, comprising: The degradation perception module is used to perform degradation perception on the input degraded image using a multimodal large model. Through item-by-item degradation identification and intensity evaluation, it obtains information on the degradation type and its corresponding degree of degradation, and generates degradation perception results. The prior planning module is used to map the degradation type in the degradation perception result to scene degradation, imaging degradation and compression degradation based on the real-world image degradation formation process, and generate a phased restoration plan with prior constraints according to the reverse order of the physical order of the degradation generation process. The intelligent scheduling module is used to input the degraded image, user instructions, degradation perception results, phased restoration plan and experience information into the scheduling agent, which generates the overall restoration plan and determines the execution order of the degradation expert agent. The execution restoration module is used to sequentially activate the corresponding degradation expert agents according to the overall restoration plan. The degradation expert agents combine the current image state, degradation degree and tool registration information to select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results. The iterative decision module is used to evaluate the intermediate restoration results using no-reference image quality assessment after each round of restoration, adaptively update the subsequent restoration operation strategy based on image quality changes, iterates until the restoration termination condition is met, and outputs the final restored image, degradation perception result and overall restoration plan to complete the image restoration task.

[0022] Regarding the system in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0023] To achieve the above objectives, a third aspect of this application provides an electronic device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing a visual intelligent agent adaptive restoration method based on physical degradation prior as described in the first aspect embodiment.

[0024] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements a visual intelligent agent adaptive restoration method based on physical degradation priors as described in the first aspect embodiment.

[0025] The embodiments of the present invention have the following beneficial effects: 1. This invention constructs a structured multi-degradation perception mechanism. By splitting single degradation identification subtasks and quantifying degradation intensity, it accurately identifies the type and degree of complex mixed degradation, effectively improving the accuracy and reliability of image degradation diagnosis in complex scenes.

[0026] 2. This invention introduces a priori physical degradation of real images to generate reverse-order constraints on the restoration path, which greatly reduces the search space of restoration paths and tools, avoids the distortion and artifact problems caused by disordered restoration, and improves the rationality and stability of restoration planning.

[0027] 3. This invention adopts a hierarchical collaborative architecture of scheduling intelligent agent and degradation expert intelligent agent to achieve decoupled execution of global planning and fine-grained repair of restoration tasks, breaking through the limitations of traditional static fixed repair process, and can adaptively adapt to the restoration needs of various hybrid degradation scenarios.

[0028] 4. This invention establishes a closed-loop iterative mechanism for quality evaluation feedback, which adaptively adjusts the restoration strategy based on changes in image quality and terminates invalid iterations, effectively reducing redundant calculations, improving restoration efficiency and repair quality, and outputting a complete degradation diagnosis and restoration plan, significantly improving the interpretability and practicality of the restoration process. Attached Figure Description

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating an adaptive restoration method for visual intelligent agents based on physical degradation priors, provided in an embodiment of the present invention; Figure 2This is a structural diagram of a visual intelligent agent adaptive restoration system based on physical degradation prior, provided in an embodiment of the present invention. Detailed Implementation

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] The following description, with reference to the accompanying drawings, describes an adaptive restoration method and system for visual intelligent agents based on physical degradation priors, according to embodiments of the present invention.

[0033] Example 1 This invention provides an adaptive restoration method for visual intelligent agents based on physical degradation priors. Figure 1 This is a flowchart illustrating an adaptive restoration method for visual intelligent agents based on physical degradation priors, provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: Step S1: Use a multimodal large model to perform degradation perception on the input degraded image. Through item-by-item degradation identification and intensity evaluation, obtain the degradation type and its corresponding degradation degree information, and generate degradation perception results.

[0034] In this embodiment of the invention, after receiving a degraded image from an external input, a multimodal large model is invoked to perform comprehensive degradation perception processing on the image. Considering that images in practical applications often have multiple different types of degradation problems superimposed, direct overall recognition is prone to judgment errors. Therefore, this solution reasonably decomposes the overall multi-degradation recognition task into multiple independent single-degradation recognition subtasks. By adopting a divide-and-conquer approach, it better adapts to various complex mixed degradation scenarios and ensures the orderly progress of the recognition work.

[0035] In this embodiment of the application, a complete range of candidate degradations is defined in advance, and a candidate degradation set is constructed, the expression of which is:

[0036] in, The set of candidate degenerates representing the whole. Refers to the first in the set The specific candidate degradation type of the class, parameters This represents the total number of candidate degenerate categories.

[0037] Based on actual image degradation scenarios, the candidate degradation methods included in this scheme specifically include noise, motion blur, defocus blur, low illumination, low resolution, rain streaks, haze, and JPEG compression artifacts, basically covering common image degradation problems in everyday scenes.

[0038] During the formal identification phase, existence detection will be performed sequentially for each type of degradation within the candidate degradation set, following a pre-defined order. For any given candidate degradation... The existence judgment results are all output by the model's built-in perception module. The rules for determining the value of this parameter are as follows: Among them, when This indicates that such degradation does indeed exist in the degraded image currently being processed. ;when If the result is positive, it indicates that no such degradation problem has occurred in the image, and no further intensity assessment is required.

[0039] If the existence determination result shows that a certain type of degradation truly exists, that is, the parameter When the value is 1, the system will automatically proceed to the next evaluation stage, where a refined intensity quantification analysis will be performed on this type of degradation to ultimately obtain the corresponding degradation intensity level. The intensity level here. There are three levels: low, medium, and high, corresponding to low-level degradation, moderate-level degradation, and high-level degradation, respectively, to distinguish the severity of different degradation levels.

[0040] After the existence and intensity assessments of all categories of degradation have been completed, all valid identification data are summarized and integrated to form a complete degradation perception result. (This is for the input image.) The corresponding degradation perception result can be expressed as:

[0041] in, This is specifically used to represent the set of degradation-perceived results corresponding to the input image $X$, and the set contains multiple sets of data pairs. Composition, only including degenerate information that is determined to exist; conditions This means selecting only the actual types of degradation that appear in the image. The value range is from 1 to , represents traversing all candidate degeneracy categories.

[0042] By adopting a processing mode that first breaks down tasks, then identifies each item individually, and finally evaluates the intensity, the recognition error caused by mutual interference from mixed degradation can be effectively avoided, significantly improving the perception accuracy of multimodal large models in complex mixed degradation scenarios. At the same time, the complete degradation perception results output in this step can also provide detailed and reliable basic data support for all aspects of the image restoration process, including subsequent planning, strategy formulation, and restoration execution.

[0043] Step S2: Based on the real-world image degradation formation process, the degradation type in the degradation perception result is mapped to scene degradation, imaging degradation and compression degradation, and a phased restoration plan with prior constraints is generated according to the reverse order of the physical order of the degradation generation process.

[0044] After completing the degradation perception processing, the workflow officially enters step S2. This step combines the formation rules of image degradation that objectively exist in real-world scenes, using physical degradation priors as the core basis, to hierarchically classify the degradation perception results output from the previous step, thus building a reasonable execution framework for the subsequent restoration process.

[0045] In real-world environments, various forms of image degradation are not randomly superimposed, but rather follow a fixed generation order. The entire image degradation process can be represented by a formula, as follows:

[0046] in, Refers to the entire degradation process experienced by the image as a whole; Scene degradation is caused by the external environment in which the image was taken, and specifically includes three typical problems: rain streaks, fog and haze, and low light. Image degradation mainly occurs during the image capture and acquisition process, and includes four types of issues: noise, motion blur, defocus blur, and low resolution. This refers to compression degradation, specifically problems caused by post-processing compression of images, and corresponds to JPEG compression artifacts.

[0047] Following the physical generation logic of image degradation, image restoration operations need to proceed in the exact opposite order to degradation formation. Based on this principle, this application makes explicit constraints on the overall restoration process, and the corresponding restoration process expression is as follows:

[0048] Among them, symbols This represents the entire image restoration process. , , These are specific restoration operations corresponding to scene degradation, imaging degradation, and compression degradation, respectively.

[0049] Combining formulas and physical logic, it can be determined that the restoration work needs to be carried out in a strict order: first, compression degradation should be dealt with; second, imaging degradation should be repaired; and finally, scene degradation should be addressed.

[0050] In the actual operation of this embodiment, the degradation perception results obtained in step S1 are first retrieved. Each type of degradation identified in the results is then classified into three physical levels: scene degradation, imaging degradation, and compression degradation. After degradation classification, the overall restoration order is constrained by the reverse restoration rules determined earlier. Traditional image restoration methods often involve random path searches, which are not only computationally intensive but also prone to logically contradictory restoration processes. This solution, however, utilizes physical degradation priors to transform the unconstrained restoration path search into a directional search mode restricted by predetermined rules. This approach effectively reduces the search range of restoration paths, minimizes invalid computations, and fundamentally ensures that the restoration execution order conforms to physical laws and possesses sufficient rationality. After the above series of processes, a phased restoration plan with physical prior constraints can be generated.

[0051] Step S3: Input the degraded image, user instructions, degradation perception results, phased restoration plan, and experience information into the scheduling agent. The scheduling agent generates the overall restoration plan and determines the execution order of the degradation expert agents.

[0052] After the process progresses to step S3, multiple types of key information are collected and sent to the scheduling agent for processing. The specific inputs include the degraded image to be repaired, the operation instructions issued by the user in real time, the degradation perception results obtained through the aforementioned steps, the phased restoration plan with physical prior constraints, and the historical experience information accumulated during long-term operation. All types of data work together to provide a complete basis for the agent's decision-making.

[0053] This application adopts a hierarchical intelligent agent collaborative architecture, where different agents perform their respective functions and cooperate with each other to complete the entire image restoration process. Among them, the scheduling agent, as the core functional module of the first layer of the architecture, plays a key role in overall coordination and task planning, controlling the direction of the entire restoration process. After the scheduling agent has fully received all input information, it combines the physical degradation prior constraint rules established in step S2 with reference to past restoration experience to meticulously break down the entire image restoration task and coordinate the arrangement of various tasks from a global perspective.

[0054] The scheduling agent strictly follows the hierarchical and phased division requirements of the phased restoration plan, systematically reviewing each repair task and determining the execution order of each degradation expert agent. After comprehensive analysis and orderly arrangement, the scheduling agent finally generates and outputs a complete overall restoration plan. This plan is detailed, clearly specifying the exact order of invocation for different degradation expert agents and clearly defining the restoration goals to be achieved in each execution phase. This provides a clear execution framework for subsequent specialized restoration operations by each degradation expert agent, ensuring that subsequent repair work can proceed in a sequential and standardized manner.

[0055] Step S4: According to the overall restoration plan, the corresponding degradation expert agents are activated sequentially. The degradation expert agents combine the current image state, degradation degree and tool registration information to select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results.

[0056] Next, step S4 is executed, strictly following the execution order set in the overall restoration plan generated by the scheduling agent in step S3. Degradation expert agents matching the current degradation type to be repaired are activated sequentially. In the hierarchical collaborative architecture of this embodiment, each degradation expert agent corresponds to a single degradation category, possessing strong specialized repair capabilities. It is only responsible for completing the specialized repair work for the corresponding category of image degradation, effectively avoiding functional confusion and processing deviations that may occur during multi-degradation repair processes.

[0057] To achieve intelligent tool invocation and precise restoration, this application integrates and categorizes all independently callable image restoration tools into a standardized set of restoration tools. The expression for this tool set is:

[0058] in, This represents the complete set of preset restoration tools. Represents the total number of independent restoration tools. This specifically refers to the first item in the toolset. Each tool is a separate image restoration tool, and different restoration tools can be adapted to the image restoration needs of different types of degradation.

[0059] Once any degradation expert agent is activated, it automatically retrieves three types of core reference information in real time: image state data for the current iteration stage, degradation severity information corresponding to the current degradation to be repaired, and pre-entered and stored tool registration information. Based on this, the degradation expert agent intelligently selects candidate restoration tools from the global restoration tool set that fit the current image restoration scenario and match the corresponding degradation type, combining the actual degradation characteristics and severity of the current image with the adaptation attributes of each restoration tool, thus providing precise tool support for subsequent iterative restoration.

[0060] In the iterative repair process of this application embodiment, the current repair execution round is set to the th . A cycle is the original image state before the start of the current cycle of restoration, after previous cycles of restoration. The specific degradation type corresponding to the currently activated degradation expert agent is the one identified in the aforementioned steps. Based on the above parameters, the tool sequence consisting of all available candidate restoration tools obtained after this round of screening and matching can be represented as:

[0061] Simultaneously satisfy the constraints:

[0062] in, Specifically refers to the first During the restoration process, the current set of all candidate restoration tools that the degradation expert agent can call upon; Representing the The first candidate in the round of repair A restoration tool; Represents the total number of candidate restoration tools obtained in this round of screening; constraints. This indicates that all candidate restoration tools selected in this round belong to the adaptation-degeneration type. A dedicated subset of tools to ensure that the tools are perfectly matched to the degradation type.

[0063] After completing the selection of candidate restoration tools, this round of image restoration processing officially commenced. For any one restoration tool in this round of candidate tool set... Apply it to the images to be repaired in this round. After the image degradation restoration process is completed, entirely new image data can be generated. The corresponding image iterative update formula is:

[0064] in, Representative after the first The intermediate restoration result of the new image after processing by the wheel restoration tool.

[0065] The operation mode of continuously looping the above-mentioned screening tools, performing repairs, and outputting intermediate results is used to complete the progressive restoration of the image in an iterative manner, providing intermediate restored image data corresponding to each iteration for subsequent image quality assessment and adaptive strategy update.

[0066] Step S5: After each round of restoration is completed, the intermediate restoration results are evaluated using the quality assessment of the no-reference image. The subsequent restoration operation strategy is adaptively updated based on the changes in image quality. The process is iterated until the restoration termination condition is met, and the final restored image, degradation perception result and overall restoration plan are output to complete the image restoration task.

[0067] In this embodiment of the invention, after S4 completes each round of image restoration operation and outputs the corresponding intermediate restoration result, the image quality assessment and restoration strategy adaptive update process will be automatically started. The intelligent decision-making of iterative restoration will be realized according to the quantitative quality evaluation standard, thus getting rid of the limitations of the traditional fixed restoration order.

[0068] This application's embodiments abandon the one-sidedness of a single evaluation index, select five mainstream no-reference image quality evaluation indicators to jointly conduct comprehensive image quality assessment, and construct a dedicated set of evaluation indicators. The expression for the indicator set is as follows:

[0069] Among them, parameters Represents the NIQE no-reference image quality assessment metric. This represents the BRISQUE no-reference image quality assessment metric. This represents the image quality evaluation index for CPBD without reference. This represents the CLIP-IQA no-reference image quality evaluation index. The Hyper-IQA no-reference image quality evaluation index consists of five categories that work together to comprehensively quantify the visual effects of degraded and restored images from different dimensions.

[0070] Based on the above five evaluation indicators, this application constructs a proprietary formula for calculating the overall image quality value, the specific formula being:

[0071] in, Refers to the input image The corresponding overall quality score is the core quantitative value for measuring the image restoration effect; These correspond to the quantitative scores of the five categories of evaluation indicators without reference mentioned above.

[0072] The overall quality value calculated under the evaluation system rules of this application. The higher the value, the better the overall visual effect and restoration quality of the current image; conversely, the lower the value, the more obvious the degradation problem in the image, requiring further iterative repair.

[0073] Regarding the first During the round-robin restoration process, multiple intermediate restored images are obtained by processing different candidate restoration tools. It will iterate through all intermediate image samples and calculate the overall quality value for each image one by one. To achieve the optimal restoration decision, this application employs a quality-enhancing greedy strategy to select the optimal repair scheme for this round. The selection is completed using the optimal solution calculation formula, as follows:

[0074] in, For the first Round Intermediate images obtained by processing candidate tools This corresponds to the overall quality value of the intermediate image. The optimal tool number for maximizing the overall image quality score.

[0075] The optimal tool number is determined by calculating using the above formula. Then, the optimal restoration tool for this round can be identified. The image obtained after processing by this tool is used as the final repaired image for this round and assigned a value. This serves as the processing target for the next round of iterative restoration. Simultaneously, based on the changes in image quality throughout each round, the selection of subsequent restoration tools and the overall execution order of restoration are adaptively adjusted, achieving dynamic updates to the restoration strategy and significantly reducing ineffective repair operations.

[0076] During the entire iterative restoration process, the preset restoration termination conditions are continuously verified. The iteration process ends as long as any termination condition is met. This application embodiment sets two complete restoration termination criteria. The first criterion is termination upon complete restoration of degradation, and the corresponding determination expression is:

[0077] in, For the first The set of degradation-perceived results for the images in the round of iteration. Representing an empty set, this condition indicates that all previously identified degradation types in the current image have been repaired, and there are no remaining degradation tasks to be processed.

[0078] The second category is termination due to a quality bottleneck, and the corresponding judgment criteria are as follows:

[0079] in, This represents the overall quality value of the image after this round of restoration. This is the overall quality value of the original image before this round of restoration. This condition indicates that the new round of restoration operations cannot improve the image quality, and continued iterative repair cannot optimize the image effect, indicating a quality saturation bottleneck.

[0080] When any of the above-mentioned restoration termination conditions is triggered, all iterative restoration operations will immediately stop, ending the image restoration loop. The final restored image after iteration is then output, along with the degradation perception results and the complete overall restoration execution plan. By simultaneously outputting the restoration results and the entire decision data, the image degradation diagnosis process, restoration planning process, and iterative restoration process can be fully reconstructed, clearly demonstrating the decision logic and execution path of the entire solution. This greatly enhances the interpretability and credibility of this application, thus completing the entire adaptive image restoration task based on physical degradation priors for visual intelligent agents.

[0081] Example 2 This invention provides a visual intelligent agent adaptive restoration system based on physical degradation priors. Figure 2 This is a schematic diagram of the structure of a visual intelligent agent adaptive restoration system based on physical degradation prior, provided in an embodiment of the present invention. Figure 2 As shown, the system includes: The degradation perception module 100 is used to perform degradation perception on the input degraded image using a multimodal large model. Through item-by-item degradation identification and intensity evaluation, it obtains degradation type and corresponding degradation degree information, and generates degradation perception results. The prior planning module 200 is used to map the degradation type in the degradation perception result to scene degradation, imaging degradation and compression degradation based on the real-world image degradation formation process, and generate a phased restoration plan with prior constraints according to the reverse order of the physical order of the degradation generation process. The intelligent scheduling module 300 is used to input the degraded image, user instructions, degradation perception results, phased restoration plan and experience information into the scheduling agent, which generates the overall restoration plan and determines the execution order of the degradation expert agent. The restoration module 400 is used to sequentially activate the corresponding degradation expert agents according to the overall restoration plan. The degradation expert agents combine the current image state, degradation degree and tool registration information to select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results. The iterative decision module 500 is used to evaluate the intermediate restoration results using the quality assessment of the no-reference image after each round of restoration, adaptively update the subsequent restoration operation strategy based on the changes in image quality, iterate in a loop until the restoration termination condition is met, and output the final restored image, degradation perception result and overall restoration plan to complete the image restoration task.

[0082] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0083] Example 3 To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0084] Example 4 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0086] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A visual intelligent agent adaptive restoration method based on physical degradation prior, characterized in that, include: A multimodal large model is used to perform degradation perception on the input degraded image. By identifying each degradation and assessing its intensity, the degradation type and its corresponding degree of degradation information are obtained, and degradation perception results are generated. Based on the real-world image degradation formation process, the degradation types in the degradation perception results are mapped to scene degradation, imaging degradation, and compression degradation, and a phased restoration plan with prior constraints is generated according to the reverse order of the physical sequence of the degradation generation process. Input the degraded image, user instructions, degradation perception results, phased restoration plan, and experience information into the scheduling agent, which generates the overall restoration plan and determines the execution order of the degradation expert agents; According to the overall restoration plan, the corresponding degradation expert agents are activated sequentially. The degradation expert agents combine the current image state, degradation degree and tool registration information to select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results. After each round of restoration is completed, the intermediate restoration results are evaluated using no-reference image quality assessment. The subsequent restoration operation strategy is adaptively updated based on the changes in image quality. The process is iterated until the restoration termination condition is met, and the final restored image, degradation perception result, and overall restoration plan are output to complete the image restoration task.

2. The method according to claim 1, characterized in that, A multimodal large model is used to perform degradation perception on the input degraded image. Through item-by-item degradation identification and intensity assessment, degradation type and its corresponding degree information are obtained, and degradation perception results are generated, including: The multimodal large model is used to analyze the input degraded image, and the hybrid degradation perception task is decomposed into multiple single degradation recognition sub-tasks. The existence of each type of degradation is detected in a pre-defined order to determine whether the corresponding degradation type exists in the degraded image. For the types of degradation identified by the detection, further quantitative assessment of the degradation intensity will be conducted to obtain information on the degree of degradation corresponding to each type of degradation; Integrate the identification results and intensity data of all degradation types to generate complete degradation perception results.

3. The method according to claim 2, characterized in that, The degradation types in the degradation perception results are mapped to scene degradation, imaging degradation, and compression degradation. Based on the reverse order of the physical sequence of the degradation generation process, a phased restoration plan with prior constraints is generated, including: Based on the physical formation order of real-world image degradation, the degradation types in the degradation perception results are classified into three physical levels: scene degradation, imaging degradation, and compression degradation. The restoration order is constrained according to the order of the degradation generation process, so that the restoration process is executed in reverse order of compression degradation, imaging degradation, and scene degradation. The unconstrained restoration path search is transformed into a constrained search process under physical prior constraints, generating a phased restoration plan with prior constraints.

4. The method according to claim 3, characterized in that, Inputting degraded images, user commands, degradation perception results, phased restoration plans, and experiential information into the scheduling agent, the scheduling agent generates an overall restoration plan and determines the execution order of degradation expert agents, including: The input degraded images, user commands, degradation perception results, phased restoration plans, and historical experience information are all uniformly input into the scheduling agent; By utilizing a scheduling agent that combines physical degradation priors with various input information, the entire image restoration process can be comprehensively planned and task decomposed. Based on the phased division of the phased restoration plan, the execution order of different degradation expert agents is arranged. Integrate the overall planning content with the execution order of the intelligent agent to output an overall restoration plan adapted to the current image restoration task.

5. The method according to claim 4, characterized in that, According to the overall restoration plan, the corresponding degradation expert agents are activated sequentially. These agents, combining the current image state, degradation level, and tool registration information, select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results, including: According to the execution order determined in the overall restoration plan, the degradation expert agents corresponding to the current degradation type are activated sequentially. The activated degradation expert agent reads the current image state, the corresponding degree of degradation, and tool registration information in real time; By combining the current state of the image with its degradation features, candidate restoration tools that match the restoration needs are selected from the toolset; The selected candidate restoration tool is invoked to perform round-by-round restoration processing on the image, and the corresponding intermediate restoration result is output after each round of processing.

6. The method according to claim 5, characterized in that, The intermediate restoration results are evaluated using no-reference image quality assessment, and subsequent restoration strategies are adaptively updated based on changes in image quality, including: After each round of image restoration processing is completed, multiple no-reference image quality evaluation indicators are used to quantitatively evaluate the intermediate restoration results generated in this round. Based on a greedy strategy for quality improvement, the image quality changes brought about by different candidate restoration operations are compared, and the restoration operation that can maximize the improvement of image quality is selected as the next execution strategy, thereby achieving adaptive updating of the restoration operation strategy. The process involves iteratively performing recovery processing, quality assessment, and strategy updates, while continuously verifying whether the recovery termination conditions are met.

7. The method according to claim 6, characterized in that, The restoration termination condition includes situations where all degradation in the degradation perception results has been repaired and multiple consecutive restoration operations have failed to improve image quality. When any of these termination conditions is triggered, all restoration iteration operations will be stopped immediately.

8. A visual intelligent agent adaptive restoration system based on physical degradation priors, characterized in that, include: The degradation perception module is used to perform degradation perception on the input degraded image using a multimodal large model. Through item-by-item degradation identification and intensity evaluation, it obtains information on the degradation type and its corresponding degree of degradation, and generates degradation perception results. The prior planning module is used to map the degradation type in the degradation perception result to scene degradation, imaging degradation and compression degradation based on the real-world image degradation formation process, and generate a phased restoration plan with prior constraints according to the reverse order of the physical order of the degradation generation process. The intelligent scheduling module is used to input the degraded image, user instructions, degradation perception results, phased restoration plan and experience information into the scheduling agent, which generates the overall restoration plan and determines the execution order of the degradation expert agent. The execution restoration module is used to sequentially activate the corresponding degradation expert agents according to the overall restoration plan. The degradation expert agents combine the current image state, degradation degree and tool registration information to select candidate restoration tools for step-by-step restoration processing and output intermediate restoration results. The iterative decision module is used to evaluate the intermediate restoration results using no-reference image quality assessment after each round of restoration, adaptively update the subsequent restoration operation strategy based on image quality changes, iterates until the restoration termination condition is met, and outputs the final restored image, degradation perception result and overall restoration plan to complete the image restoration task.

9. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images