Method and system for restoring distortion of text graph image based on intelligent agent

Abstract

The invention provides a method and a system for restoring distortion of a text map image based on an agent. The method comprises the following steps: constructing and training a distortion region sensing and positioning agent, a distortion region labeling and describing agent and a distortion region restoring agent; iteratively executing the following operations: detecting a distortion region of the text map image by using a distortion region sensing and positioning agent: if the distortion region is detected, reasoning a distortion type and a distortion description by using a distortion region labeling and describing agent based on the distortion region; based on the distortion area, the distortion type and the distortion description, using a distortion area restoration agent to execute distortion restoration and update the text graph image; and until the distorted area sensing and positioning intelligent agent does not detect the distorted area any more. According to the method, the intelligent agent is adopted, a sensing-reasoning-action closed loop is formed, the distortion problem of various artificial intelligence generated images can be effectively solved, and the method has excellent generalization and high efficiency.

Description

Technical Field

[0001] This application relates to the field of image restoration, and in particular to a method and system for restoring distortion in text-based images based on intelligent agents. Background Technology

[0002] In recent years, next-generation text-to-image (Text-to-Image) diffusion models, represented by SDXL, FLUX, and Imagen, have achieved significant breakthroughs in network scale, training data quality, and multimodal alignment capabilities. They are now able to generate highly realistic and artistically expressive images in most scenarios. These models have been widely applied in various industries, including digital art creation, visual design, advertising production, film and television pre-production, game content generation, and even assisted medical image synthesis and industrial simulation, gradually becoming a crucial infrastructure for content production.

[0003] Despite this, even the most advanced large-scale diffusion models still have significant shortcomings in actual generation, with the semantic structural distortion of local details being the most prominent. Specifically, existing models often exhibit the following typical problems: errors in human anatomy, including abnormal number of fingers, fused or broken fingers, unreasonable limb proportions, and incorrect orientation of joints such as elbows and knees; damage to facial details, such as asymmetrical eye shapes, broken lip structures, illogical arrangement of facial features, and blurred or overflowing facial textures; low quality of generated text, often resulting in unreadable text, inverted characters, disordered strokes, and inconsistent language; and abnormal entity interaction relationships, such as hands dangling in mid-air, fused contact surfaces, and grasping postures that do not conform to physical logic between people and objects, or between people and animals. Recent research has mainly addressed these problems from three directions: cue word enhancement, reinforcement learning-based model optimization, and fine-grained noise spatial alignment. While these methods can improve the overall image realism, they all lack explicit spatial reasoning capabilities and cannot explain or correct local failure areas. Post-hoc editing methods such as Imagic, Bagel, and Step1x-Edit can achieve local image restoration, but they rely on manually drawn masks of distorted areas or heuristic text indicators, and therefore cannot autonomously identify the areas that need restoration. Full image regeneration, on the other hand, leads to high restoration costs and potential style shift issues.

[0004] A literature search of existing technologies revealed a Chinese patent with publication number CN120807681A, which proposes a method, apparatus, device, and storage medium for generating images based on a text-to-image reasoning model. This method uses a visual language model to receive user requests for image generation, performs reasoning and analysis, obtains retrieval instructions, plans the image generation steps, and finally generates the image. Visual-Language Models (VLMs) are considered potential automated methods for image distortion repair due to their semantic reasoning capabilities. However, even the most advanced VLM models struggle to reliably locate distorted regions. For explicitly given queries, their answers are often contradictory or incorrect, even misclassifying obviously abnormal areas as normal. There are two key reasons for this phenomenon: First, VLM training focuses on high-level semantic alignment rather than pixel-level verification, resulting in poor spatial accuracy and a tendency to miss fine-grained artifacts; second, their powerful prior knowledge often overlies actual visual evidence, leading to "illusionary judgments."

[0005] Therefore, there is an urgent need for a self-correcting image distortion restoration method and system that can autonomously detect local distortions, pinpoint accurate areas, provide diagnoses and descriptions that are close to human aesthetic preferences, and perform controllable restoration operations. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the purpose of this application is to provide a method and system for distortion repair of text-based images based on intelligent agents.

[0007] According to a first aspect of this application, a method for distortion restoration of text-based images based on intelligent agents is provided, comprising: Construct and train a saliency map prediction model, and build a distortion region perception and localization agent based on the trained saliency map prediction model; Construct and train a distorted region annotation and description agent, and construct a distorted region repair agent; Iteratively perform the following operations: The distortion region perception and localization agent is used to detect distortion regions in the text image: If a distorted region is detected, the distortion type and distortion description of the raw image are inferred by the distortion region annotation agent based on the raw image and its corresponding distorted region; and the distortion repair agent performs distortion repair and updates the raw image based on the distorted region, the distortion type and the distortion description. The process continues until the distortion region sensing and localization agent no longer detects the distortion region, at which point the current raw image is taken as the final raw image distortion repair result.

[0008] Optionally, the step of constructing and training a saliency map prediction model, and constructing a distorted region-aware localization agent based on the trained saliency map prediction model, includes: A saliency map prediction model is constructed, comprising a dual encoder structure, an attention module, and an output module. The dual encoder structure includes a visual encoder and a text encoder configured in parallel. The visual encoder receives the input text image, extracts visual features from the text image, and outputs image features. The text encoder receives the input prompt words corresponding to the generated text image, extracts semantic features from the input prompt words, and outputs text features. The attention module performs cross-modal fusion processing on the input visual and text features using a self-attention mechanism to generate cross-modal features that simultaneously carry visual structural information and text semantic information. The output module outputs a distorted saliency map based on the cross-modal features. Train the saliency map prediction model to obtain a trained saliency map prediction model; A mask model is constructed, and the output of the trained saliency map prediction model is connected to the input of the mask model to form a distortion region perception and localization agent. The mask model is used to perform thresholding and morphological dilation operations on the input distortion saliency map in sequence to obtain the distortion region mask of the text image, and the region marked as distortion in the distortion region mask is taken as the distortion region of the text image. If no region marked as distorted appears in the distorted region mask, it indicates that the distorted region sensing and localization agent has not detected the distorted region.

[0009] Optionally, during the training process of the saliency map prediction model, a hybrid loss function is used, defined as: in, For loss; Preset balance parameters; For mean square error loss, S is the KL divergence loss, and S is the distortion saliency map predicted by the model. This is a manually labeled, true saliency map.

[0010] Optionally, the construction and training of the distorted region annotation description agent includes: A pre-trained multimodal visual language large model was selected as the basic visual model; A multimodal inference model is obtained by supervising the fine-tuning of the basic visual model using a dataset of manually annotated distorted regions. The multimodal reasoning model is trained by reinforcement learning using a group relative strategy optimization algorithm to introduce reinforcement signals, thereby obtaining a distortion region labeling and description agent. The reinforcement signals are specifically quantized signals constructed based on human perception preferences for distortion diagnosis. Optionally, the objective function of the group relative strategy optimization algorithm is: in, For GRPO loss, The expected value of the input question and the model's answer sample pair is taken; min() represents the minimum value function; This is the ratio of the probabilities of the previous strategy and the old strategy on the same sample. This is the normalized dominance function; Represents the interval restriction function; for clip Hyperparameters; Let KL divergence penalty coefficient be denoted as . For the KL divergence term, This is the policy model that is being trained. This serves as a reference strategy model.

[0011] Optionally, the step of inferring the distortion type and distortion description of the raw image based on the raw image and its corresponding distortion region, using the distortion region annotation description agent, includes: The raw image and its corresponding distorted regions are input into the distortion region labeling and description agent. The agent performs inferential diagnosis of the distorted regions. During the inferential diagnosis process, structured interactive question text is received in real time. This interactive question text is used to input diagnostic guidance information into the agent, triggering it to output structured response text that conforms to human perception preferences for distortion diagnosis. The structured response text is then parsed to obtain the distortion type and distortion description. The distortion types include hand distortion, interaction distortion, facial distortion, and redundant objects. The distortion description is descriptive text of the distorted content in the raw image.

[0012] Optionally, the step of using the distortion region repair agent to perform distortion repair and update the text image based on the distortion region, the distortion type, and the distortion description includes using the distortion region repair agent to perform the following operations: The editing space range of the target repair area is determined based on the distorted area; Based on the user preferences, a suitable image restoration model is selected from the restoration tool library; A repair instruction sequence is generated based on the distortion description and the distortion type; The selected image restoration model is invoked, and distortion restoration operation is performed on the raw text image to be restored within the editing space according to the restoration instruction sequence. The restored raw text image is then used as the updated raw text image.

[0013] According to a second aspect of this application, a distortion restoration system for text-based images based on intelligent agents is provided, comprising: The localization module is used to build and train a saliency map prediction model, and to build a distorted region-aware localization agent based on the trained saliency map prediction model. The model building module is used to build and train the intelligent agent for labeling and describing distorted regions, and to build the intelligent agent for repairing distorted regions. The iteration module is used to iterate and perform the following operations: The distortion region perception and localization agent is used to detect distortion regions in the text image: If a distorted region is detected, the distortion type and distortion description of the generated distorted image are determined by the distorted region annotation agent based on the raw image and its corresponding distorted region; and the distortion repair agent performs distortion repair and updates the distorted image based on the distorted region, the distortion type, and the distortion description. The process continues until the distortion region sensing and localization agent no longer detects the distortion region, at which point the current raw image is taken as the final raw image distortion repair result.

[0014] According to a third aspect of this application, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described agent-based text image distortion restoration method.

[0015] According to a fourth aspect of this application, an electronic device is provided, comprising: At least one memory for storing program instructions; At least one processor is configured to invoke program instructions stored in the memory and execute the steps of the above-described agent-based text image distortion restoration method according to the obtained program instructions.

[0016] This application provides a method for distortion restoration of raw text images based on an agent. A distortion region perception and localization agent is constructed based on a saliency map prediction model to accurately locate local distortion regions in the raw text image. A distortion region annotation and description agent can infer the distortion region to generate distortion type and description. Finally, a distortion region restoration agent performs restoration based on the inference results. This application obtains local distortion regions and performs proactive, reasoning-based restoration of these regions, achieving high-quality retouching of raw text images.

[0017] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0018] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of a method for distortion restoration of Chinese raw images in one embodiment of this application; Figure 2 This is a flowchart of the distortion restoration method for raw Chinese images in Embodiment 1 of this application; Figure 3 This is a schematic diagram of a saliency map prediction model in one embodiment of this application; Figure 4 A schematic diagram of the supervised fine-tuning method in one embodiment of this application; Figure 5 A schematic diagram of a group relative strategy optimization method in one embodiment of this application; Figure 6 This is a schematic diagram of a Chinese raw image distortion repair system according to one embodiment of this application. Detailed Implementation

[0019] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0020] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0021] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and this application does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0022] In recent years, next-generation text-to-image (text-generated image) diffusion models, such as SDXL, FLUX, and Imagen, still have significant shortcomings in practical generation processes, with the semantic and structural distortion of local details being the most prominent issue. Existing post-hoc editing methods rely on manually provided masks of distorted regions or heuristic textual indications, while visual-language models (VLMs) struggle to stably locate distorted regions and describe distorted content. To address these issues, this application provides an agent-based text-generated image distortion repair method to resolve the aforementioned problems.

[0023] Reference Figure 1 As shown in the figure, this application provides a method for distortion restoration of text-based images based on intelligent agents, including: S1. Construct and train a saliency map prediction model, and construct a distortion region perception and localization agent based on the trained saliency map prediction model; S2. Construct and train the intelligent agent for labeling and describing distorted regions, and construct the intelligent agent for repairing distorted regions; S3, perform the following operations in iteration: Detecting distorted regions in raw image files using a distortion-region perception and localization agent: If a distorted region is detected, the distortion type and description of the raw image are inferred by the distortion region labeling agent based on the raw image and its corresponding distorted region; and the distortion region repair agent is used to perform distortion repair and update the raw image based on the distorted region, distortion type and distortion description. The process continues until the distortion-aware localization agent no longer detects the distortion region, at which point the current raw image is taken as the final result of the raw image distortion repair.

[0024] It should be noted that distortion type and distortion description are used to characterize distortion features and their relationship to the context.

[0025] In the embodiments described above, a distortion region perception and localization agent is constructed based on a saliency map prediction model to accurately locate local distortion regions in raw text images. A distortion region annotation and description agent can infer the distortion region to generate distortion types and descriptions. Finally, a distortion region repair agent performs repair based on the inference results. This application obtains local distortion regions and performs proactive, reasoning-based repair of these regions, achieving high-quality retouching of raw text images.

[0026] In some specific embodiments of this application, a distortion-region-aware localization agent is constructed based on the construction and training of a saliency map prediction model, and based on the trained saliency map prediction model, including: A saliency map prediction model is constructed, comprising a dual encoder structure, an attention module, and an output module. The dual encoder structure includes a visual encoder and a text encoder configured in parallel. The visual encoder receives the input text image, extracts visual features from the text image, and outputs image features. The text encoder receives the input prompt words corresponding to the generated text image, extracts semantic features from the input prompt words, and outputs text features. The attention module performs cross-modal fusion processing on the input visual and text features through a self-attention mechanism to generate cross-modal features that simultaneously carry visual structural information and text semantic information. The output module outputs a distorted saliency map based on the cross-modal features. Train the saliency map prediction model to obtain a well-trained saliency map prediction model; A mask model is constructed, and the output of the trained saliency map prediction model is connected to the input of the mask model to form a distortion region perception and localization agent. The mask model is used to perform thresholding and morphological dilation operations on the input distortion saliency map in sequence to obtain the distortion region mask of the text image, and the region marked as distortion in the distortion region mask is taken as the distortion region of the text image. If no region marked as distorted appears in the distorted region mask, it indicates that the distorted region perception and localization agent has not detected the distorted region.

[0027] In the embodiments described above, this application designs a saliency map prediction model for distorted raw text images, and combines it with a mask model that can convert the distorted saliency map into distorted range coordinates to form a distorted region perception and localization agent. Inputting a distorted raw text image into the distorted region perception and localization agent yields the distorted region range. Specifically, the following steps are included: Step 1: Employ a dual encoder structure consisting of a visual encoder and a text encoder to perform cross-modal fusion of the text image and the corresponding input prompt text; through training with an attention module and hybrid loss, output a pixel-level distortion saliency map; Step 2: The saliency map is thresholded and morphologically dilated to obtain the pixel coordinates of the distorted region mask. Thresholding is used to compare the saliency value of each pixel in the distorted saliency map with a preset threshold to select a set of candidate pixels representing the distorted region. Morphological dilation is used to expand the boundary of the region formed by the set of candidate pixels, optimize the regional integrity of the distorted region mask, and thus ensure the accurate extraction of the pixel coordinates corresponding to the distorted region mask.

[0028] In some specific embodiments of this application, the loss function used in the training process of the saliency map prediction model is a hybrid loss function, defined as: in, For loss; Preset balance parameters; For mean square error loss, S is the KL divergence loss, and S is the distortion saliency map predicted by the model. This is a manually labeled, true saliency map.

[0029] In some specific embodiments of this application, a distortion region annotation and description agent is constructed and trained, including: S21. Select a pre-trained multimodal visual language large model as the basic visual model; S22. Supervised fine-tuning training of the basic visual model is performed using a manually labeled distorted region dataset to obtain a multimodal inference model; S23. The group relative strategy optimization algorithm is used to introduce reinforcement signals to train the multimodal reasoning model for reinforcement learning, so as to obtain the intelligent agent that describes the distortion region. The reinforcement signal is a quantitative signal constructed based on human perception preference for distortion diagnosis. In the above embodiments of this application, a multimodal visual language large model is selected, and progressive training is performed using manually labeled data. Specifically, progressive training involves first performing supervised fine-tuning to initialize the basic descriptive output capability, and then using reinforcement learning to enhance the consistency of human aesthetic preferences in the descriptive text.

[0030] The manually labeled distorted region dataset includes text-based images and manually labeled distorted region ranges. The specific generation method of the reinforcement signal is as follows: a reinforcement learning sample set is constructed, which includes the manually labeled distorted region dataset and multiple sets of candidate inference diagnostic results generated by the multimodal inference model for each sample in the manually labeled distorted region dataset. Based on human perception preferences for distorted diagnosis (diagnostic accuracy, descriptive standardization, contextual relevance, aesthetic consistency, etc.), the human annotators rank the multiple sets of candidate inference diagnostic results corresponding to each set of samples in the reinforcement learning sample set according to their preferences, and output preference labels with relative relationships of "better" and "poorer". The preference labels are then converted into quantitative signals that can be learned by the multimodal inference model, and the magnitude of the quantitative signal is positively correlated with the degree to which the candidate inference diagnostic results conform to human preferences.

[0031] In some specific embodiments of this application, the objective function of the population relative strategy optimization algorithm is: in, For GRPO loss, The expected value of the input question and the model's answer sample pair is taken; min() represents the minimum value function; This is the ratio of the probabilities of the previous strategy and the old strategy on the same sample. This is the normalized dominance function; Represents the interval restriction function; for clip Hyperparameters; Let KL divergence penalty coefficient be denoted as . For the KL divergence term, This is the policy model that is being trained. This serves as a reference strategy model.

[0032] In some specific embodiments of this application, based on the raw image and its corresponding distorted region, the distortion type and distortion description of the raw image are inferred by the intelligent agent using distortion region annotation, including: The raw image and its corresponding distorted regions are input into the distortion region labeling and description agent. The agent performs inferential diagnosis of the distorted regions. During the inferential diagnosis process, structured interactive question text is received in real time. The interactive question text is used to provide diagnostic guidance information to the agent, triggering it to output structured response text that conforms to human perception preferences for distortion diagnosis. The structured response text is then parsed to obtain the distortion type and distortion description. The distortion types include hand distortion, interaction distortion, facial distortion, and redundant objects. The distortion description is descriptive text of the distorted content in the raw image.

[0033] For example, diagnostic guidance information, as a constraint parameter for reasoning diagnosis, is embedded in the entire reasoning diagnosis process. This ensures that the reasoning logic of the agent describing the distorted region always aligns with human perceptual preferences for distortion diagnosis, ultimately directly outputting a structured response text that meets the requirements. The distortion types include a series of common text-based image distortions, such as hand distortion, interaction distortion, facial distortion, and redundant objects; the distortion description is the descriptive text describing the specific distorted content in the image.

[0034] In the embodiments described above, the multimodal reasoning model trained by reinforcement learning acts as an intelligent agent for labeling and describing distorted regions. It receives the text image and the range of distorted regions, and outputs accurate distortion types and distortion descriptions that conform to human preferences.

[0035] In some specific embodiments of this application, based on the distorted region, distortion type, and distortion description, a distortion region repair agent performs distortion repair and updates the raw image, including performing the following operations: Determine the editing space range of the target repair area based on the distorted area; Based on user preferences, a suitable image restoration model is selected from the restoration tool library; Generate parameterized repair instruction sequences based on distortion description and distortion type; The selected image restoration model is invoked, and distortion restoration is performed on the raw image to be restored within the editing space according to the restoration instruction sequence. The restored raw image is then used as the updated raw image.

[0036] In the embodiments described above, based on the distorted region output by the distortion region perception and localization agent, the distortion type and description output by the distortion region annotation and description agent, and user-provided preference information, an image restoration model from the restoration tool library is automatically selected to perform image restoration. The models in the restoration tool library include mask-based restoration models (such as SD-inpainting and FlUX-FILL) and instruction-driven restoration models (such as Qwen-Edit and Gemini 2.5 FlashImage). The present application will be further described below with reference to specific embodiments in order to better understand the above technical solutions of the present application. It should be understood that the following are only some examples and are not intended to limit the present application.

[0037] Example 1: A distortion restoration method for raw text images based on agent-based intelligent agents, such as... Figure 2 As shown, the method includes the following steps: Step S1: A saliency map prediction model for distorted text images was designed, and combined with a saliency map to masking mechanism, a distorted region-aware localization agent was obtained: Figure 3 This paper presents a saliency map prediction model architecture for a distortion-region-aware localization agent. The architecture employs a dual-encoder structure consisting of a visual encoder (ViT) and a text encoder (T5), encoding image and text representations respectively. These representations are then concatenated and fused using a self-attention mechanism in the first self-attention layer to capture the inherent correspondence between visual structure and text semantics. A lightweight attention refinement module (i.e., the second self-attention layer) further aggregates multi-scale contextual cues, thereby improving the detection performance of distortion regions that rely on global image context for recognition.

[0038] The saliency map prediction model is trained using a hybrid loss function that balances pixel accuracy and distribution consistency. : Where S is the predicted saliency map. This is a manually labeled true saliency map. Balance parameters. Used to control the balance between reconstruction accuracy and perception alignment. The mean squared error loss; the KLD term, i.e. This helps align the model's output with the human gaze distribution, thereby maintaining discriminative power in blurred areas and avoiding over-smoothing of the saliency map.

[0039] The obtained predicted saliency map is binarized and morphologically dilated to generate pixel coordinates for a distortion region mask used in subsequent inference.

[0040] Step S2: Design and train the distortion region annotation and description agent. Input the located distortion region range and image into the distortion region annotation and description agent to obtain the distortion type and distortion description: After obtaining the distorted region mask, the distorted region labeling agent performs inferential diagnosis on these distorted regions, generating textual diagnostic content, including the distortion type and distortion description, to characterize the distortion features and their relationship with the context. This task requires structured reasoning consistent with human perception and judgment, rather than simple classification or image description. To this end, this application adopts a progressive preference alignment paradigm consisting of two complementary stages: namely, structural initialization through supervised fine-tuning (SFT), and reinforcement alignment conforming to human preferences through group relative policy optimization (GRPO) in reinforcement learning.

[0041] Reference Figure 4 As shown, in the first stage, SFT establishes a standardized response format and distortion category system under limited supervision. To reduce computational overhead, this application employs the LoRA (Low-Rank Adaptation) method, enabling the inference model to achieve efficient task specialization without full parameter fine-tuning. Figure 4 In this context, A is the reduced-dimensional matrix and B is the increased-dimensional matrix. The training weight parameters W are updated by updating some weight parameters (B*A) to reduce training resource consumption.

[0042] Reference Figure 5 As shown, in the second stage, GRPO is used to align reasoning behavior with human preferences using reinforcement signals, and its optimization objective is: in The normalized dominance function is used to characterize the consistency with preferences. Policy optimization is guided by two reward signals: the classification accuracy of the distortion type and the semantic alignment between the model-generated distortion description and the human annotation. This stage effectively reduces reasoning illusions, enabling the agent to generate stable analytical results consistent with human judgment under diverse distortion modes. Specifically, a VLM model is used to align reasoning behavior with human preferences. For a user's question, the VLM model adopts different strategies and outputs n answers; then, the reward corresponding to each answer is obtained according to the reward calculation function, and the optimal strategy is selected based on the relative size of the reward (dominance function), thereby completing the optimization of the policy model. The reward function in this application generally consists of two parts: classification accuracy reward (correct output distortion type) and semantic alignment reward (output distortion description conforms to human preferences).

[0043] Step S3: Input the distorted region, distortion type, and description into the distortion region repair agent to complete the distortion repair of the text image.

[0044] Based on the inference results, the distortion region repair agent transforms the distortion region mask, distortion type, and distortion description into controllable local editing operations. Specifically, the agent determines the editing scope, tool selection, and specific repair instructions for each region. Depending on the computational budget or user preference, the agent can choose between mask-based repair models (such as SD-inpainting and Flux-FILL) and instruction-driven repair models (such as Qwen-Edit and Gemini 2.5 Flash Image), all of which are derived from a repair tool library.

[0045] Step S4: Re-input the repair result into the distorted area perception and localization agent. If distortion can still be detected, continue with steps S2 and S3 until the distortion is eliminated.

[0046] It should be noted that all the construction and training of the agents are done outside the loop. The training and construction of the agents are completed in advance, and the loop only includes the process of calling the agents for inference.

[0047] The image repaired by the distortion region repair agent is then evaluated again by the distortion region perception and localization agent, thus forming a complete "perception-reasoning-action" closed loop. If significant distortion is still detected, iterative repair is achieved by repeating steps S2 and S3. Through iterative perception and reasoning, the framework gradually converges to output image results with reliable details and higher quality. Table 1 presents a comparison of image quality metrics between images restored using mainstream text-based image models (such as Qwen-Edit, Gemini 2.5 FlashImage, Flux-fill, and SD-inpainting) and images restored using the method described in this application. The self-built dataset GenBlemish-27K was used, and the metrics were derived from RichHF. It can be seen that all models demonstrate improved image quality after using this application, with comprehensive improvements in aesthetic metrics, rationality metrics, image-text consistency metrics, and overall image metrics. Here, "original image" refers to the distorted image in the dataset, which serves as the model input; the score corresponding to "mainstream text-based image model" is the score of the image output after redrawing and restoration using this model; and the score corresponding to "mainstream text-based image model + this application" is the score of the image output after image restoration using the process described in this application. The mainstream text-based image model included here refers to the specific text-based image model (i.e., restoration tool) used by the distortion region restoration agent.

[0048] Table 1 This application's method transforms post-processing from traditional passive error correction to proactive, reasoning-based restoration. By integrating perception-based diagnosis, human preference-aligned reasoning, and an adaptive local restoration system within a unified closed loop, this framework achieves high-quality retouching of text-based images.

[0049] Based on the same inventive concept, another embodiment of this application provides a distortion repair system for text-based images based on intelligent agents, referring to... Figure 6 As shown, the distortion repair system 100 includes: The localization module 110 is used to build and train a saliency map prediction model, and to build a distorted region perception localization agent based on the trained saliency map prediction model. Model building module 123 is used to build and train the intelligent agent for labeling and describing distorted regions, and to build the intelligent agent for repairing distorted regions. Iteration module 130 is used to iteratively perform the following operations: Detecting distorted regions in raw image files using a distortion-region perception and localization agent: If a distorted region is detected, the distortion type and distortion description of the generated distorted image are determined by the distortion region annotation agent based on the raw image and its corresponding distorted region. Based on the distorted region, distortion type, and distortion description, the distortion region repair agent performs distortion repair and updates the distorted image. The process continues until the distortion-aware localization agent no longer detects the distortion region, at which point the current raw image is taken as the final result of the raw image distortion repair.

[0050] It should be noted that the modules in the agent-based image distortion repair system provided in the above embodiments of this application correspond to the steps of the agent-based image distortion repair method in any of the above embodiments. Those skilled in the art can refer to the step features of the agent-based image distortion repair method to implement the corresponding modules in the agent-based image distortion repair system, which will not be described again here.

[0051] This application achieves distortion repair of AI-generated images by designing a proxy-based pipeline and employing different proxy agents to handle tasks such as distortion region perception and localization, distortion region annotation and description, and distortion region restoration. For the proxy agents designed in the pipeline, this application has performed adjustments, optimizations, and training for specific tasks, forming a "perception-reasoning-action" closed loop, enabling them to effectively handle various distortion problems in AI-generated images and exhibiting excellent generalization and efficiency. For AI-generated images with distortion, the restoration results achieved through this application's proxy-based pipeline, as shown in Table 1, demonstrate higher subjective quality (e.g., aesthetic indicators) and objective quality (e.g., rationality indicators) compared to restoration results obtained by directly using visual language models or image redrawing models.

[0052] In another embodiment of this application, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described agent-based text image distortion repair method.

[0053] In another embodiment of this application, an electronic device is also provided, including a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions stored in the memory and execute the steps of the above-described agent-based text image distortion repair method according to the obtained program instructions.

[0054] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0055] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0056] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0057] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0058] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0059] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0062] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0063] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A method for distortion restoration of text-based images based on intelligent agents, characterized in that, include: Construct and train a saliency map prediction model, and build a distortion region perception and localization agent based on the trained saliency map prediction model; Construct and train a distorted region annotation and description agent, and construct a distorted region repair agent; Iteratively perform the following operations: The distortion region perception and localization agent is used to detect distortion regions in the text image: If a distorted region is detected, the distortion type and distortion description of the raw image are inferred by the distortion region annotation agent based on the raw image and its corresponding distorted region; and the distortion repair agent performs distortion repair and updates the raw image based on the distorted region, the distortion type and the distortion description. The process continues until the distortion region sensing and localization agent no longer detects the distortion region, at which point the current raw image is taken as the final raw image distortion repair result.

2. The method for distortion restoration of text-based images based on intelligent agents according to claim 1, characterized in that, The process of constructing and training a saliency map prediction model, and then constructing a distorted region-aware localization agent based on the trained saliency map prediction model, includes: A saliency map prediction model is constructed, comprising a dual encoder structure, an attention module, and an output module. The dual encoder structure includes a visual encoder and a text encoder configured in parallel. The visual encoder receives the input text image, extracts visual features from the text image, and outputs image features. The text encoder receives the input prompt words corresponding to the generated text image, extracts semantic features from the input prompt words, and outputs text features. The attention module performs cross-modal fusion processing on the input visual and text features using a self-attention mechanism to generate cross-modal features that simultaneously carry visual structural information and text semantic information. The output module outputs a distorted saliency map based on the cross-modal features. Train the saliency map prediction model to obtain a trained saliency map prediction model; A mask model is constructed, and the output of the trained saliency map prediction model is connected to the input of the mask model to form a distortion region perception and localization agent. The mask model is used to perform thresholding and morphological dilation operations on the input distortion saliency map in sequence to obtain the distortion region mask of the text image, and the region marked as distortion in the distortion region mask is taken as the distortion region of the text image. If no region marked as distorted appears in the distorted region mask, it indicates that the distorted region sensing and localization agent has not detected the distorted region.

3. The method for distortion restoration of text-based images based on intelligent agents according to claim 2, characterized in that, During the training process, the saliency map prediction model employs a hybrid loss function, defined as follows: in, For loss; Preset balance parameters; For mean square error loss, S is the KL divergence loss, and S is the distortion saliency map predicted by the model. This is a manually labeled, true saliency map.

4. The method for distortion restoration of text-based images based on intelligent agents according to claim 1, characterized in that, The construction and training of the distorted region annotation and description agent includes: A pre-trained multimodal visual language large model was selected as the basic visual model; A multimodal inference model is obtained by supervising the fine-tuning of the basic visual model using a dataset of manually annotated distorted regions. A group relative strategy optimization algorithm is used to introduce reinforcement signals to train the multimodal reasoning model through reinforcement learning, thereby obtaining a distortion region labeling and description agent. The reinforcement signals are specifically quantized signals constructed based on human perceptual preferences for distortion diagnosis.

5. The method for distortion restoration of text-based images based on intelligent agents according to claim 4, characterized in that, The objective function of the group relative strategy optimization algorithm is: in, For GRPO loss, The expected value of the input question and the model's answer sample pair is taken; min() represents the minimum value function; This is the ratio of the probabilities of the previous strategy and the old strategy on the same sample. This is the normalized dominance function; Represents the interval restriction function; for clip Hyperparameters; Let KL divergence penalty coefficient be denoted as . For the KL divergence term, This is the policy model that is being trained. This serves as a reference strategy model.

6. The method for distortion restoration of text-based images based on intelligent agents according to claim 4, characterized in that, The process of inferring the distortion type and description of a text-based image based on its distorted regions and using the distortion region annotations to describe the intelligent agent includes: The raw image and its corresponding distorted regions are input into the distortion region labeling and description agent. The agent performs inferential diagnosis of the distorted regions. During the inferential diagnosis process, structured interactive question text is received in real time. This interactive question text is used to input diagnostic guidance information into the agent, triggering it to output structured response text that conforms to human perception preferences for distortion diagnosis. The structured response text is then parsed to obtain the distortion type and distortion description. The distortion types include hand distortion, interaction distortion, facial distortion, and redundant objects. The distortion description is descriptive text of the distorted content in the raw image.

7. The method for distortion restoration of text-based images based on intelligent agents according to claim 1, characterized in that, The step of using the distortion region repair agent to perform distortion repair and update the text image based on the distortion region, the distortion type, and the distortion description includes using the distortion region repair agent to perform the following operations: The editing space range of the target repair area is determined based on the distorted area; Based on the user preferences, a suitable image restoration model is selected from the restoration tool library; A repair instruction sequence is generated based on the distortion description and the distortion type; The selected image restoration model is invoked, and distortion restoration operation is performed on the raw text image to be restored within the editing space according to the restoration instruction sequence. The restored raw text image is then used as the updated raw text image.

8. A distortion restoration system for text-based images based on intelligent agents, characterized in that, include: The localization module is used to build and train a saliency map prediction model, and to build a distorted region-aware localization agent based on the trained saliency map prediction model. The model building module is used to build and train the intelligent agent for labeling and describing distorted regions, and to build the intelligent agent for repairing distorted regions. The iteration module is used to iterate and perform the following operations: The distortion region perception and localization agent is used to detect distortion regions in the text image: If a distorted region is detected, the distortion type and distortion description of the generated distorted image are determined by the distorted region annotation agent based on the raw image and its corresponding distorted region; and the distortion repair agent performs distortion repair and updates the distorted image based on the distorted region, the distortion type, and the distortion description. The process continues until the distortion region sensing and localization agent no longer detects the distortion region, at which point the current raw image is taken as the final raw image distortion repair result.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.

10. An electronic device, characterized in that, include: At least one memory for storing program instructions; At least one processor is configured to invoke program instructions stored in the memory and execute the steps of the method described in any one of claims 1-7 according to the obtained program instructions.

Images