A reinforcement learning optimized diffusion model based image fusion method

Abstract

The application discloses a kind of reinforcement learning optimization diffusion model-based image fusion methods, it is related to artificial intelligence reinforcement learning field.The method includes: processing background image and the main body image to be merged, pre-processing image and standardizing the action space of intelligent agent;Obtain the action of intelligent agent according to current state space;According to action and pre-processed image, obtain binary mask;Intermediate image is processed, and the guide prompt word of final fusion is obtained;The fused image is evaluated using image evaluation function, and an evaluation score is obtained;The evaluation score is mapped so that the intelligent agent can learn better;Finally, a trained intelligent agent is obtained, thereby performing image fusion.The application solves the quality and cost problem of fusion position selection in the image fusion process based on diffusion model, can be practically applied in the field of actual image generation, can meet the diversified image needs of more users, and bring richer, more convenient and efficient image synthesis experience.

Description

TECHNICAL FIELD

[0001] The application relates to an optimized diffusion model image fusion method, in particular to a diffusion model-based image fusion method optimized through reinforcement learning. BACKGROUND

[0002] In recent years, the continuous development of science and technology has promoted the rapid progress of deep learning technology. Text-driven diffusion models have accelerated development and matured rapidly in the rapid development of social media technology and the increasing demand of the market for high-quality and high-playability images. In order to meet user needs, diffusion models can support various image editing tasks and specific instance generation tasks. For example, image fusion, text-driven image modification, low-order adaptation model Lora of large language model, and Dreambooth training of fine-tuning diffusion model. Existing image editing tasks usually involve very expensive specific instance-based optimization or require pre-training of the model on a given dataset. To some extent, these methods will destroy the prior knowledge of the model, resulting in a decline in the model's generation ability and scalability. Lora training and Dreambooth training of fine-tuning diffusion model will cause the model's scalability to decline, resulting in pictures that do not meet the expected effect. Therefore, a training-free cross-domain image fusion method based on diffusion model, TF-ICON (TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition), has emerged. TF-ICON does not require any training of the diffusion model, only some improvements to the structure are needed to achieve training-free image fusion. This method does not require training, has low cost, and will not damage the prior knowledge of the diffusion model. In addition, TF-ICON also supports the fusion of two images in different styles, which can meet the user's many needs for playability. However, TF-ICON is very sensitive to the selection of image fusion positions when performing fusion. In order to achieve good fusion results, users need to constantly try different fusion positions, which requires a lot of time and effort, and the final synthesized result may not be ideal, and the user experience is not friendly.

[0003] The existing TF-ICON algorithm is optimized and improved in this paper, which can automatically select the position with better fusion effect in the picture, avoiding the situation of poor fusion effect when users blindly choose. A lot of human resources are saved, and better fusion results are achieved through prompt word optimization technology. In addition, the reinforcement learning optimized diffusion model-based image fusion method is applied to the practical application of Lora, solving the shortcoming of poor scalability of Lora in practical scenarios, so that the generated pictures can better meet the user's requirements and bring users a richer experience. SUMMARY

[0004] To solve the above technical problems, the application provides a reinforcement learning optimized diffusion model based image fusion method.

[0005] In order to achieve the above application purposes, the technical scheme adopted by the application is: a reinforcement learning optimized diffusion model based image fusion method, the diffusion model based image fusion method is a position sensitive fusion method, the selection of fusion position greatly affects the fusion effect, the reinforcement learning optimized fusion method includes the following steps:

[0006] 1. Process the background image and the subject image to be merged; obtain the aspect ratio of the subject image, pre-process the background image and the subject image to be merged, and obtain the action space of the agent according to the pre-processed background image and the subject image.

[0007] 2. Extract the action of the agent; the agent selects in the action space of step 1 to obtain the action action=(x,y,w,h), wherein x represents the selected horizontal coordinate position, y represents the selected vertical coordinate position, w represents the width of the selected region, and h represents the height of the selected region.

[0008] 3. Obtain a binary mask; obtain a binary mask image of the background image fusion position according to the action of the agent.

[0009] 4. Extract the fused intermediate image; input the background image of step 1 and the subject image to be merged, and the binary mask image of step 3 into the training free cross-domain image fusion method based on diffusion model, referred to as TF-ICON, to obtain the fused intermediate image.

[0010] 5. Obtain the final fused guide prompt word; use a graphic text multi-modal network BLIP model to analyze the intermediate image of step 4 to obtain a description of the image, input the image and the image prompt word into a Transformer based generative large model to further expand and weight the image description word, and obtain the prompt word for guiding image generation.

[0011] 6. Evaluate the fused image using an image evaluation function; input the prompt word of step 5 into the TF-ICON algorithm as a conditional parameter to guide image synthesis, obtain the fused image, and evaluate the image using an image evaluation method LPIPS to obtain an evaluation score.

[0012] 7. Map the evaluation score; process the score obtained in step 6 to obtain the current selected image region features and position information, feed back to the agent, and continue training in step 2 until the reinforcement learning agent converges. Obtain the converged reinforcement learning model and enter step 8.

[0013] 8. The trained agent obtained by using the reinforcement learning mechanism acquires the image fusion area according to the background image and the subject image, and guides the final fusion.

[0014] In step 3, the selected action action = (x, y, w, h) of the agent is used to acquire the selected area on the background image, and the binary mask image is converted as the input of step 4.

[0015] The input of step 4 is the processing in the TF-ICON, which scales the subject image of step 1 according to the proportion of the binary mask image of step 3, adjusts the proportion of the subject image to be fused, and places it according to the center position alignment mode in the position of the binary mask image on the background image, thereby obtaining the required intermediate image.

[0016] In step 6, the image evaluation function is used to evaluate the perceptual loss LPIPS image difference between the subject image area of the fused image and the subject image area before fusion, and record the evaluation score; the self-attention map and cross-attention map generated during the process of acquiring the background image and the subject image to be fused from the fusion method TF-ICON are obtained, and then spliced to form a new attention map; at the same time, the prompt word of step 5 is received as the text condition of the diffusion model image generation, which jointly guides the fusion of the fusion image.

[0017] In step 7, in order to guide the agent to learn better fusion area selection according to the image perceptual loss LPIPS score obtained in step 6, the perceptual loss LPIPS score is linearly mapped; the lower the perceptual loss LPIPS score, the higher the score obtained by the agent. Because the perceptual loss LPIPS score is the difference between the two images, the lower the score, the better the fusion effect of the subject image after fusion and before fusion, so linear mapping is performed here.

[0018] The lower the LPIPS score, the higher the score obtained by the agent, specifically when the LPIPS score is less than or equal to 0.5, the agent is given a larger reward score reward = 10; the higher the LPIPS score, the poorer the fusion effect, and the agent is punished for poor action effect, specifically when the LPIPS score is greater than or equal to 0.65, the agent is given a score reward = -1. Through the negative score, the agent tries more in the high score area.

[0019] The application has the following beneficial technical effects: the application provides an image fusion method based on a diffusion model optimized by reinforcement learning. First, a background image and a main body image to be merged are preprocessed, and an action space of an agent is obtained; a binary mask on the background image is obtained according to the action taken by the agent; the binary mask, the preprocessed background image and the main body image are input into a TF-ICON method to obtain an intermediate image in the fusion process, that is, a spliced image of the background image and the main body image; the image is analyzed by a BLIP model of a graphic-text multimodal network to obtain a description of the image; the description is input into a generative large model based on a Transformer to further weight and expand, and a final prompt word is obtained; the final prompt word is used to guide image fusion to obtain a fused image, and an image evaluation method LPIPS is used to evaluate the image; the LPIPS evaluation score is processed and fed back to the agent together with the image features and position information of the selected region of the agent for learning; a trained agent is obtained for final fusion. The application can provide a convenient and efficient image fusion method for users, bring excellent fusion effect, free the hands of users and reduce the trial and error cost of users; the application can meet the diversified image needs of more users and bring a better image synthesis experience to more people. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of an embodiment of the application.

[0021] Figure 2 is a complete flowchart of an embodiment of the application.

[0022] Figure 3 is a flowchart of prompt word generation of an embodiment of the application. DETAILED DESCRIPTION

[0023] To make the objects, technical solutions and advantages of the application clearer, further detailed description will be made to the application with reference to the specific embodiments and the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the application. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the application; in addition, the technical features involved in the different embodiments of the application described below can be combined with each other as long as they do not conflict with each other. The application will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are denoted by similar reference numerals. For the sake of clarity, each part in the drawings is not drawn to scale.

[0024] The application example provides an image fusion method based on a diffusion model optimized by reinforcement learning, as shown in Figure 1 The method comprises the following steps:

[0025] S1, processing the background image and the subject image to be merged. The background image Xbg and the subject image Xfg to be processed are preprocessed, the background image is processed into a standard 512x512 pixel Xbg', and the subject image is segmented into a subject part Xfg' by a semantic segmentation method SAC; the aspect ratio w / h of the subject image Xfg is obtained, wherein w represents the width of the background image Xbg', and h represents the height of Xbg', and then the action space of the agent is initialized with the aspect ratio, so that the action of the agent conforms to the aspect ratio of the subject image, so as to better perform fusion.

[0026] S2, extracting the action of the agent. The agent selects in the action space of S1, and obtains an action action=(x, y, w, h) according to the current state space, wherein x represents the horizontal coordinate position in the background image Xbg', y represents the vertical coordinate position in the background image Xbg', w represents the width of the selected region, and h represents the height of the selected region.

[0027] S3, obtaining a binary mask. According to the action action selected by the agent in S2, the selected region of the action action in the background image is obtained, and then converted into a binary image, wherein the selected region is a white region, and the remaining background image is a black region.

[0028] S4, extracting the intermediate image for fusion. The background image Xbg' of S1, the subject image Xfg' after semantic segmentation, and the binary mask in S3 are input into the TF-ICON algorithm for splicing and combination. Specifically, Xfg' is scaled to adapt to the size of the white region in the binary mask, and then covers the background image region to achieve the purpose of combination.

[0029] S5, obtaining the final fusion guide prompt word. As shown in Figure 3 , the intermediate image obtained in S4 is input into the graphic-text multimodal network BLIP model to obtain a description of the intermediate image, and then the description and the text prompt are input into the Transformer-based generative large model to obtain the final guide prompt word.

[0030] S6, evaluating the fused image by using an image evaluation function; as shown in Figure 2 , the final guide prompt word is input into the TF-ICON algorithm to guide the continuous fusion, and the fused image X is obtained. The image X is evaluated by using the image evaluation function LPIPS.

[0031] S61, the binary mask obtained in S3 is used to extract the subject part in the fused image X, and the image evaluation method LPIPS is used to evaluate the evaluation loss of Xfg in S1, so as to measure the similarity between the two images. The higher the similarity, the closer the features of the fused subject image to the original image, and the better the fusion effect.

[0032] S62, LPIPS perceptual loss is a deep learning-based image quality evaluation index, which can more accurately simulate human perception of image quality. Therefore, using it as an evaluation index can make the fused image as close to human perception as possible.

[0033] S7, mapping the evaluation score; in order to enable the agent to learn according to the fusion effect, so as to achieve the effect of being able to select a better fusion position, the evaluation score is used as the reward of the agent here. Since the lower the score, the better the fusion effect, the perceptual loss LPIPS evaluation score in S6 needs to be mapped.

[0034] S71, the obtained perceptual loss LPIPS score is lpips_socre, linear mapping is performed to obtain the score score=(1-lpips_socre)*10, so that the perceptual loss LPIPS score and the reward of the agent are proportional, and the difference is expanded, which further promotes the agent to select a better fusion area;

[0035] S72, in order to guide the agent to learn to select a better fusion area, a larger reward is given when the action taken is a lower LPIPS score, and a negative reward is given when the LPIPS score obtained by the action taken is higher, so as to further promote the agent to prefer to select an area with a lower LPIPS score. Specifically, when the LPIPS score is less than or equal to 0.5, the reward score reward given to the agent is 10, and when the LPIPS score is greater than or equal to 0.65, the score reward given to the agent is reward=-1. By further distinguishing between good actions and bad actions, the agent is promoted to learn better;

[0036] S73, the score score, the features of the current image extracted by the ResNet residual network, and the position information of the area selected by the agent are returned to step S2 as new state information, guiding the selection of the next action of the agent, so as to guide the agent to achieve better learning, until the reinforcement learning agent converges, and the converged model is obtained and transferred to step S8.

[0037] S8, obtaining the trained agent to guide the fusion; after training, the trained agent is obtained, the background image and the subject image to be fused are provided, and finally the fused image is obtained.

[0038] Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

Claims

1. A reinforcement learning optimized image fusion method based on diffusion model, characterized in that: It includes the following steps: Step 1: Process the background image and the subject image to be integrated; obtain the aspect ratio of the subject image, pre-process the background image and the subject image to be integrated, and obtain the action space of the intelligent agent based on the pre-processed background image and subject image; Step 2: Extract the agent's action. The agent selects in the action space of step 1 and obtains action action = (x, y, w, h), where x represents the selected horizontal coordinate position, y represents the selected vertical coordinate position, w represents the width of the selected area, and h represents the height of the selected area. Step 3: Obtain a binary mask; obtain a binary mask image of the background image fusion position according to the action of the intelligent agent; Step 4: Extract the fused intermediate image. The background image from step 1, the subject image to be fused, and the binary mask image from step 3 are input into a training-free cross-domain image fusion method based on a diffusion model, referred to as TF-ICON, to obtain the fused intermediate image. Step 5: Obtain the final fused guiding words. Use the image-text multimodal network BLIP model to parse the intermediate image from step 4 to obtain a description of the image. Input the image and image prompt words into the Transformer-based generative model, further expand and weight the image description words, and obtain the prompt words that guide image generation. Step 6: Evaluate the fused image using the image evaluation function. Input the prompt word from step 5 into the TF-ICON algorithm as a conditional parameter to guide image synthesis and obtain the fused image. Use the image evaluation method LPIPS to evaluate the image and obtain an evaluation score. Step 7: Map the evaluation scores; process the scores obtained in step 6, obtain the features and position information of the currently selected image region, feed back to the agent, and continue training in step 2 until the reinforcement learning agent converges. After the convergence, the reinforcement learning model is obtained and transferred to step 8. Step 8: Using the trained agent obtained by the reinforcement learning mechanism, the image fusion area is obtained according to the background image and the subject image, and the final fusion is guided.

2. The image fusion method based on diffusion model and reinforcement learning optimization according to claim 1, characterized in that: In step 3, the action action = (x, y, w, h) selected by the agent is used to obtain the selected area on the background image and convert it into a binary mask image as input to step 4.

3. The image fusion method based on diffusion model and reinforcement learning optimization according to claim 1, characterized in that: The processing in the input TF-ICON of step 4 is to scale the main image of step 1 according to the ratio of the binary mask image of step 3, adjust the ratio of the main image to be integrated, and place it in a manner that aligns the center position of the binary mask image in the background image, so as to obtain the required intermediate image.

4. The image fusion method based on diffusion model and reinforcement learning optimization according to claim 1, characterized in that: In the processing of step 6, the perceptual loss LPIPS image difference between the main image area of ​​the fused image and the main image area before fusion is evaluated using an image evaluation function, and the evaluation score is recorded; the self-attention map and the cross-attention map during the generation process of the background image and the main image to be fused are obtained from the fusion method TF-ICON, and then the two are spliced ​​to form a new attention map; at the same time, the prompt word in step 5 is received as the text condition of the diffusion model generation image, which jointly guides the fusion of the fused image.

5. The image fusion method based on diffusion model and reinforcement learning optimization according to claim 1, characterized in that: The processing of step 7 is to facilitate the reinforcement learning agent to learn better fusion area selection based on the image perception loss LPIPS score obtained in step 6, and linearly map the perception loss LPIPS score; when the perception loss LPIPS score is lower, the score obtained by the agent is higher.

6. The image fusion method based on diffusion model and reinforcement learning optimization according to claim 5 is characterized in that: When the LPIPS score is less than or equal to 0.5, the agent is given a larger reward score of reward = 10; a higher LPIPS score indicates a poorer fusion effect, and a penalty is given for poor action performance. Specifically, when the LPIPS score is greater than or equal to 0.65, the agent is given a reward score of -1; negative scores encourage the agent to try more in areas with high scores.

Images