Multi-round ai image generation content consistency transformation method based on reinforcement learning

Abstract

The application relates to the technical field of image processing, and particularly discloses a multi-round AI image generation content consistency transformation method based on reinforcement learning, which comprises the following steps: acquiring an initial image data set containing semantic annotations, extracting multi-scale features, constructing a reinforcement learning agent containing a strategy network and a value function network, generating rotation, scaling and other transformation actions, executing multi-round transformation, calculating structure, perception and semantic multidimensional consistency rewards, updating network parameters by using a proximal policy optimization algorithm, optimizing the transformation strategy, and finally outputting an optimal transformation image and a consistency evaluation report. By constructing the agent based on the reinforcement learning, combining multi-scale image feature extraction, multi-round transformation strategy optimization and multidimensional consistency evaluation, the application realizes the consistency transformation of content in the multi-round AI image generation process under a complex scene, effectively deals with the consistency decline problem caused by scenes such as occlusion, complex illumination and dense interaction of multiple objects, and ensures the stability of core content.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multi-round AI image generation content consistency transformation method based on reinforcement learning. Background Technology

[0002] Multi-round image transformation is a crucial step in tasks such as data augmentation, scene simulation, and virtual content generation. Its core requirement is to maintain the consistency of the image's core content, including object categories, structural relationships, and semantic logic, even after multiple transformations such as rotation, scaling, lighting adjustments, and occlusion simulation. This requirement is particularly prominent in complex scenarios involving occlusion, complex lighting, and dense interactions between multiple objects. Occlusion can obscure key object features, complex lighting can alter pixel distribution characteristics, and dense interactions between multiple objects place higher demands on the stability of the relative positions and morphological relationships of objects. Slight errors can cause the transformed image to deviate from the original semantics or historical transformation trajectory, affecting the reliability of subsequent tasks.

[0003] Existing technologies have significant limitations in maintaining content consistency across multiple rounds of image transformation. First, traditional image transformation methods often employ fixed-rule single-transformation strategies, such as preset rotation angles and scaling ratios, without considering the cumulative effect of multiple rounds of transformation. In complex scenes, this can easily lead to issues where a single transformation is reasonable, but multiple rounds result in offsets. Continuous small-amplitude scaling over multiple rounds may cause distortion of object proportions, and multiple lighting adjustments may cause the image tone to deviate completely from its initial state. Second, while some learning-based methods introduce iterative optimization, consistency evaluation often relies on a single metric, making it difficult to cover the multi-dimensional consistency requirements at the structural, perceptual, and semantic levels. In scenarios with dense interaction between multiple objects, pixel similarity alone cannot determine whether the interaction relationship between objects is maintained.

[0004] Therefore, there is an urgent need for a multi-round AI image generation content consistency transformation method based on reinforcement learning to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-round AI image generation content consistency transformation method based on reinforcement learning, comprising the following steps:

[0006] An initial image dataset is obtained, which includes multiple original images and their corresponding semantic annotation information. An image feature extraction network is constructed, and multi-scale feature maps of each original image are extracted based on a convolutional neural network structure, and an initial image feature vector set is generated.

[0007] A reinforcement learning agent is constructed, which includes a policy network and a value function network. The policy network is used to generate image transformation actions, and the value function network is used to evaluate the long-term consistent reward of the actions. The initial image feature vector is input into the policy network, and the output includes image transformation actions such as rotation, scaling, brightness adjustment, and occlusion simulation.

[0008] Perform multiple rounds of image transformation. In each round, the original image is transformed according to the action output by the policy network to generate the transformed image. The consistency reward value between the current transformation and the historical transformation is calculated based on the value function network.

[0009] Based on the reward values ​​collected during the multi-round transformation process, the parameters of the policy network and the value function network are updated to optimize the image transformation strategy, which is used to maintain the content consistency of the generated image in complex scenes.

[0010] After reaching the preset number of rounds or reward convergence, the final transformed image and its consistency evaluation report are output.

[0011] Furthermore, the steps of obtaining the initial image dataset, which includes multiple original images and their corresponding semantic annotation information, constructing an image feature extraction network, extracting multi-scale feature maps of each original image based on a convolutional neural network structure, and generating an initial image feature vector set include:

[0012] Multiple original images are acquired, and each original image has corresponding semantic annotation information, including object category labels, location information labels, and occlusion status labels;

[0013] The original image is preprocessed, including adjusting the image size to a preset standard size and normalizing the color channels to standardize the range of pixel values.

[0014] A feature extraction network based on a convolutional neural network is constructed. This network contains multiple convolutional layers and pooling layers to extract feature maps of different scales from preprocessed images.

[0015] Each preprocessed original image is processed using a feature extraction network to generate feature maps at multiple scales, each scale feature map corresponding to a different spatial resolution;

[0016] Global average pooling is applied to the feature maps at each scale, and the pooled feature vectors are concatenated to form an initial image feature vector of fixed dimensions, which is then aggregated into an initial image feature vector set.

[0017] Furthermore, the step of constructing a reinforcement learning agent, which includes a policy network and a value function network, wherein the policy network is used to generate image transformation actions and the value function network is used to evaluate the long-term consistent reward of the actions, and the initial image feature vector is input into the policy network to output image transformation actions including rotation, scaling, brightness adjustment, and occlusion simulation, includes:

[0018] Define an image transformation action space, which includes rotation angle parameters, scaling ratio parameters, brightness adjustment parameters, and occlusion area simulation parameters;

[0019] A policy network is constructed, which adopts a multi-layer fully connected structure. The input is the initial image feature vector, and the output layer uses an activation function to output the specific values ​​of each parameter in the corresponding action space.

[0020] A value function network is constructed, which adopts a multi-layer fully connected structure similar to the policy network. The input is the initial image feature vector, and the output is a single scalar value, which is used to evaluate the long-term consistent reward of the action performed in the current state.

[0021] Each feature vector in the initial image feature vector set is sequentially input into the policy network, which then generates specific transformation action parameters corresponding to each original image.

[0022] Furthermore, the step of performing multiple rounds of image transformation, in each round transforming the original image according to the action output by the policy network to generate a transformed image, and calculating the consistency reward value between the current transformation and historical transformations based on the value function network, includes:

[0023] Initialize the transformation round counter, set the original image as the input image for the current round, and store the initial image feature vector and initial semantic mask as the benchmark for consistency comparison;

[0024] For each round of transformation, the corresponding transformation operation is performed on the current input image according to the action parameters output by the policy network, including rotating by a specified angle, scaling by a ratio, adjusting the overall brightness value of the image, and adding an occlusion effect with a specific transparency in a specified area to generate the transformed image;

[0025] Extract multi-scale feature maps from the transformed image and generate its feature vectors. At the same time, use a semantic segmentation model to generate a semantic mask for the transformed image.

[0026] The consistency reward value for the current round is calculated by: calculating the structural similarity between the transformed image and the previous image in terms of pixel structure, calculating the perceptual similarity between the transformed image and the initial image in the feature space, and calculating the overlap similarity between the current semantic mask and the initial semantic mask in the region.

[0027] The consistency reward value between the current transformation and the historical transformation is obtained based on the above three similarities.

[0028] Furthermore, the step of updating the parameters of the policy network and value function network based on the reward values ​​collected during the multi-round transformation process, and optimizing the image transformation strategy to maintain the content consistency of the generated image in complex scenes includes:

[0029] Multiple sets of experience data are randomly sampled from the experience replay buffer. Each set of experience data includes a state feature vector, the parameters of the executed action, the reward value obtained, and the next state feature vector.

[0030] For each set of sampled data, the value function network is used to calculate the current state value and the next state value, and the advantage function estimate is calculated to evaluate the superiority or inferiority of the current action relative to the average performance.

[0031] Based on the advantage function estimate and the ratio of new to old action probabilities output by the policy network, the loss function of the policy network is calculated. This loss function includes a policy gradient term and a pruning term used to limit the update magnitude.

[0032] The loss function of the value function network is calculated. This loss function is based on the difference between the state value predicted by the value function network and the actual cumulative reward value, and the mean squared error is used as the loss calculation standard.

[0033] Simultaneously, the parameters of the policy network and value function network are optimized, and the ability of the actions generated by the policy network to maintain image content consistency in complex scenes is gradually improved through multiple iterations.

[0034] Furthermore, the step of outputting the final transformed image and its consistency evaluation report after reaching a preset number of rounds or reward convergence includes:

[0035] The training process is monitored, and the training process ends when the number of rounds reaches the preset maximum number of rounds threshold or the average reward value of the most recent rounds changes less than the preset convergence threshold.

[0036] Select the transformed image with the highest reward value from the last round of transformation results as the final output image, and record all transformation action parameters corresponding to that image.

[0037] Generate a consistency evaluation report, which includes structural similarity indicators, perceptual similarity indicators, and semantic mask overlap indicators between the final image and the initial image, as well as reward value change curves and final stable values ​​during multiple rounds of training;

[0038] The final transformed image and its consistency evaluation report are packaged and output as a reference standard for maintaining content consistency and a basis for configuring transformation parameters in subsequent image generation tasks.

[0039] Furthermore, this invention also discloses a multi-round AI image generation content consistency transformation system based on reinforcement learning, comprising:

[0040] The acquisition module is used to acquire an initial image dataset, which includes multiple original images and their corresponding semantic annotation information. An image feature extraction network is constructed to extract multi-scale feature maps of each original image based on a convolutional neural network structure, and an initial image feature vector set is generated.

[0041] A construction module is used to construct a reinforcement learning agent, which includes a policy network and a value function network. The policy network is used to generate image transformation actions, and the value function network is used to evaluate the long-term consistent reward of the actions. The initial image feature vector is input into the policy network, and the output includes image transformation actions such as rotation, scaling, brightness adjustment, and occlusion simulation.

[0042] The execution module is used to perform multiple rounds of image transformation. In each round, the original image is transformed according to the action output by the policy network to generate the transformed image, and the consistency reward value between the current transformation and the historical transformation is calculated based on the value function network.

[0043] The generation module is used to update the parameters of the policy network and value function network based on the reward values ​​collected during multiple rounds of transformation, optimize the image transformation strategy, and maintain the content consistency of the generated image in complex scenes.

[0044] The output module is used to output the final transformed image and its consistency evaluation report after reaching a preset number of rounds or reward convergence.

[0045] Furthermore, the output module includes:

[0046] The monitoring unit is used to monitor the training process. When the number of transformation rounds reaches the preset maximum number of rounds threshold or the average reward value change of the most recent rounds is less than the preset convergence threshold, the training process is determined to end.

[0047] The selection unit is used to select the transformed image with the highest reward value from the last round of transformation results as the final output image, and at the same time record all transformation action parameters corresponding to the image.

[0048] A generation unit is used to generate a consistency evaluation report, which includes structural similarity indicators, perceptual similarity indicators, and semantic mask overlap indicators between the final image and the initial image, as well as reward value change curves and final stable values ​​during multiple rounds of training.

[0049] The output unit is used to package and output the final transformed image and its consistency evaluation report, which serves as a reference standard for maintaining content consistency and a basis for configuring transformation parameters in subsequent image generation tasks.

[0050] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described reinforcement learning-based multi-round AI image generation content consistency transformation method.

[0051] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described reinforcement learning-based multi-round AI image generation content consistency transformation method.

[0052] The beneficial effects of this application are as follows:

[0053] This invention constructs a reinforcement learning-based intelligent agent, combining multi-scale image feature extraction, multi-round transformation strategy optimization, and multi-dimensional consistency evaluation to achieve consistent content transformation during multi-round AI image generation in complex scenes. It effectively addresses the consistency degradation problem caused by scenarios such as occlusion, complex lighting, and dense interaction of multiple objects, ensuring that the image can maintain the stability of its core content after undergoing multiple transformations. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a method flow proposed in an embodiment of this application.

[0055] Figure 2 This is a schematic diagram of the system structure proposed in an embodiment of the present invention.

[0056] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] like Figure 1 As shown, this application provides a multi-round AI image generation content consistency transformation method based on reinforcement learning, including the following steps:

[0059] S1. Obtain an initial image dataset, which includes multiple original images and their corresponding semantic annotation information. Construct an image feature extraction network, extract multi-scale feature maps of each original image based on a convolutional neural network structure, and generate an initial image feature vector set.

[0060] S2, Construct a reinforcement learning agent, which includes a policy network and a value function network. The policy network is used to generate image transformation actions, and the value function network is used to evaluate the long-term consistent reward of the actions. The initial image feature vector is input into the policy network, and the output includes image transformation actions such as rotation, scaling, brightness adjustment, and occlusion simulation.

[0061] S3, perform multiple rounds of image transformation. In each round, the original image is transformed according to the action output by the policy network to generate the transformed image. The consistency reward value between the current transformation and the historical transformation is calculated based on the value function network. The calculation of the consistency reward value includes: calculating the structural similarity index between the current image and the previous image, the perceptual similarity measure between the current image and the initial image, and the intersection-union ratio between the current semantic mask and the initial semantic mask. The above three indicators are assigned preset weight coefficients and weighted summed to obtain the final consistency reward value.

[0062] S4, based on the reward values ​​collected during the multi-round transformation process, uses the near-end policy optimization algorithm to update the parameters of the policy network and the value function network, and optimizes the image transformation policy to maintain the content consistency of the generated image in complex scenes;

[0063] S5, after reaching the preset number of rounds or reward convergence, outputs the final transformed image and its consistency evaluation report for reference in subsequent image generation tasks.

[0064] As described in steps S1-S5 above, this invention constructs an intelligent agent based on reinforcement learning, combines multi-scale image feature extraction, multi-round transformation strategy optimization, and multi-dimensional consistency evaluation, to achieve consistent content transformation during multi-round AI image generation in complex scenes. This effectively addresses the consistency degradation problem caused by scenarios such as occlusion, complex lighting, and dense interaction of multiple objects, ensuring that the image can maintain the stability of its core content after undergoing multiple transformations.

[0065] When images undergo multiple transformations in complex scenes, occlusion can obscure parts of the content, complex lighting can alter pixel intensity distribution, and dense interactions between multiple objects can lead to dynamic changes in the relative positions and shapes of objects. These factors can all disrupt the consistency of the core image content (such as object categories, key structures, and semantic relationships). If the relationship between transformation operations and content consistency cannot be effectively balanced, the generated image may deviate from the original semantics or historical transformation trajectory, affecting the reliability of subsequent image generation tasks (such as data augmentation and scene simulation). Therefore, a method that can dynamically optimize transformation strategies and comprehensively evaluate consistency is needed.

[0066] In existing technologies, traditional image transformation methods are mostly single transformations with fixed rules, failing to consider the cumulative effect of multiple transformations. Furthermore, consistency evaluation often relies on a single metric (such as pixel-level similarity), making it difficult to address the consistency requirements at the semantic or high-level perceptual levels in complex scenarios. While some learning-based methods introduce iterative optimization, they lack consideration for long-term consistency, easily leading to problems where short-term transformations are reasonable but content shifts after multiple rounds. This invention, through a reinforcement learning framework, allows the agent to continuously learn the optimal strategy in multiple transformation rounds. By combining consistency rewards across structural, perceptual, and semantic dimensions, it achieves a dynamic balance between transformation actions and consistency, specifically addressing the consistency maintenance problem in complex scenarios.

[0067] The steps of acquiring the initial image dataset and extracting features lay the foundation for the entire transformation process. The initial image dataset contains the original images and their semantic annotations (such as object category, location, and occlusion state), which provide a reference benchmark for the semantic consistency of subsequent transformations. An image feature extraction network based on a convolutional neural network is constructed, extracting multi-scale feature maps through multiple convolutional and pooling layers. For example, low-level convolutional layers capture basic visual features such as edges and textures, while high-level convolutional layers extract abstract features such as object contours and semantic categories. Global average pooling is then used to obtain a fixed-dimensional feature vector, ensuring a comprehensive representation of different levels of image information. The significance of this step lies in the fact that only by fully grasping the multi-scale features of the image can subsequent transformations change the appearance of the image without destroying the content corresponding to the core features. For example, in scenes with dense interaction between multiple objects, high-level semantic features can help identify the interaction relationships between objects, ensuring that these relationships remain consistent after transformation.

[0068] Constructing a reinforcement learning agent is the core of realizing dynamic transformation strategies. The agent's policy network is responsible for generating specific transformation actions (rotation, scaling, brightness adjustment, occlusion simulation). Its multi-layer fully connected structure takes an initial feature vector as input and outputs continuous action parameters through the Tanh activation function (e.g., rotation angles are mapped to ±30 degrees to adapt to changes in the pose of objects in the scene, and occlusion simulation parameters are mapped to transparency of 0-1 to simulate different degrees of occlusion). These actions directly correspond to typical transformation requirements in complex scenes. The value function network evaluates the long-term consistent rewards of the actions, avoiding the policy network's pursuit of the rationality of a single transformation while ignoring the cumulative consistency degradation over multiple rounds. For example, in a complex lighting scene, if the brightness adjustment action only considers the rationality of the current round, it may cause the overall brightness to deviate from the original image after multiple rounds. The long-term evaluation of the value function network can constrain this deviation.

[0069] Performing multiple rounds of image transformation and calculating consistency rewards is a crucial step in maintaining consistency. In each round of transformation, actions based on the policy network adjust the image, generating the transformed image while extracting its features and semantic masks. The calculation of the consistency reward integrates three dimensions: the Structural Similarity Index (SSIM) reflects the pixel-level structural consistency between the current image and the previous image; for example, after scaling transformation, SSIM ensures that the image's edges and texture structure are not excessively destroyed. The Perceptual Similarity metric (such as cosine distance based on feature vectors) evaluates the consistency between the current image and the initial image at a high-level perceptual level; for example, even if changes in illumination cause pixel values ​​to change, perceptual similarity can still ensure the perceptual consistency of the semantic content "car". The Semantic Mask Intersection over Union (IoU) ensures the consistency of the position and extent of objects at the semantic level; for example, in multi-object interaction scenarios, the relative positions of object A and object B are maintained after transformation through the high IoU of the semantic mask. The weighted sum of these three factors provides a comprehensive measure of the consistency performance of the transformation at different levels, providing accurate feedback for subsequent policy optimization.

[0070] The step of updating network parameters based on reward values ​​is the core mechanism for optimizing transformation strategies. Data is sampled through an experience replay buffer, and a dominance function is calculated using temporal difference error to evaluate the relative merits of actions. This is then combined with the loss function of the PPO algorithm (including policy gradient and pruning terms) to update the policy network, ensuring the stability of parameter updates and avoiding consistency fluctuations caused by drastic adjustments. The value function network is optimized through mean squared error loss to improve the accuracy of reward evaluation. This process allows the policy network to continuously learn in complex scenarios: for example, in occluded scenes, through continuous optimization, the policy network learns to prioritize occluding non-core areas when adding occlusion, while adjusting brightness and scaling to compensate for visual changes caused by occlusion, thereby maintaining overall content consistency.

[0071] The steps of outputting the final result and evaluation report demonstrate the practicality of the method. When the number of transformation rounds reaches a preset value or the reward converges, the final image with the highest reward value is output. The corresponding transformation parameters have been optimized through multiple rounds and can maintain optimal consistency in complex scenarios. The various indicators in the consistency evaluation report, such as structural, perceptual, and semantic similarity and reward change curves, provide quantifiable reference standards for subsequent image generation tasks. For example, in the generation of drone aerial images, this report can guide how to generate multi-view images consistent with the content of the original image under cloud cover and lighting changes.

[0072] This invention utilizes a closed-loop optimization mechanism based on reinforcement learning, combined with multi-scale feature extraction and multi-dimensional consistency evaluation, to enable multi-round image transformations to perform the required transformation operations while maintaining the consistency of core content in complex scenarios such as occlusion, lighting, and multi-object interaction. This provides a reliable transformation strategy and evaluation basis for AI image generation tasks.

[0073] In one embodiment, the steps of obtaining an initial image dataset, which includes multiple original images and their corresponding semantic annotation information, constructing an image feature extraction network, extracting multi-scale feature maps of each original image based on a convolutional neural network structure, and generating an initial image feature vector set include:

[0074] S11, acquire multiple original images, each original image has corresponding semantic annotation information, the semantic annotation information includes object category label, location information label and occlusion status label, wherein the location information label is represented by bounding box coordinates and the occlusion status label is represented by a binary mask;

[0075] S12, preprocess the original image, including adjusting the image size to a preset standard size and normalizing the color channels to standardize the pixel value range;

[0076] S13, Construct a feature extraction network based on a convolutional neural network. This network contains multiple convolutional layers and pooling layers to extract feature maps of different scales from the preprocessed image.

[0077] S14, use a feature extraction network to process each preprocessed original image to generate feature maps at multiple scales, each scale feature map corresponding to a different spatial resolution;

[0078] S15. Apply global average pooling to the feature maps at each scale, concatenate the pooled feature vectors to form an initial image feature vector of fixed dimensions, and summarize them into an initial image feature vector set.

[0079] As described in steps S11-S15 above, by obtaining an initial image dataset containing complete semantic annotations, and after standardization preprocessing, a multi-scale feature map is extracted using a feature extraction network based on a convolutional neural network, and a fixed-dimensional initial image feature vector set is generated. This provides an accurate and comprehensive image feature foundation for the subsequent generation of transformation strategies and consistency evaluation of the reinforcement learning agent, ensuring that the content consistency of multiple rounds of image transformation in complex scenes has a reliable feature benchmark.

[0080] In complex scenes, such as occlusion, dense interaction of multiple objects, and complex lighting, the expression of visual features and semantic information in images exhibits multi-scale and complexity. Low-level features, such as edges and textures, are easily affected by lighting and occlusion, while high-level features, such as object categories and interaction relationships, determine the consistency of core content. If the feature extraction of the initial image is incomplete or lacks semantic information, subsequent transformation strategies may fail to accurately grasp the core content to be preserved. For example, in scenes with dense interaction of multiple objects, the lack of semantic annotations of object positions and occlusion states may lead to erroneous changes in the relative positions between objects during transformation. If only single-scale features are extracted, the loss of details or global information may result in biased consistency evaluation. Therefore, it is necessary to lay an accurate feature foundation for the entire transformation process through standardized dataset construction, preprocessing, and multi-scale feature extraction.

[0081] Acquire multiple original images and their corresponding semantic annotation information. The semantic annotation includes object category labels (such as "car" and "pedestrian") and location information labels (in bounding box coordinates). The semantic annotation process involves several steps: first, semantic annotation of the image content; and second, semantic annotation of the occlusion state (represented by a binary mask, where 1 indicates an occluded area and 0 indicates a visible area). This step is significant because semantic annotation provides clear "semantic anchors" for the image content, which is particularly crucial in complex scenes. For example, in scenes with dense interaction between multiple objects, bounding box coordinates can accurately locate the relative positions of "cars" and "pedestrians," and the binary mask can mark the areas where "cars" are occluded by "buildings." This information ensures that during subsequent transformations, the policy network clearly understands the object positional relationships and occlusion states that need to be maintained, avoiding consistency disruptions caused by semantic ambiguity.

[0082] The original image undergoes preprocessing, including resizing it to a preset standard size and normalizing the pixel values ​​of the RGB color channels (e.g., mapping pixel values ​​from the 0-255 range to the 0-1 range). Size normalization eliminates the influence of different image sizes on the input of the convolutional neural network, ensuring that the feature extraction network processes all images consistently. For example, if the original image sizes differ significantly (e.g., 100×100 vs. 500×500), the feature map sizes output by the convolutional layer will be inconsistent, making subsequent feature vectors incomparable. Color channel normalization stabilizes the distribution range of pixel values, reducing the interference of pixel value fluctuations caused by complex lighting on feature extraction. For example, in backlit scenes, the original pixel values ​​may be concentrated in the low range; normalization ensures consistency with the pixel distribution of images under normal lighting, allowing the feature extraction network to focus on content features rather than differences in lighting intensity.

[0083] A feature extraction network based on a convolutional neural network was constructed, consisting of five convolutional layers and three pooling layers: the first two convolutional layers use 3×3 kernels (64 and 128 output channels respectively) to extract low-level features such as edges and textures; the third to fifth convolutional layers use 5×5 kernels (256, 512, and 512 output channels respectively) to extract mid-level features such as object contours and local structures, as well as high-level features such as object categories and interaction relationships; the pooling layers use 2×2 max pooling with a stride of 2 to reduce the dimensionality of the feature map while retaining key features. The physical significance of this structural design is that, through the combination of multiple convolutional and pooling layers, features at different levels can be gradually abstracted from the original image, adapting to the multi-scale feature requirements of complex scenes. For example, in occluded scenes, the first convolutional layer can capture the edge features of the unoccluded areas, while the fifth convolutional layer can still identify the category of the occluded object based on global information, ensuring the integrity of the features.

[0084] The feature extraction network is used to process the preprocessed original image, generating feature maps at three scales: a 112×112 feature map output from the second convolutional layer (corresponding to low-level detail features), a 56×56 feature map output from the fourth convolutional layer (corresponding to mid-level structural features), and a 28×28 feature map output from the fifth convolutional layer (corresponding to high-level semantic features). The generation of multi-scale feature maps is to accommodate feature representation at different levels. For example, in scaling scenarios, the 112×112 feature map preserves the image's detailed texture, ensuring consistency after scaling; the 28×28 feature map reflects the overall semantics of the object, ensuring that the object category is not misclassified after scaling. The combination of both allows the features to comprehensively support subsequent consistency evaluation.

[0085] Global average pooling is applied to the feature maps at each scale (calculating the average value of each channel), resulting in 64-dimensional (corresponding to a 112×112 feature map), 512-dimensional (corresponding to a 56×56 feature map), and 512-dimensional (corresponding to a 28×28 feature map) feature vectors. These three vectors are then concatenated sequentially to form a 1088-dimensional initial image feature vector. The feature vectors from all images are then aggregated to form the initial image feature vector set. Global average pooling reduces the number of parameters while preserving global features at that scale, thus avoiding overfitting. The concatenated fixed-dimensional feature vector provides a unified input dimension for the policy network, ensuring the comparability of features from different images. For example, in a brightness adjustment scenario, the 1088-dimensional feature vector integrates low-level brightness-sensitive features and high-level semantic features, enabling the policy network to ensure that brightness changes meet scene requirements while maintaining the consistency of object categories corresponding to high-level semantic features.

[0086] Through the above steps, this invention provides an initial feature base for subsequent reinforcement learning agents, which includes complete semantic information, is standardized, and covers multi-scale features. This effectively supports content consistency evaluation and strategy optimization for multi-round image transformations in complex scenarios, avoids transformation deviations caused by missing or inconsistent features, and significantly improves the applicability and reliability of the method in scenarios such as occlusion, multi-object interaction, and complex lighting.

[0087] In one embodiment, the step of constructing a reinforcement learning agent, the agent comprising a policy network and a value function network, wherein the policy network generates image transformation actions and the value function network evaluates the long-term consistent reward of the actions, and the step of inputting the initial image feature vector into the policy network and outputting image transformation actions including rotation, scaling, brightness adjustment, and occlusion simulation includes:

[0088] S21, Define the image transformation action space, which includes rotation angle parameters, scaling ratio parameters, brightness adjustment parameters, and occlusion area simulation parameters, each of which is a continuous value;

[0089] S22, Construct a policy network. The policy network adopts a multi-layer fully connected structure. The input is the initial image feature vector, and the output layer uses the Tanh activation function to output the specific values ​​of each parameter in the corresponding action space.

[0090] S23. Construct a value function network. The value function network adopts a multi-layer fully connected structure similar to the policy network. The input is the initial image feature vector, and the output is a single scalar value, which is used to evaluate the long-term consistent reward of the action performed in the current state.

[0091] S24, each feature vector in the initial image feature vector set is sequentially input into the policy network, and the policy network generates specific transformation action parameters corresponding to each original image. The action parameters output by the policy network are mapped to a predefined physical change range, where the rotation angle is mapped to a range of ±30 degrees, the scaling ratio is mapped to a range of 0.8 to 1.2 times, the brightness adjustment is mapped to a range of ±50 units, and the occlusion simulation parameters are mapped to transparency values ​​between 0 and 1.

[0092] As described in steps S21-S24 above, by defining a continuous action space that covers the main transformation requirements of complex scenes, a policy network and value function network adapted to image features are constructed. The action parameters output by the network are mapped to a reasonable physical range, forming a reinforcement learning agent that can generate accurate transformation actions and evaluate their long-term consistency. This provides a stable action generation mechanism and reward evaluation basis for multi-round image transformation, ensuring that in scenarios such as occlusion, complex lighting, and dense interaction of multiple objects, the transformation actions can meet the scene requirements and maintain the long-term consistency of the content.

[0093] The image transformation action space is defined, including rotation angle parameters, scaling parameters, brightness adjustment parameters, and occlusion region simulation parameters, with each parameter being a continuous value. The significance of this design lies in the fact that continuous parameters provide adjustment precision down to multiple decimal places, covering diverse transformation needs in complex scenes. For example, in occlusion scenes, continuous values ​​of the occlusion region simulation parameter (e.g., 0.2, 0.3, 0.4) can achieve a smooth transition from slight to moderate occlusion, avoiding abrupt changes in occlusion effects caused by discrete values ​​(e.g., 0, 0.5, 1), and protecting key features of objects in the image from sudden obscuring. In complex lighting scenes, continuous changes in the brightness adjustment parameter (e.g., +10, +20 units) can gradually adapt to changes in light intensity, avoiding blurring of object outlines due to sudden brightness changes.

[0094] A policy network is constructed using a multi-layer fully connected structure: the input layer has 1088 dimensions, the first hidden layer has 512 dimensions (using ReLU activation), the second hidden layer has 256 dimensions (ReLU activation), the third hidden layer has 128 dimensions (ReLU activation), and the output layer has 4 dimensions (corresponding to four action parameters) using the Tanh activation function. The output range of the Tanh function is [-1, 1], which provides a standardized basis for subsequent mapping to the physical scale. The significance of this structure lies in the fact that it deeply processes the initial image feature vector through a multi-layer fully connected network, transforming multi-scale image features (low-level texture, high-level semantics) into specific action parameters, achieving precise adaptation between actions and image content. For example, for a densely interactive image containing "pedestrians" and "bicycles," the policy network analyzes the positional relationship features between the two in the feature vector, generating a small rotation angle parameter (e.g., +5 degrees) and a scaling parameter close to 1.0, ensuring that the relative positions of the two remain unchanged after rotation and that details are clear after scaling.

[0095] A value function network is constructed, employing a multi-layer fully connected structure similar to the policy network: an input layer of 1088 dimensions, hidden layers of 512, 256, and 128 dimensions, and an output layer of 1 dimension (a single scalar). Its significance lies in evaluating the long-term impact of an action on consistency across multiple rounds of transformation based on image features and the current action, avoiding the policy network's pitfall of "optimal in a single transformation but cumulative deviation over multiple rounds." For example, an action that adjusts the brightness to +50 units in the current round might produce a striking visual effect. However, the value function network, by analyzing the initial image's illumination features in the feature vector, predicts that the brightness will deviate from the initial state after multiple rounds, thus outputting a lower reward value and constraining the policy network to adjust the brightness parameter to +20 units to maintain long-term consistency.

[0096] Each feature vector in the initial image feature vector set is sequentially input into the policy network to generate specific transformation action parameters for each original image. The initial image feature vectors here come from the 1088-dimensional vector generated in step S15 above, containing multi-scale features and semantic information of the image. This step ensures that the action parameters are highly correlated with the image content. For example, for an image with an occlusion status label displaying "the left 1 / 3 of the area is occluded," after receiving a vector containing this occlusion feature, the policy network will generate a rotation angle biased to the right (e.g., -3 degrees) and a lower occlusion transparency (e.g., 0.3) to avoid further occluding the already exposed key areas.

[0097] The action parameters output by the policy network are mapped to a predefined range of physical changes: rotation angle is mapped to ±30 degrees by multiplying the output value by 30 (e.g., Tanh output 0.1 corresponds to 3 degrees, -0.2 corresponds to -6 degrees), scaling ratio is mapped to 0.8-1.2 times by "0.8+(output value+1)×0.2" (e.g., output 0 corresponds to 1.0 times, 0.5 corresponds to 1.1 times), brightness adjustment is mapped to ±50 units by "output value×50" (e.g., output -0.3 corresponds to -15 units), and occlusion simulation parameters are mapped to 0-1 by "(output value+1) / 2" (e.g., output -1 corresponds to 0, 0 corresponds to 0.5). The physical meaning of this mapping is to transform the standardized values ​​output by the network into effective parameters that conform to the physical laws of image transformation, avoiding invalid actions that exceed reasonable ranges: for example, limiting the rotation angle to ±30 degrees can prevent objects from moving out of the image boundary due to excessive rotation; the scaling ratio is not less than 0.8 times to ensure that objects do not lose identifiable features due to excessive shrinkage, thereby providing a basic guarantee for content consistency while meeting the needs of complex scene transformations.

[0098] The reinforcement learning agent constructed in this invention can generate continuous transformation actions adapted to complex scenes based on the multi-scale features of images. It evaluates the long-term consistency impact of the actions through a value function network and ensures the effectiveness of the actions through physical range mapping. This provides a stable, accurate action generation and evaluation mechanism that takes into account long-term consistency for multi-round image transformations. It effectively solves the problem of consistency decline caused by coarse actions and short-sighted evaluation in traditional methods and significantly improves the content stability of multi-round transformations in complex scenes.

[0099] In one embodiment, the step of performing multiple rounds of image transformation, in each round transforming the original image according to the action output by the policy network to generate a transformed image, and calculating the consistency reward value between the current transformation and historical transformations based on the value function network includes:

[0100] S31, initialize the transformation round counter, set the original image as the input image for the current round, and store the initial image feature vector and initial semantic mask as the benchmark for consistency comparison;

[0101] S32, For each round of transformation, perform corresponding transformation operations on the current input image according to the action parameters output by the policy network, including rotating by a specified angle, scaling by a ratio, adjusting the overall brightness value of the image, and adding an occlusion effect with specific transparency in a specified area to generate the transformed image;

[0102] S33 uses an image feature extraction network to extract multi-scale feature maps of the transformed image and generate its feature vectors, while using a semantic segmentation model to generate a semantic mask of the transformed image.

[0103] S34, calculate the consistency reward value for the current round, including: calculating the structural similarity between the transformed image and the previous round image in terms of pixel structure, calculating the perceptual similarity between the transformed image and the initial image in the feature space, and calculating the overlap similarity between the current semantic mask and the initial semantic mask in the region.

[0104] S35. Based on the above three similarities, obtain the consistency reward value between the current transformation and the historical transformation. That is, multiply the above three similarity measurement results by preset weight coefficients and add them together to obtain the final consistency reward value of the current transformation round. Then, store the reward value, the current state feature vector and the action parameters taken into the experience playback buffer.

[0105] As described in steps S31-S35 above, by initializing the multi-round transformation benchmark, multi-round image transformation is performed according to the action parameters output by the policy network. The features and semantic information of the transformed image are extracted, and the consistency reward value covering multiple dimensions of structure, perception, and semantics is calculated and stored. This provides real-time and comprehensive feedback for the policy optimization of the reinforcement learning agent, ensuring that the cumulative effect of multi-round transformation can maintain the consistency of image content in complex scenarios such as occlusion, complex lighting, and dense interaction of multiple objects.

[0106] Multi-round image transformations in complex scenes are a dynamic and cumulative process. Each round of rotation, scaling, and other actions affects the pixel distribution, feature representation, and semantic regions of the image. Small deviations in a single transformation can be amplified over multiple rounds, leading to a severe decline in content consistency. For example, in scenarios with dense multi-object interactions, a slight scaling of an object in the first round might cause subtle deviations in the relative positions of objects. If these deviations are not corrected in subsequent rounds, objects may eventually become completely detached from the interaction. In occlusion scenarios, if the newly added occluded area in each round does not match the initial semantics, it will gradually obscure key objects, compromising semantic integrity. Therefore, it is necessary to track the consistency of each round of transformation with the historical and initial states in real time, quantitatively evaluate the transformation effect, and record experience to provide a basis for strategy optimization and avoid the accumulation of deviations.

[0107] The transformation round counter is initialized, the original image is set as the input image for the current round, and the initial image feature vector and initial semantic mask are stored as a benchmark for consistency comparison. The significance of this step is to establish an origin for multiple rounds of transformation. The consistency evaluation of all subsequent transformations must refer to the initial state, while simultaneously tracking the relationship with the previous round, forming a three-dimensional comparison system of initial-current-historical. For example, in a scene with dense interaction of multiple objects, the initial semantic mask marks the "pedestrian" as being to the left of the "bicycle" (bounding box coordinates are (50, 100, 150, 200) and (160, 100, 260, 200)). Storing this benchmark ensures that after each round of transformation, the system can accurately check whether the positional relationship between the two remains consistent, avoiding situations such as the "pedestrian" appearing to the right of the "bicycle" after multiple rounds of transformation.

[0108] For each transformation round, corresponding operations are performed on the current input image based on the action parameters output by the policy network (mapped physical parameters, such as rotation of 5 degrees, scaling by 1.1 times, brightness +20 units, and occlusion transparency of 0.3): rotation is achieved through affine transformation, ensuring that the object pixel coordinates are accurately mapped by angle; scaling uses bilinear interpolation to preserve object details during enlargement or reduction; brightness adjustment is achieved by uniformly adding or subtracting adjustment parameters to the RGB channel pixel values ​​to maintain color balance; occlusion simulation overlays a gray mask with transparency of 0.3 on a specified area (a non-core area located by the semantic segmentation model) to simulate a realistic occlusion effect. The physical significance of these operations is that, while meeting the transformation requirements of complex scenes, precise parameter control reduces the noise introduced by the transformation. For example, in complex lighting scenes, a brightness adjustment of +20 units can improve image clarity without causing distortion of the color features of "traffic lights" due to over-adjustment, providing a reliable transformed image for subsequent consistency evaluation.

[0109] The constructed image feature extraction network (containing 5 convolutional layers and 3 pooling layers) is used to extract multi-scale feature maps of the transformed image. After global average pooling and concatenation, a 1088-dimensional feature vector is generated. Simultaneously, a semantic segmentation model (such as U-Net, which includes an encoder and decoder structure; the encoder extracts features through convolutional layers, and the decoder restores spatial resolution through deconvolutional layers) is used to generate a binary semantic mask for the transformed image. Here, 1 represents an object region, 0 represents a background region, and occluded regions are separately labeled as 2. This step is significant because it provides comparable feature and semantic data to the initial state for consistency evaluation. For example, in occluded scenes, the transformed feature vector needs to retain the high-level semantic features of the occluded object (e.g., the "car" category), and the semantic mask needs to accurately label the newly added occluded regions to ensure overlap calculation with the initial mask and avoid evaluation bias caused by inconsistencies in feature or mask formats.

[0110] The consistency reward value for the current round is calculated as follows: The Structural Similarity Index (SSIM) calculates the mean, variance, and covariance of the transformed image and the previous image to obtain a pixel-level structural consistency score (range 0-1). For example, if the edges of the "buildings" in the previous image are clear, the SSIM after the current round is 0.92, indicating that the edge structure has not been destroyed. The perceptual similarity measure calculates the cosine distance between the feature vectors of the transformed image and the feature vectors of the initial image (range 0-1, the smaller the value, the higher the similarity). For example, the cosine distance between the tall features of the "trees" after the transformation and the initial features is 0.15, indicating good perceptual consistency. The Semantic Mask Intersection-Union Ratio (IoU) calculates the ratio of the intersection and union of the object regions in the current semantic mask and the initial semantic mask (range 0-1). For example, the IoU of the "pedestrian" region is 0.85, indicating high position and range consistency. The three factors are multiplied by preset weight coefficients of 0.3, 0.4, and 0.3 respectively, and then added together to obtain the final reward value (e.g., 0.92×0.3+(1-0.15)×0.4+0.85×0.3=0.88). The physical meaning of this calculation is to comprehensively evaluate consistency from different levels: structural similarity ensures the coherence of low-level visual features, perceptual similarity ensures the stability of high-level semantic perception, and semantic mask IoU ensures that object regions and relationships remain unchanged. The weighted average of the three factors balances the consistency requirements of different dimensions in complex scenarios. For example, in multi-object interaction scenarios, the high weight (0.3) of semantic mask IoU ensures that the core semantic of the relative position of objects is not violated.

[0111] The reward value (0.88), the current state feature vector (1088-dimensional features of the transformed image), and the action parameters (rotation of 5 degrees, etc.) are stored in an experience replay buffer (using a FIFO queue with a capacity of 10,000 entries). The physical significance of this step is to accumulate "experience cases" from multiple rounds of transformations, providing real and diverse training data for subsequent network parameter updates. For example, the buffer stores action-reward correspondences under different occlusion levels and lighting conditions, enabling the policy network to learn which actions to take in which scenarios to maintain consistency and avoid policy overfitting due to insufficient data.

[0112] Through the above steps, this invention achieves dynamic execution of multi-round image transformation, accurate evaluation of multi-dimensional consistency, and effective storage of experience data. It provides real-time and comprehensive feedback for reinforcement learning agents, ensuring that each round of transformation can be optimized in the direction of maintaining content consistency in complex scenarios. This avoids the distortion of the final image content caused by the accumulation of deviations, and significantly improves the stability and reliability of multi-round transformation.

[0113] In one embodiment, the step of updating the parameters of the policy network and the value function network based on the reward values ​​collected during the multi-round transformation process, and optimizing the image transformation policy to maintain the content consistency of the generated image in complex scenes includes:

[0114] S41, randomly sample multiple sets of experience data from the experience replay buffer. Each set of experience data includes a state feature vector, the parameters of the executed action, the reward value obtained, and the next state feature vector.

[0115] S42, For each set of sampled data, the current state value and the next state value are calculated using a value function network, and the advantage function estimate is calculated based on the time difference error method to evaluate the superiority or inferiority of the current action relative to the average performance.

[0116] S43. Based on the advantage function estimate and the ratio of new to old action probabilities output by the policy network, calculate the loss function of the policy network. This loss function includes a policy gradient term and a pruning term used to limit the update magnitude, ensuring the stability of the policy update.

[0117] S44, calculate the loss function of the value function network. This loss function is based on the difference between the state value predicted by the value function network and the actual cumulative reward value, and uses the mean square error as the loss calculation standard.

[0118] S45 uses the stochastic gradient descent algorithm to simultaneously optimize the parameters of the policy network and the value function network, and gradually improves the ability of the actions generated by the policy network to maintain the consistency of image content in complex scenes through multiple iterations.

[0119] As described in steps S41-S45 above, this invention samples empirical data from multiple rounds of transformation from the empirical playback buffer, calculates the advantage function based on the temporal difference error, constructs a policy network loss function containing a policy gradient term and a pruning term, and a value function network loss function based on the mean square error. The network parameters are optimized using the stochastic gradient descent algorithm to achieve iterative improvement of the transformation strategy. This enables the reinforcement learning agent to generate more stable transformation actions that maintain the consistency of image content in complex scenarios such as occlusion, complex lighting, and dense interaction of multiple objects.

[0120] Optimizing multi-round image transformation strategies is a dynamic learning process: the actions generated by the initial policy network may be effective in simple scenes, but in complex scenes (such as those with occlusion and lighting changes), consistency deviations are prone to occur, and these deviations accumulate with each round. For example, in scenes with dense interaction of multiple objects, the initial policy may overscale a particular object to avoid occlusion, leading to an imbalance in the relative proportions of that object to other objects after multiple rounds. In complex lighting scenes, if the brightness adjustment strategy does not consider the long-term cumulative effect, the image may gradually deviate from the initial tone. Therefore, it is necessary to continuously adjust the network parameters based on the actual reward feedback of multiple rounds of transformation, allowing the policy network to gradually learn the optimal action pattern that balances transformation requirements and consistency, thus avoiding the insufficient adaptability of fixed policies in complex scenes.

[0121] Multiple sets of empirical data are randomly sampled from the experience replay buffer. Each set includes a state feature vector (a 1088-dimensional feature vector of the transformed image), the parameters of the executed action (e.g., rotation by 5 degrees, scaling by 1.1 times), the reward value obtained (e.g., 0.88), and the next state feature vector (the next round of image features generated after the action is executed). Sampling is done using a uniform random method, with 32 sets of data sampled per batch. The significance of this step is to break the temporal correlation of data through random sampling, avoiding the network learning irrelevant temporal noise, and at the same time, to make parameter updates more generalizable by utilizing the diverse scene data accumulated in the buffer (e.g., experience under different occlusion levels and lighting conditions). For example, in occluded scenes, multiple sets of data including those with slight and moderate occlusion are sampled to ensure that the policy network learns actions adapted to different occlusion levels, rather than only adapting to a specific occlusion situation.

[0122] For each set of sampled data, the current state value is calculated using a value function network. and the next state value The advantage function estimate is calculated based on the time difference error (TD error), and the calculation formula is as follows:

[0123] ;

[0124] Where Q represents the estimate of the advantage function, This is the current reward value. This is a discount factor (which can be set to 0.9) used to balance immediate and future rewards. A positive dominance function indicates that the current action is better than the average, while a negative one indicates that it is worse than the average. The significance of this step is to quantify the relative value of each action, avoiding misjudgments caused by relying solely on absolute reward values. For example, in a complex lighting scenario, an action might have a reward value of 0.7 (not a high absolute value), but a dominance function of 0.2 (indicating it's better than the average action in that state). In this case, the action should be encouraged to ensure the policy adjusts towards a relatively better outcome.

[0125] Based on the advantage function estimate and the ratio of new to old action probabilities output by the policy network, calculate the loss function of the policy network:

[0126] ;

[0127] in, The loss value of the policy network is a quantitative indicator that measures the degree of deviation between the generated transformation action and the optimal consistency policy under the current policy network parameters. E[·] represents the expectation of the expression in parentheses, which is achieved by calculating the average value of multiple sets of data (such as 32 sets) sampled in the experience replay buffer. Its significance is to eliminate the influence of noise in single data and make the loss function more stably reflect the optimization direction of the overall policy. For example, in the experience data of multiple rounds of transformation, individual samples may have abnormal reward values ​​due to extreme scenarios (such as severe occlusion). Taking the expectation can smooth out such fluctuations and ensure that parameter updates are not disturbed by outliers. This represents the ratio of the probability of actions under the new strategy to that under the old strategy. This is the pruning parameter, a preset constant (usually set to 0.2), used to limit the fluctuation range of the probability ratio between the new and old policies. Its purpose is to prevent training instability caused by excessively large policy updates, ensuring that the new policy maintains a certain degree of similarity to the old policy. For example, When the value is 0.2, the probability ratio is limited to the interval [0.8, 1.2] to avoid the strategy suddenly jumping from moderate rotation to excessive rotation due to a single data point. It is a clipping function, when <1- When, output 1- ;when >1+ When, output 1+ Otherwise, output directly. Its physical meaning is to set a safety boundary for the probability ratio, avoiding excessive differences between the new and old strategies. For example, when... =1.5 (more than 1+) When =1.2), the output after clipping is 1.2; when =0.7 (less than 1- When the value is 0.8, the output after pruning is 0.8, ensuring that the strategy update is always within a controllable range.

[0128] The loss function of the value function network is calculated using the mean squared error, and its formula is as follows:

[0129] ;

[0130] Here, L represents the loss value of the value function network, which is the squared difference between the predicted current state value and the actual cumulative reward (immediate reward + discounted next state value). Its physical meaning is that by minimizing the prediction error, the value function network improves the accuracy of reward evaluation, providing a reliable basis for the calculation of the dominance function. For example, in an occluded scenario, if the value function network predicts a reward of 0.8 for a certain state, while the actual cumulative reward is 0.6, the mean squared error will drive the network to adjust its parameters, making the predicted value closer to 0.6, ensuring the accuracy of subsequent dominance function calculations.

[0131] Stochastic gradient descent is used to simultaneously optimize the parameters of both networks, with a learning rate of 0.0003. Each iteration is based on the same batch of data, and the number of iterations is set to 10. The parameters of the policy network (such as the weights and biases of fully connected layers) and the value function network are updated synchronously to ensure their coordinated adaptation. The physical significance of this step is to allow action generation and reward evaluation to evolve synchronously, preventing the value function network from failing to accurately evaluate the value of actions after the policy network is updated. For example, in complex lighting scenarios, when the policy network learns a better action by reducing the brightness adjustment by 5 units, the value function network synchronously updates its evaluation criteria to accurately identify the long-term consistent value of the action, forming a closed loop of generation-evaluation-optimization.

[0132] Through the above steps, the present invention achieves stable and efficient updating of network parameters, enabling the policy network to gradually learn the optimal transformation strategy in complex scenarios through multiple iterations. For example, in scenarios where occlusion and illumination changes are superimposed, it can generate a combination of actions such as small-amplitude rotation to avoid the core occlusion area and moderate brightness adjustment to match the illumination, ensuring the long-term consistency of image content and significantly improving the adaptability and reliability of the method in complex scenarios.

[0133] In one embodiment, the step of outputting the final transformed image and its consistency evaluation report after reaching a preset number of rounds or reward convergence includes:

[0134] S51, monitor the training process, and determine the end of the training process when the number of transformation rounds reaches the preset maximum number of rounds threshold or the average reward value change of the most recent rounds is less than the preset convergence threshold.

[0135] S52, select the transformed image with the highest reward value from the last round of transformation results as the final output image, and record all transformation action parameters corresponding to the image;

[0136] S53, Generate a consistency evaluation report, which includes structural similarity index, perceptual similarity index, and semantic mask overlap index between the final image and the initial image, as well as the reward value change curve and the final stable value during multiple rounds of training;

[0137] S54 packages and outputs the final transformed image and its consistency evaluation report, which will serve as a reference standard for maintaining content consistency and a basis for configuring transformation parameters in subsequent image generation tasks.

[0138] As described in steps S51-S54 above, the training process is monitored to determine a reasonable time to end the training. The optimal image and corresponding parameters are selected from the final round of transformation results, and an evaluation report containing multi-dimensional consistency indicators and training process data is generated. This provides a quantifiable and reusable content consistency transformation reference for subsequent image generation tasks, ensuring the reliability and practicality of the transformation results in complex scenarios such as occlusion, complex lighting, and dense interaction of multiple objects.

[0139] The training process for multi-round image transformations requires finding a balance between sufficient optimization and resource consumption. Insufficient training means the policy network fails to learn the optimal transformation strategy, potentially resulting in inconsistent images. Overtraining not only wastes computational resources but may also lead to decreased model adaptability in new scenes due to overfitting. Furthermore, the transformation results must be accompanied by detailed evaluation information; otherwise, subsequent tasks (such as data augmentation and scene transfer) cannot determine whether the results meet consistency requirements, and transformation parameters are difficult to reuse. For example, in scenes with dense multi-object interactions, a transformed image that has not undergone sufficient evaluation may superficially meet visual consistency, but the semantic relationships between objects have been disrupted, and direct reuse will lead to errors in subsequent generation tasks. Therefore, it is necessary to define clear training termination conditions, select the optimal results, and quantify their evaluation to provide a basis for the application of the results.

[0140] The training process is monitored to determine when to end the training. A preset maximum number of rounds threshold is set to 50 rounds, and a convergence threshold of 0.01 (i.e., the average reward value change in the most recent 5 rounds is less than 0.01). The average reward value of each round is tracked in real time. When the number of rounds reaches 50, or the average reward value fluctuation in 5 consecutive rounds (e.g., from 0.91 to 0.92 and then to 0.915, with a change of 0.005) is less than 0.01, training is considered to end. The physical meaning of this step is to ensure the sufficiency and efficiency of training through two conditions: the maximum number of rounds avoids infinite training, and the convergence condition ensures that the strategy has stabilized in the optimal state. For example, in a complex lighting scene, if the average reward value increases from 0.88 to 0.92 in round 30, and stabilizes at 0.92±0.004 in rounds 31-35, satisfying the convergence condition, ending the training at this point is more efficient than completing 50 rounds, and the results are equally reliable.

[0141] The image with the highest reward value from the final transformation is selected as the final output, and all corresponding transformation action parameters are recorded (e.g., rotation angle of 3 degrees, scaling ratio of 1.05 times, brightness adjustment of +15 units, occlusion transparency of 0.2). The final transformation typically includes multiple candidate images (e.g., 3-5 generated from different random seeds), each corresponding to a set of action parameters and a reward value. Selecting the image with the highest reward value (e.g., 0.92) signifies that this image exhibits the best overall performance in terms of structural, perceptual, and semantic consistency. The physical significance of this step is to ensure the quality of the output result, while recording parameters provides a basis for subsequent reuse. For example, in occluded scenes, the recorded occlusion transparency parameter of 0.2 can be directly reused for image transformations with similar occlusion levels, ensuring the stability of consistent performance.

[0142] A consistency evaluation report is generated, which includes: a structural similarity index between the final image and the initial image (e.g., 0.91, reflecting pixel-level structural consistency), a perceptual similarity metric (e.g., cosine distance of feature vectors, 0.12; a smaller value indicates greater consistency in high-level perception), a semantic mask overlap index (e.g., intersection-over-union ratio, 0.89, reflecting consistency in object regions and relationships), a reward value change curve during multiple training rounds (gradually increasing from 0.65 in the initial rounds to 0.92 in the stable period), and a final stable value (0.92). The physical meaning of these indices lies in quantifying consistency from different dimensions: structural similarity ensures the coherence of low-level visual features, perceptual similarity guarantees the stability of high-level semantic cognition, semantic mask overlap maintains the core information of objects unchanged, and the reward curve reflects the process and effect of strategy optimization. For example, in a multi-object dense interaction scenario, a semantic mask overlap index of 0.89 indicates that the relative position and range of the transformed objects are highly consistent with the initial state, providing a clear semantic consistency standard for subsequent scene simulation tasks.

[0143] The final transformed image and its consistency evaluation report are packaged and output as a reference standard and parameter configuration basis for maintaining content consistency in subsequent image generation tasks. For example, in autonomous driving dataset augmentation tasks, parameters such as rotation angle and scaling ratio in the report can be directly referenced to ensure that the new data and the original data maintain consistency in content. In virtual scene generation, the generation strategy can be adjusted based on structural similarity and perceptual similarity indicators to match the best historical consistency performance. The physical significance of this step is to achieve the transferability of transformation results, transforming the optimization results of a single task into a universal reference standard, thereby improving the efficiency and reliability of the overall image generation workflow.

[0144] Through the above steps, this invention not only outputs the transformed image with optimal content consistency in complex scenarios, but also provides complete evaluation criteria and parameter records. It solves the problems of difficult verification of result reliability and difficulty in parameter reuse in traditional methods, enabling the content consistency transformation results of multi-round AI image generation to effectively support various subsequent image generation tasks, and significantly improving the practical value and scalability of the method.

[0145] like Figure 2 As shown, this invention also discloses a multi-round AI image generation content consistency transformation system based on reinforcement learning, comprising:

[0146] The acquisition module 1 is used to acquire an initial image dataset, which includes multiple original images and their corresponding semantic annotation information. An image feature extraction network is constructed to extract multi-scale feature maps of each original image based on a convolutional neural network structure, and an initial image feature vector set is generated.

[0147] Module 2 is used to construct a reinforcement learning agent. The agent includes a policy network and a value function network. The policy network is used to generate image transformation actions, and the value function network is used to evaluate the long-term consistent reward of the actions. The initial image feature vector is input into the policy network, and the output includes image transformation actions such as rotation, scaling, brightness adjustment, and occlusion simulation.

[0148] Execution module 3 is used to perform multiple rounds of image transformation. In each round, the original image is transformed according to the action output by the policy network to generate the transformed image, and the consistency reward value between the current transformation and the historical transformation is calculated based on the value function network.

[0149] The generation module 4 is used to update the parameters of the policy network and the value function network based on the reward values ​​collected during the multi-round transformation process, optimize the image transformation strategy, and maintain the content consistency of the generated image in complex scenes.

[0150] Output module 5 is used to output the final transformed image and its consistency evaluation report after reaching a preset number of rounds or reward convergence.

[0151] In one embodiment, the output module includes:

[0152] The monitoring unit is used to monitor the training process. When the number of transformation rounds reaches the preset maximum number of rounds threshold or the average reward value change of the most recent rounds is less than the preset convergence threshold, the training process is determined to end.

[0153] The selection unit is used to select the transformed image with the highest reward value from the last round of transformation results as the final output image, and at the same time record all transformation action parameters corresponding to the image.

[0154] A generation unit is used to generate a consistency evaluation report, which includes structural similarity indicators, perceptual similarity indicators, and semantic mask overlap indicators between the final image and the initial image, as well as reward value change curves and final stable values ​​during multiple rounds of training.

[0155] The output unit is used to package and output the final transformed image and its consistency evaluation report, which serves as a reference standard for maintaining content consistency and a basis for configuring transformation parameters in subsequent image generation tasks.

[0156] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described reinforcement learning-based multi-round AI image generation content consistency transformation method.

[0157] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described reinforcement learning-based multi-round AI image generation content consistency transformation method.

[0158] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0159] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0160] The above description is merely a preferred embodiment of the present invention and does not limit the scope of this application. Any equivalent results or equivalent process transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.

Claims

1. A multi-round AI image generation content consistency transformation method based on reinforcement learning, characterized in that, Includes the following steps: An initial image dataset is obtained, which includes multiple original images and their corresponding semantic annotation information. An image feature extraction network is constructed, and multi-scale feature maps of each original image are extracted based on a convolutional neural network structure, and an initial image feature vector set is generated. A reinforcement learning agent is constructed, which includes a policy network and a value function network. The policy network is used to generate image transformation actions, and the value function network is used to evaluate the long-term consistent reward of the actions. The initial image feature vector is input into the policy network, and the output includes image transformation actions such as rotation, scaling, brightness adjustment, and occlusion simulation. Perform multiple rounds of image transformation. In each round, the original image is transformed according to the action output by the policy network to generate the transformed image, and the consistency reward value between the current transformation and the historical transformation is calculated based on the value function network. Based on the reward values ​​collected during the multi-round transformation process, the parameters of the policy network and the value function network are updated to optimize the image transformation strategy, which is used to maintain the content consistency of the generated image in complex scenes. After reaching the preset number of rounds or reward convergence, the final transformed image and its consistency evaluation report are output.

2. The multi-round AI image generation content consistency transformation method based on reinforcement learning according to claim 1, characterized in that, The steps of obtaining the initial image dataset, which includes multiple original images and their corresponding semantic annotation information, constructing an image feature extraction network, extracting multi-scale feature maps of each original image based on a convolutional neural network structure, and generating an initial image feature vector set include: Multiple original images are acquired, and each original image has corresponding semantic annotation information, including object category labels, location information labels, and occlusion status labels; The original image is preprocessed, including adjusting the image size to a preset standard size and normalizing the color channels to standardize the range of pixel values. A feature extraction network based on a convolutional neural network is constructed. This network contains multiple convolutional and pooling layers to extract feature maps of different scales from preprocessed images. Each preprocessed original image is processed using a feature extraction network to generate feature maps at multiple scales, each scale feature map corresponding to a different spatial resolution; Global average pooling is applied to the feature maps at each scale, and the pooled feature vectors are concatenated to form an initial image feature vector of fixed dimensions, which is then aggregated into an initial image feature vector set.

3. The multi-round AI image generation content consistency transformation method based on reinforcement learning according to claim 1, characterized in that, The step of constructing a reinforcement learning agent, which includes a policy network and a value function network, wherein the policy network is used to generate image transformation actions and the value function network is used to evaluate the long-term consistent reward of the actions, and the initial image feature vector is input into the policy network to output image transformation actions including rotation, scaling, brightness adjustment, and occlusion simulation, includes the following steps: Define an image transformation action space, which includes rotation angle parameters, scaling ratio parameters, brightness adjustment parameters, and occlusion area simulation parameters; A policy network is constructed, which adopts a multi-layer fully connected structure. The input is the initial image feature vector, and the output layer uses an activation function to output the specific values ​​of each parameter in the corresponding action space. A value function network is constructed, which adopts a multi-layer fully connected structure similar to the policy network. The input is the initial image feature vector, and the output is a single scalar value, which is used to evaluate the long-term consistent reward of the action performed in the current state. Each feature vector in the initial image feature vector set is sequentially input into the policy network, which then generates specific transformation action parameters corresponding to each original image.

4. The multi-round AI image generation content consistency transformation method based on reinforcement learning according to claim 1, characterized in that, The steps of performing multiple rounds of image transformation, in each round transforming the original image according to the action output by the policy network to generate a transformed image, and calculating the consistency reward value between the current transformation and the historical transformations based on the value function network, include: Initialize the transformation round counter, set the original image as the input image for the current round, and store the initial image feature vector and initial semantic mask as the benchmark for consistency comparison; For each round of transformation, the corresponding transformation operation is performed on the current input image according to the action parameters output by the policy network, including rotating by a specified angle, scaling by a ratio, adjusting the overall brightness value of the image, and adding an occlusion effect with a specific transparency in a specified area to generate the transformed image; Extract multi-scale feature maps from the transformed image and generate its feature vectors. At the same time, use a semantic segmentation model to generate a semantic mask for the transformed image. The consistency reward value for the current round is calculated by: calculating the structural similarity between the transformed image and the previous image in terms of pixel structure, calculating the perceptual similarity between the transformed image and the initial image in the feature space, and calculating the overlap similarity between the current semantic mask and the initial semantic mask in the region. The consistency reward value between the current transformation and the historical transformation is obtained based on the above three similarities.

5. The multi-round AI image generation content consistency transformation method based on reinforcement learning according to claim 1, characterized in that, The steps of updating the parameters of the policy network and value function network based on the reward values ​​collected during the multi-round transformation process, and optimizing the image transformation strategy to maintain the content consistency of the generated image in complex scenes include: Multiple sets of experience data are randomly sampled from the experience replay buffer. Each set of experience data includes a state feature vector, the parameters of the executed action, the reward value obtained, and the next state feature vector. For each set of sampled data, the value function network is used to calculate the current state value and the next state value, and the advantage function estimate is calculated to evaluate the superiority or inferiority of the current action relative to the average performance. Based on the advantage function estimate and the ratio of new to old action probabilities output by the policy network, the loss function of the policy network is calculated. This loss function includes a policy gradient term and a pruning term used to limit the update magnitude. The loss function of the value function network is calculated. This loss function is based on the difference between the state value predicted by the value function network and the actual cumulative reward value, and the mean squared error is used as the loss calculation standard. Simultaneously, the parameters of the policy network and value function network are optimized, and the ability of the actions generated by the policy network to maintain image content consistency in complex scenes is gradually improved through multiple iterations.

6. The multi-round AI image generation content consistency transformation method based on reinforcement learning according to claim 1, characterized in that, The step of outputting the final transformed image and its consistency evaluation report after reaching a preset number of rounds or reward convergence includes: The training process is monitored, and the training process ends when the number of rounds reaches the preset maximum number of rounds threshold or the average reward value of the most recent rounds changes less than the preset convergence threshold. Select the transformed image with the highest reward value from the last round of transformation results as the final output image, and record all transformation action parameters corresponding to that image. Generate a consistency evaluation report, which includes structural similarity indicators, perceptual similarity indicators, and semantic mask overlap indicators between the final image and the initial image, as well as reward value change curves and final stable values ​​during multiple rounds of training; The final transformed image and its consistency evaluation report are packaged and output as a reference standard for maintaining content consistency and a basis for configuring transformation parameters in subsequent image generation tasks.

7. A multi-round AI image generation content consistency transformation system based on reinforcement learning, characterized in that, include: The acquisition module is used to acquire an initial image dataset, which includes multiple original images and their corresponding semantic annotation information. An image feature extraction network is constructed to extract multi-scale feature maps of each original image based on a convolutional neural network structure, and an initial image feature vector set is generated. A construction module is used to construct a reinforcement learning agent, which includes a policy network and a value function network. The policy network is used to generate image transformation actions, and the value function network is used to evaluate the long-term consistent reward of the actions. The initial image feature vector is input into the policy network, and the output includes image transformation actions such as rotation, scaling, brightness adjustment, and occlusion simulation. The execution module is used to perform multiple rounds of image transformation. In each round, the original image is transformed according to the action output by the policy network to generate the transformed image, and the consistency reward value between the current transformation and the historical transformation is calculated based on the value function network. The generation module is used to update the parameters of the policy network and value function network based on the reward values ​​collected during multiple rounds of transformation, optimize the image transformation strategy, and maintain the content consistency of the generated image in complex scenes. The output module is used to output the final transformed image and its consistency evaluation report after reaching a preset number of rounds or reward convergence.

8. The multi-round AI image generation content consistency transformation system based on reinforcement learning according to claim 7, characterized in that, The output module includes: The monitoring unit is used to monitor the training process. When the number of transformation rounds reaches the preset maximum number of rounds threshold or the average reward value change of the most recent rounds is less than the preset convergence threshold, the training process is determined to end. The selection unit is used to select the transformed image with the highest reward value from the last round of transformation results as the final output image, and at the same time record all transformation action parameters corresponding to the image. A generation unit is used to generate a consistency evaluation report, which includes structural similarity indicators, perceptual similarity indicators, and semantic mask overlap indicators between the final image and the initial image, as well as reward value change curves and final stable values ​​during multiple rounds of training. The output unit is used to package and output the final transformed image and its consistency evaluation report, which serves as a reference standard for maintaining content consistency and a basis for configuring transformation parameters in subsequent image generation tasks.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images