A combined reward feedback training image rendering method for remote sensing target detection
By combining reward feedback to train image rendering methods, the problems of data scarcity and high annotation costs in remote sensing target detection are solved, and high-quality, controllable synthetic data generation is achieved, which improves the training effect and cross-regional adaptability of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-04-17
- Publication Date
- 2026-06-02
AI Technical Summary
In remote sensing target detection, real data is scarce and annotation costs are high. Furthermore, existing rendering and synthesis methods cannot guarantee the diversity, controllability, and consistency of data, which affects the performance of models when deployed across regions.
A combined reward feedback training image rendering method is adopted. By constructing a background pool, round-fixed sampling, controllable rendering synthesis and annotation generation, combined with detection feedback indicators and perceptual consistency indicators, a combined reward is constructed and the strategy is trained. The rendering control parameters are automatically updated to ensure the quality and consistency of the synthesized image.
It improves the quality and controllability of synthetic data, enhances the training effectiveness of downstream target detection models, reduces engineering implementation costs, and strengthens the adaptability of models under different backgrounds and conditions.
Smart Images
Figure CN122135144A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a combined reward feedback training image rendering method for remote sensing target detection, belonging to the fields of remote sensing image processing, computer vision, image rendering, and intelligent training. Background Technology
[0002] Remote sensing imaging can continuously acquire surface information over a large spatial scale, featuring wide coverage, short revisit cycles, and consistent observation across regions. It has been widely applied in fields such as land resource surveys, transportation and urban management, disaster monitoring and assessment, and marine and border surveillance. With the continuous improvement of optical remote sensing sensor resolution and enhanced multi-satellite collaboration and high-frequency revisit capabilities, the demand for automatic detection and identification of typical targets such as vehicles, aircraft, and ships continues to grow. Compared with natural scene images, remote sensing images exhibit significant differences in imaging geometry, lighting conditions, scale distribution, and background texture. The same type of target shows obvious appearance variations in different regions, seasons, and observation conditions. Targets often appear at small scales and overlap with complex backgrounds. These factors collectively lead to generalization fluctuations in remote sensing target detection models in real-world applications, especially when deployed across regions and migrated over time, where model performance is more significantly affected by differences in data distribution.
[0003] From a data supply perspective, the training effect of remote sensing target detection highly depends on high-quality labeled data. However, high-resolution remote sensing images are large in area, and targets account for a small proportion of the entire image. Annotators need to search and accurately locate targets over a large area, resulting in high labeling costs and low efficiency. At the same time, target categories and background types in real scenes often exhibit a long-tail distribution, and a small number of categories cannot form sufficient coverage, leading to a prominent class imbalance problem. Furthermore, remote sensing images are affected by factors such as solar altitude and azimuth angles, surface materials and reflectivity, shadow occlusion, imaging noise, and compression resampling. The appearance statistical differences of the same type of target under different imaging conditions are significant, and limited real data cannot cover diverse distributions, resulting in insufficient representativeness of the training set. In addition, some remote sensing data is limited in scale due to restrictions on acquisition channels, usage rights, and security compliance, further exacerbating the problems of data scarcity and insufficient distribution coverage. Therefore, how to expand training samples, improve data diversity, and reduce domain shift risk while ensuring labeling consistency and data controllability has become a key issue of long-term concern in the field of remote sensing target detection.
[0004] To alleviate the problem of data scarcity, common approaches include data augmentation, generative synthesis, and rendering-based controlled synthesis. Traditional data augmentation typically expands samples through geometric transformations, color perturbations, and noise injection. However, these methods are essentially local transformations of existing samples, making it difficult to systematically change the combination relationship between the target and the background, and also failing to fully cover the variations in imaging conditions in real-world scenes. Generative methods can generate diverse appearances to a certain extent, but in remote sensing scenarios with small targets, it is difficult to maintain stable target edge details and geometric consistency. Furthermore, the generated results often fail to naturally align with strictly consistent target annotations, leading to insufficient controllability and annotation reliability in the engineering application of synthesized data. In contrast, rendering-based controlled synthesis can construct the combination process of foreground targets and background scenes with explicit geometric and lighting mechanisms. It can also automatically output annotation information that is strictly consistent with the synthesized results while generating images, making it more suitable for building scalable training data and providing stable supervision signals for downstream detection models.
[0005] However, the practical application of rendering synthesis in remote sensing scenarios still faces significant challenges. The visual appearance of remote sensing images is determined not only by the texture and shape of the target, but also by multiple factors, including illumination direction and intensity, shadow projection relationships, surface reflection and material differences, background texture statistics, sensor imaging processes, and post-processing. Rendering parameters are typically multi-dimensional and strongly coupled, containing both discrete and continuous variables. For example, shadow direction is a typical discrete choice, while brightness, contrast, shadow intensity, and blur intensity are closer to continuous adjustment. Different background regions exhibit significant differences in surface type, texture scale, and color statistics. Changes in the sun's angle at different times can alter shadow direction and length, making it difficult to maintain consistent results across all background sets using the same set of rendering parameters. If the parameters are set improperly, the synthesized results may deviate from the real image in terms of illumination consistency, shadow consistency, and color texture statistics, introducing a significant domain shift and weakening the effectiveness of the synthesized data for training detection models.
[0006] In existing workflows, rendering parameters often rely on manual experience to set or random sampling to generate. Manual parameter tuning typically requires repeated trials and depends on the operator's long-term experience, making it difficult to quickly transfer between large-scale background sets, different target types, and different imaging conditions. Furthermore, the efficiency and stability of parameter tuning are significantly affected by subjective factors. Random sampling or domain randomization can improve diversity to some extent, but the range and distribution of random perturbations are difficult to maintain consistency with the actual remote sensing imaging process. Especially in small target detection tasks, excessive random perturbations may destroy discernible details of the target and introduce artifacts that do not conform to imaging rules. This results in an increase in the quantity of synthetic data, but limited improvement to the detection model, or even a decrease in training gains. Meanwhile, some solutions prioritize visual realism, neglecting the training requirements of downstream detection tasks. This makes the synthesized images appear more natural subjectively, but does not significantly improve the detector's discrimination ability. Other solutions focus more on short-term improvements in detection metrics, but lack constraints on the consistency between the synthetic and real distributions, which may bias the synthetic results and reduce their universality across models and scenes.
[0007] Furthermore, in large-scale data production scenarios, it is also necessary to balance engineering usability and quality consistency. Remote sensing synthesis typically involves repeatedly placing targets and generating annotations on a large background. This requires the synthesis process to have clear constraints and quality control mechanisms. For example, target placement should avoid exceeding boundaries, unreasonable overlap between targets should be reduced, and the shadow direction and background lighting relationship should be kept as consistent as possible to reduce obvious artifacts and the proportion of anomalous samples. Without stable quality constraints and traceable data management mechanisms, the synthesized data is prone to distribution drift between different batches, which in turn affects the stability and reproducibility of downstream training. Therefore, how to form a controllable, reproducible, and scalable parameter management and data generation chain in the rendering synthesis process is also an important problem that existing technologies need to solve.
[0008] To obtain better rendering parameter configurations, some methods attempt to employ black-box optimization strategies such as grid search, random search, or Bayesian optimization, iteratively evaluating and selecting parameters in the parameter space. While these methods can automate the search for better parameters to some extent, they typically require a large number of evaluations, each involving rendering composition and model inference, resulting in high computational costs and difficulty adapting to large-scale background sets and long-term training scenarios. Furthermore, basing evaluations on a finite subset of backgrounds can easily lead to overfitting to specific subsets, causing parameters to perform inconsistently across a wider range of backgrounds. Meanwhile, the rendering pipeline often includes non-differentiable or difficult-to-differentiate-end-to-end steps, and detection metrics and distribution consistency metrics often exhibit non-convex and noisy characteristics, making analytical gradient-based optimization methods difficult to apply directly. Therefore, a learning-based closed-loop method is needed that can achieve iterative optimization under non-differentiable rendering and complex evaluation metrics, freeing parameter updates from reliance on manual experience or inefficient exhaustive searches.
[0009] Against this backdrop, incorporating combined reward feedback into the training image rendering process for remote sensing target detection has become a promising technical approach. This approach abstracts the image rendering process into a feedback-based parameter control and update process, using both the effectiveness of the target detection task and the consistency of image perception as feedback signals, thereby achieving adaptive adjustment of shadow direction and multi-dimensional rendering control parameters. The learned rendering control strategy can not only be continuously optimized during training but also reused under different background sets and target material conditions, thus improving the automation level and stability of training image rendering and synthetic data generation.
[0010] In summary, remote sensing target detection has long faced challenges such as scarcity of real data, high annotation costs, and insufficient distribution coverage. Rendering synthesis provides an effective way to construct controllable and scalable training data. However, manual setting and random sampling of high-dimensional rendering control parameters make it difficult to stably balance realism and task effectiveness. Traditional search-based optimization suffers from limitations such as high computational cost and insufficient scalability. Therefore, a combined reward feedback training image rendering method for remote sensing target detection is needed. This method should achieve adaptive updates of shadow direction and rendering control parameters through combined reward feedback during image rendering and training, while maintaining quality consistency and reproducibility under large-scale data production conditions. This would improve the ability of synthetic data to support downstream detection tasks and reduce engineering implementation costs. Summary of the Invention
[0011] 1. Objective: The objective of this invention is to provide a combined reward feedback training image rendering method for remote sensing target detection. Addressing the difficulties in acquiring real data and the high cost of annotation in remote sensing target detection training, this method introduces a combined reward feedback training mechanism into the controllable rendering synthesis process. This allows shadow direction and multi-dimensional rendering control parameters to be automatically updated during the iterative process of synthesis and evaluation. While ensuring the reasonableness of the visual distribution of the synthesized image, it also improves the training effectiveness for downstream target detection tasks, thereby achieving high-quality, controllable, and large-scale generation of synthesized data.
[0012] 2. Technical Solution: This invention is achieved through the following technical solution: This invention is a combined reward feedback training image rendering method for remote sensing target detection, comprising seven steps: training data preparation, background pool construction and round-fixed sampling, controllable rendering synthesis and annotation generation, detection feedback index calculation, perception consistency index calculation, combined reward construction, policy training, and rendering control update. The overall process of this method is as follows: Figure 1 As shown, the specific steps are as follows:
[0013] Step 1: Training Data Preparation
[0014] A basic image library is acquired, comprising at least a background image set and a foreground target image library. The background image set provides background textures and feature distributions for the remote sensing scene, while the foreground target image library provides target instances to be synthesized. The foreground target image library includes target images with alpha channels and corresponding shadow images. Further, the background image set and foreground target image library undergo unified formatting and quality screening to meet preset resolution, band type, and pixel value range requirements. Simultaneously, a target detection model is acquired for calculating the detection feedback index. This model performs target detection inference on the synthesized image and outputs predicted bounding boxes, categories, and confidence information. A real image reference set and a feature extraction network are also acquired for calculating the perceptual consistency index. The real image reference set represents the distribution of real remote sensing images corresponding to the target task, and the feature extraction network extracts feature representations from the real and synthesized images to support the subsequent calculation of the perceptual consistency index. Furthermore, establish a configuration file and parameter recording mechanism for training process management, including at least parameters such as the initialization range of rendering parameters, background pool size, fixed sampling number per round, reward fusion weight, and training rounds, to ensure that the training process is traceable and reproducible.
[0015] Step 2: Background Pool Construction and Round-Based Fixed Sampling
[0016] A background pool is constructed based on the background image set obtained in step one, and the background pool is hierarchically managed according to preset rules to represent the differences in different scene types or texture statistical features, thereby ensuring the coverage and balance of the background distribution during training. Specifically, each background image in the background pool is converted into a grayscale image. Calculate its average brightness The calculation method for the brightness feature is as follows:
[0017]
[0018] The image width is Gao Wei , and These represent the coordinates of pixels in the image, corresponding to their positions in the horizontal and vertical directions, respectively. Further calculations are then performed on the pixel gradient magnitude. As an edge feature, the calculation method is as follows:
[0019]
[0020] And based on the preset threshold Obtain edge binary indicator value As shown in the following formula
[0021]
[0022] Calculate the edge density accordingly. As shown in the following formula:
[0023]
[0024] The average brightness With the edge density Brightness grouping identifiers and edge grouping identifiers are generated by discretizing according to preset threshold intervals, and these two are combined to form a hierarchical grouping identifier, thereby completing the background pool stratification. At the beginning of each training round, a fixed background subset is sampled from each hierarchical group according to a preset ratio. The number of the fixed background subset is a preset value and remains unchanged within the same round. The controllable rendering and synthesis performed within the round selects background images from the fixed background subset to reduce the evaluation fluctuations introduced by random background switching and stabilize the reward calculation. At the same time, the background pool is partially refreshed according to a preset refresh cycle and refresh ratio, that is, some samples in the background pool are replaced with new backgrounds from the background image set, and the hierarchical grouping identifier is recalculated for the replaced backgrounds, so that the background coverage gradually expands and the evaluation distribution remains relatively stable during the training process.
[0025] Step 3: Controlled rendering compositing and annotation generation
[0026] Based on the current parameter vector, target rendering and synthesis are performed on the fixed background subset to obtain a synthesized image. Simultaneously, target annotation information corresponding to the synthesized image is generated. The training image rendering stage process for remote sensing target detection is as follows: Figure 2 As shown; wherein, the target rendering synthesis includes discrete selection of shadow direction and continuous control of rendering parameters. The shadow direction is represented by multi-category discrete variables, and the rendering parameters are used to control the superposition effect of the target and shadow in the background. Specifically, in each round, a background image is selected from the fixed background subset, and a target image with an alpha channel and a target shadow are selected from the foreground target material library. The position and appearance of the target and shadow are controlled according to the current parameter vector, and superposition is performed to generate a synthesized image. The annotation information includes the bounding box information and category information of the target. Before generating the annotation information, the target placement result is checked for boundary crossing and overlap. The boundary crossing check is used to limit the target bounding box to be within the effective area of the background, and the overlap check is used to limit unreasonable occlusion or excessive overlap between multiple targets in the same background. When the check passes, the synthesized image and its annotation information are output. When the check fails, the target is repositioned or adjusted until the preset constraints are met, thereby achieving consistent generation of the synthesized image and annotation information, providing input for subsequent detection feedback index calculation.
[0027] Step 4: Calculation of Detection Feedback Indicators
[0028] The synthesized image generated in step three is input into the target detection model for inference to obtain the corresponding detection prediction result. The detection prediction result includes at least the predicted bounding box, predicted category, and confidence information. The detection prediction result is then matched with the target annotation information generated in step three, based on the predicted bounding box. With annotation bounding box The intersection-union ratio (CIU) determines the matching relationship. The CIU is shown in the following formula:
[0029]
[0030] When the crossover ratio (CUP) is not less than a preset threshold, the corresponding prediction is recorded as a true positive. Otherwise, it will be recorded as a false positive. And unmatched labeled targets are marked as false negatives. Statistical precision at different confidence thresholds With recall rate :
[0031]
[0032]
[0033] The average precision is calculated based on the relationship between precision and recall. As a detection feedback indicator:
[0034]
[0035] in, and The precision and recall values are obtained by sorting according to the confidence threshold; the detection feedback index is used to characterize the effectiveness of the training image for the target detection task under the current rendering control parameter vector, and to provide task-level quantitative feedback for the subsequent combined reward construction.
[0036] Step 5: Calculation of Perceived Consistency Index
[0037] A perceptual consistency index is calculated based on the real image reference set and the synthetic image set generated in step three. This perceptual consistency index characterizes the similarity between the distribution of the synthetic images and the distribution of the real remote sensing images in the feature space. Specifically, the real image reference set and the synthetic images are respectively input into a feature extraction network to obtain feature vector sets. The mean vector and covariance matrix of the features in the real images and the synthetic images are calculated respectively to obtain... , as well as , The FID index is calculated based on the following Frechet distance:
[0038]
[0039] in, This represents the matrix trace operation; furthermore, to improve training stability, the perceptual consistency index can be normalized or a truncation upper bound can be set to avoid outliers having too much impact on subsequent reward construction.
[0040] Step Six: Building Combined Rewards
[0041] The detection feedback index obtained in step four and the perceptual consistency index obtained in step five are fused to construct a combined reward value for image rendering training. Specifically, the detection feedback index is used as the positive reward term, and the perceptual consistency index is used as the distribution constraint term. These are then weighted and fused according to preset weights to obtain the reward value, as shown in the following formula:
[0042]
[0043] in, and For weighting coefficients, the The average precision AP calculated in step four, wherein The Frechet distance (FID) is calculated in step five. Further, the reward value is normalized or truncated to ensure it falls within a preset range, thereby limiting the reward scale and reducing the impact of extreme values on policy updates. This improves the stability and convergence of the image rendering training process based on combined reward feedback.
[0044] Step 7: Strategy Training and Rendering Control Update
[0045] The image rendering control process is modeled as a policy training process based on combined reward feedback. The current rendering parameter vector and related state information are used as input, and the policy network outputs multi-discrete parameter adjustment actions. These multi-discrete parameter adjustment actions include at least the selection of the shadow direction category and discrete incremental instructions for each rendering control parameter. The incremental instructions for each rendering control parameter take values from the set {−1, 0, +1}, representing decrease, maintenance, and increase, respectively. The corresponding parameters are updated according to a preset step size, and the updated parameters are cropped to a preset value range. The update of the i-th rendering control parameter is shown in the following formula:
[0046]
[0047] in, For the first The discrete adjustment actions corresponding to each parameter. Step size, For the range of parameter values, For the pruning operator, steps three through six are repeated on the updated rendering parameter vector to obtain reward feedback, and the policy network is updated using a policy optimization algorithm; the policy optimization algorithm is a near-end policy optimization algorithm, and the following constraint mechanism is introduced during the policy update process, all of which must satisfy: restricting the target policy Compared to the old strategy The divergence makes
[0048] in, Indicates parameters Parameterized target policy in state The probability distribution of the following actions. This indicates the old policy used for constraints in the state. The probability distribution of the following actions. This represents the preset divergence threshold. An entropy regularization term is introduced into the objective function, making...
[0049]
[0050] in, Let represent the objective function for policy optimization with constraints. This represents the policy loss term constructed based on the sampled data. This represents the expectation operation; Represents entropy operation. This represents the weight coefficient of the entropy regularization term. The update magnitude of the value function is pruned to ensure...
[0051]
[0052] in, Indicates parameters Parameterized value function for state The value estimate, Represents the old value function with respect to the state The value estimate, This indicates that the input is clipped to a range. The clipping operator inside, This represents the pruning threshold of the value function. It also prunes the gradient norm of the parameter updates, making...
[0053]
[0054] in, This represents the gradient vector of the objective function with respect to the parameters. This represents the preset gradient clipping threshold. The process iterates until a termination condition is met, outputting an optimized rendering parameter vector or rendering parameter control strategy for subsequent rendering of remote sensing target detection training images and generation of synthetic data.
[0055] 3. Advantages and effects:
[0056] Compared with existing rendering synthesis methods that rely on manual experience or random sampling, this invention has at least the following advantages and effects: (1) By modeling the image rendering control update as a strategy training process based on combined reward feedback, the shadow direction and rendering control parameters can be automatically updated in the closed loop of synthesis and evaluation, reducing manual repeated parameter tuning and thus improving the efficiency of synthesis data production; (2) By constructing a background pool and layering it according to brightness features and edge features, and then organizing the background input in a round-fixed sampling manner, the evaluation fluctuation can be reduced while ensuring the coverage of background distribution, thereby improving the stability of the training process; (3) By constructing a combined reward by fusing detection feedback indicators and perceptual consistency indicators, it can simultaneously (3) By taking into account both the effectiveness of downstream detection and the rationality of the synthetic distribution, the actual gain of synthetic data on the training of the target detection model is improved; (4) By adjusting the action of multiple discrete parameters and the parameter range pruning mechanism, the parameter update amplitude can be controlled and parameter oscillation can be avoided, thereby improving the convergence of policy learning; (5) By introducing the divergence constraint, entropy regularization, value function pruning and gradient pruning mechanism of near-end policy optimization, the policy update amplitude can be limited and training divergence can be suppressed, thereby improving the robustness of the optimization process; (6) By generating annotation information synchronously during the synthesis process and performing boundary checks and overlap checks, the consistency between the synthetic image and the annotation can be guaranteed and abnormal samples can be reduced, thereby improving the quality and usability of the dataset. Attached Figure Description
[0057] Figure 1 This is a schematic diagram of the overall process proposed in this invention. Figure 2 This is a schematic diagram of the training image rendering stage for remote sensing target detection. Figure 3 This is a schematic diagram of the result generated by the present invention. Detailed Implementation
[0058] To better understand the technical solution of this invention, the embodiments of this invention are further described below with reference to the accompanying drawings: This invention is based on the PyTorch framework and implemented using the Python language. Specifically, this invention first completes the preparation of training data and the construction of the background pool, and performs rendering-based remote sensing image synthesis and annotation generation under a round-based fixed sampling mechanism; then, the synthesized image is input into the target detection model to calculate the detection feedback index, and the perceptual consistency index is calculated by combining the real image reference set and the feature extraction network. Based on this, a combined reward is constructed and a proximal policy optimization algorithm is used to train and update the policy network, thereby realizing the automatic optimization of shadow direction and rendering control parameters, and obtaining an optimized parameter vector or rendering control strategy for rendering training images for remote sensing target detection; finally, the optimized parameter vector or rendering control strategy is used to generate synthetic data and its annotation information, and the generated result is as follows. Figure 3 As shown.
[0059] The server hardware environment consists of a host equipped with an Intel(R) Xeon(R) Platinum 8336C CPU@2.30GHz and eight NVIDIA GeForce RTX 4090 graphics cards, each with 24.0GB of video memory. The operating system runs on the server and the runtime environment required for deep learning training and inference is installed to complete the rendering and compositing, metric calculation and reinforcement learning training processes.
[0060] The specific steps of this method are as follows:
[0061] Step 1: Training Data Preparation
[0062] Based on the training data feature requirements for controllable rendering synthesis, detection feedback index calculation, and perception consistency index calculation, this invention constructs a real remote sensing image dataset, a synthetic rendering background image set, and a foreground target material library, and simultaneously prepares a target detection model, a real image reference set, and a feature extraction network. Specifically, the real remote sensing image dataset is based on two publicly available remote sensing image datasets for target detection tasks: DOTA and SODA-A. The DOTA dataset uses version DOTA-v1.0, containing 15 common categories, 2806 images, and 188282 labeled targets, with the original labels in rotated quadrilateral form. The SODA-A dataset contains 9 target categories and exhibits typical ultra-dense small target distribution characteristics, which can be used to supplement the real distribution of targets such as vehicles and aircraft in dense scenes. To improve sample utilization efficiency and adapt to subsequent training input, this embodiment first uses a sliding window method to crop the original images with high resolution and large size in both datasets into 512×512 pixel slices, and removes invalid areas or areas with very few targets. Then, based on the original annotation files, samples such as vehicles and aircraft related to the research target are selected, and the annotations are uniformly transformed, normalizing the rotated quadrilateral or rotated box annotations into center point coordinates and length and width parameters to ensure consistency in annotation format and numerical scale across different data sources. Finally, the cropped, filtered, and annotated subsets of DOTA and SODA-A are merged to construct a unified real remote sensing image dataset. This dataset serves as both a real image reference set for calculating the perceptual consistency index and provides background source and target scale statistical reference for subsequent synthesis stages.
[0063] In terms of data preparation for synthesis, this embodiment uses slices cropped and filtered from the real dataset as the background atlas for the synthesized remote sensing image to ensure the diversity of the background environment. Scale sampling is based on the target scale statistics from the real dataset to constrain the scale distribution of target placement during synthesis. Simultaneously, to construct a foreground target material library, this embodiment selects ShapeNet's 3D model resources as the target geometry source, choosing vehicle and aircraft models. The 3D models are then rendered offline under a unified coordinate system and imaging configuration to generate target images with alpha channels. Simultaneously, shadow materials consistent with the targets are generated, thus forming a foreground target material library with a one-to-one correspondence between "target image and target shadow".
[0064] Furthermore, this embodiment obtains the target detection model YOLOv11 for calculating the detection feedback index, which is used to infer the synthetic image and output predicted bounding boxes, predicted categories, and confidence information; and prepares the feature extraction network InceptionV3 to extract feature representations from the real image reference set and the synthetic image to support the subsequent calculation of the perceptual consistency index. The above training data preparation process provides the input foundation for subsequent steps.
[0065] Step 2: Background Pool Construction and Round-Based Fixed Sampling
[0066] A set of background images is obtained, and a background pool is constructed from the set of background images. The background pool is layered according to a preset layering rule, and a fixed background subset for the current round is obtained by sampling proportionally from each layer, wherein the fixed background subset remains unchanged within the same round. To achieve layered management, this embodiment generates a background metadata record for each background image, and the background metadata record includes at least brightness features and edge features. Brightness features are obtained by converting the background image to a grayscale image. Calculate its average brightness The average brightness is calculated as follows:
[0067]
[0068] in, Image width, Image height. Edge features are calculated by measuring the gradient magnitude at each pixel. The gradient magnitude is calculated as follows:
[0069]
[0070] Then edge binarization is performed to obtain edge indicator values. As shown in the following formula
[0071]
[0072] in The preset threshold is used. Finally, the edge density is calculated. As shown in the following formula:
[0073]
[0074] The average brightness and edge density are discretized according to a preset threshold range to obtain brightness group identifiers and edge group identifiers, respectively. These two identifiers are combined to generate a hierarchical group identifier, thereby dividing the background image into multiple hierarchical groups to form a background pool. In each round, background images are extracted from the background pool according to a preset sampling ratio for each hierarchical group to form a fixed background subset for the current round. The background pool is partially refreshed according to a preset refresh cycle and a preset refresh ratio. In this embodiment, partial refresh includes removing a portion of the background images from the background pool according to the refresh ratio after a preset number of rounds, and replacing them with an equal number of background images from the background image set. The replacement process can be performed separately within each hierarchical group, or the replacement can be constrained according to the hierarchical group identifier to ensure that the replaced background pool still meets the above hierarchical grouping rules.
[0075] Step 3: Controlled rendering compositing and annotation generation
[0076] Based on the current parameter vector, target rendering and synthesis are performed on the fixed background subset to obtain a synthesized image, and target annotation information corresponding to the synthesized image is generated simultaneously. The target rendering and synthesis includes discrete selection of shadow direction and continuous control of rendering parameters, where shadow direction is represented by multi-category discrete variables. Specifically, in each round, a background image is selected from the fixed background subset, and a target image with an alpha channel and target shadow are selected from the foreground target material library. The target is superimposed on the background according to the position, scale, and pose determined by the current parameter vector, and the target shadow is projected and superimposed in a consistent direction according to the shadow direction. At the same time, the rendering parameters are continuously controlled to generate a synthesized image. The annotation information includes the target's bounding box information and category information. Before generating the annotation information, boundary check and overlap check are performed on the target placement result. Boundary check is used to ensure that the target bounding box is within the effective area of the background, and overlap check is used to avoid unreasonable occlusion or excessive overlap between multiple targets in the same background. When the check passes, the synthesized image and its annotation information are output. When the check fails, the target is repositioned or adjusted until the preset constraints are met, thereby ensuring the consistency and usability of the synthesized image and the annotation information.
[0077] Step 4: Calculation of Detection Feedback Indicators
[0078] The synthesized image is input into the target detection model to obtain a prediction result, and a detection feedback index is calculated based on the prediction result and the target annotation information. In this embodiment, the detection feedback index is the average precision (AP). First, based on the predicted bounding box... With annotation bounding box The intersection-union ratio (CIU) determines the matching relationship. The CIU is shown in the following formula:
[0079]
[0080] A true positive result is determined when the cross-union ratio is greater than or equal to a preset threshold. Otherwise, it will be recorded as a false positive. And unmatched labeled targets are marked as false negatives. Statistical precision at different confidence thresholds With recall rate :
[0081]
[0082]
[0083] The mean precision (AP) is calculated based on the precision-recall curve, and the mean precision is shown in the following formula:
[0084]
[0085] in, and The precision and recall values are obtained by sorting according to the confidence threshold; the detection feedback index is used to characterize the effectiveness of the training image for the target detection task under the current rendering control parameter vector, and to provide task-level quantitative feedback for the subsequent combined reward construction.
[0086] Step 5: Calculation of Perceived Consistency Index
[0087] A perceptual consistency index is calculated based on the set of real background images and the synthesized images. In this embodiment, the perceptual consistency index adopts the Frechet Inception Distance index based on feature distribution distance. The set of real background images and the set of synthesized images are respectively input into a pre-trained InceptionV3 feature extraction network, and the feature vector of each image is extracted at the output of the average pooling layer of the feature extraction network. The mean vector and covariance matrix are calculated for the feature vectors of the real background image set and the feature vectors of the synthesized image set, respectively. , as well as , The FID index is calculated based on the following Frechet distance:
[0088]
[0089] in, This represents the matrix trace operation; furthermore, to improve training stability, the perceptual consistency index can be normalized or a truncation upper bound can be set to avoid outliers having too much impact on subsequent reward construction.
[0090] Step Six: Building Combined Rewards
[0091] The detection feedback index and the perception consistency index are fused to obtain a combined reward value, which is then normalized or truncated. In this embodiment, the detection feedback index is used as the positive reward term, and the perception consistency index is used as the distribution constraint term. They are fused according to a preset weight, and the reward value satisfies the following formula:
[0092]
[0093] in, To serve as the detection feedback indicator, the average precision AP obtained in step four is selected. As a perceptual consistency metric, the Frechet Inception Distance metric obtained in step five is selected. and This is the weighting coefficient. Furthermore, the reward value is normalized or truncated to ensure it falls within a preset range, thereby limiting the reward scale and improving the stability of subsequent policy updates.
[0094] Step 7: Strategy Training and Rendering Control Update
[0095] A strategy optimization algorithm is used to output multi-discrete parameter adjustment actions, and the parameter vector is updated according to these actions. Steps three to seven are repeated to obtain an optimized parameter vector or rendering control strategy for rendering training images for remote sensing target detection. In this embodiment, the multi-discrete parameter adjustment actions take values from the set {−1, 0, +1} for each parameter to be optimized, and the j-th parameter is updated according to the following rule:
[0096]
[0097] in, For the first The discrete adjustment actions corresponding to each parameter. Step size, The parameter value range is defined. Steps three to six are repeated based on the updated parameter vector to obtain reward feedback, and the policy network is updated using the near-end policy optimization algorithm. The following constraint mechanism is introduced during the policy update process, and all of the constraint mechanisms must satisfy: (1) Restrict the target policy Compared to the old strategy The divergence makes
[0098]
[0099] in, Indicates parameters Parameterized target policy in state The probability distribution of the following actions. This indicates the old policy used for constraints in the state. The probability distribution of the following actions. This indicates the preset divergence threshold.
[0100] (2) Introduce an entropy regularization term into the objective function, such that
[0101]
[0102] in, Let represent the objective function for policy optimization with constraints. This represents the policy loss term constructed based on the sampled data. This represents the expectation operation; Represents entropy operation. This represents the weight coefficient of the entropy regularization term.
[0103] (3) Prune the update magnitude of the value function so that
[0104]
[0105] in, Indicates parameters Parameterized value function for state The value estimate, Represents the old value function with respect to the state The value estimate, This indicates that the input is clipped to a range. The clipping operator inside, This represents the pruning threshold of the value function.
[0106] (4) Prune the gradient norm of the parameter update so that
[0107]
[0108] in, This represents the gradient vector of the objective function with respect to the parameters. This represents the preset gradient clipping threshold. The process iterates until a termination condition is met, outputting an optimized parameter vector or parameter control strategy for subsequent rendering of remote sensing target detection training images and generation of synthetic data. Furthermore, during training, the reward value, detection feedback index, and perceptual consistency index for each round or training epoch are recorded, and the synthetic images and annotation information are stored in the corresponding directory according to the training epoch to ensure traceability of the training process.
[0109] Experimental results: Figure 3 The diagram illustrates the generation results of this invention, where the left side shows a sample of the synthesized remote sensing image generated by this invention, and the right side shows a labeled visualization result corresponding one-to-one with the synthesized image on the left. The labels are generated synchronously during the synthesis process in step three, including target category information and target bounding box information. The labeled image is used to visually demonstrate the consistency and accuracy between the synthesized image and the labeled information, indicating that this invention can automatically output labels that strictly correspond to the synthesized result while completing controllable rendering synthesis, thereby providing directly usable synthesized samples and supervision information for subsequent target detection model training.
Claims
1. A combined reward feedback training image rendering method for remote sensing target detection, characterized in that, Includes the following steps: Step 1: Training data preparation; The system acquires a basic image library for image rendering and compositing, as well as models and reference data for evaluating the compositing quality. The basic image library includes at least a background image set and a foreground target image library. The background image set is used to provide the compositing base image, and the foreground target image library is used to provide target images with alpha channels and target shadows. The system also acquires a target detection model for calculating detection feedback metrics, and a real image reference set and feature extraction network for calculating perceptual consistency metrics. Step 2: Background pool construction and round-based fixed sampling; Obtain a set of background images, construct a background pool from the set of background images; divide the background pool into layers according to a preset layering rule, and sample a fixed background subset for the current round from each layer according to a certain ratio, wherein the fixed background subset remains unchanged within the same round; and partially refresh the background pool according to a preset refresh cycle and refresh ratio. Step 3: Controlled rendering compositing and annotation generation; Based on the current rendering control parameter vector, target rendering is synthesized on the fixed background subset to obtain a synthesized image for training the remote sensing target detection model, and target annotation information corresponding to the synthesized image is generated simultaneously. Step 4: Calculation of detection feedback indicators; The synthesized image is input into the target detection model to obtain the prediction result, and the detection feedback index is calculated based on the prediction result and the target annotation information; Step 5: Calculate the perceived consistency index; Calculate the perceptual consistency index based on the real image reference set and the synthetic image; Step Six: Constructing Combined Rewards; The detection feedback index and the perception consistency index are fused to obtain a combined reward value, and the combined reward value is normalized or truncated. Step 7: Strategy training and rendering control update; A strategy optimization algorithm is used to output multi-discrete parameter adjustment actions, and the rendering control parameter vector is updated according to the multi-discrete parameter adjustment actions. Steps three to six are repeated based on the updated rendering control parameter vector, and strategy training continues until a preset stopping condition is met, thereby obtaining an optimized rendering control parameter vector or rendering control strategy for rendering training images for remote sensing target detection.
2. The method according to claim 1, characterized in that: In step two, the layering rule includes the following processing: obtaining a background metadata record for each background image, wherein the background metadata record includes at least the brightness features and edge features of the background image, and the brightness features are calculated as follows: in It is a grayscale image with a width of [missing information]. Gao Wei The average brightness is The edge features are calculated as follows: in For pixels The gradient magnitude at a given point is used to represent the edge strength at that pixel. Edge binarization is then performed. in Represents pixels The edge indicator value is set to 1 to represent an "edge pixel" and 0 to represent a "non-edge pixel". Finally, the edge density is calculated. in Edge density represents the proportion of edge pixels to total pixels in an image. The average brightness... With the edge density Discretize the background image according to a preset threshold range to obtain brightness group identifiers and edge group identifiers. The brightness group identifier represents the group containing the brightness level of the background image, and the edge group identifier represents the group containing the edge complexity of the background image. Combine the brightness group identifiers and the edge group identifiers to generate a hierarchical group identifier. Divide the background image into multiple hierarchical groups based on the hierarchical group identifiers to form the background pool. In each round, extract background images from the background pool according to a preset sampling ratio for each hierarchical group to form a fixed background subset for the current round. Partial refresh of the background pool includes: after a preset number of rounds, removing a portion of the background images from the background pool according to a preset refresh ratio, and supplementing an equal number of background images from the background image set to complete the replacement. The replacement process is performed separately within each hierarchical group or constrained according to the hierarchical group identifier during supplementation, ensuring that the replaced background pool still satisfies the hierarchical grouping rules.
3. The method according to claim 1, characterized in that: In step three, the target rendering composition includes discrete selection of shadow direction and continuous control of rendering control parameters, wherein the shadow direction is represented by multi-category discrete variables.
4. The method according to claim 1, characterized in that: In step three, the annotation information includes the bounding box information and category information of the target. Before generating the annotation information, boundary crossing and overlap checks are performed on the target placement results. When the checks pass, the synthesized image and its annotation information are output.
5. The method according to claim 1, characterized in that: In step four, the detection feedback metric is the average accuracy (AP), which is calculated from the intersection-union ratio (IU) matching relationship between the predicted bounding box and the labeled bounding box; wherein the IU satisfies: when When the value is greater than or equal to a preset threshold, the predicted bounding box is determined. With annotation bounding box A match is considered a true positive; otherwise, it is considered a false positive. Precision is calculated at different confidence thresholds. With recall rate ,in The mean precision (AP) is calculated based on the precision-recall curve, and the mean precision (AP) satisfies the following: in The number of true positives. The number of false positives. The number of false negatives. These are the precision and recall values obtained by sorting according to the confidence threshold.
6. The method according to claim 1, characterized in that: In step five, the perceptual consistency metric is the Frechet Inception Distance (FID) metric based on feature distribution distance. The real image reference set and the synthetic image set are respectively input into a pre-trained InceptionV3 feature extraction network, and feature vectors for each image are extracted at the output of the average pooling layer of the feature extraction network. The mean vector and covariance matrix are calculated for the feature vectors of the real background image set and the synthetic image set, respectively, to obtain... , as well as , The FID index is calculated based on the following Frechet distance:
7. The method according to claim 1, characterized in that: In step six, the combined reward value satisfies the following form: in, To detect and provide feedback indicators, AP The perceived consistency index is FID. and The weighting coefficients are used; and the reward values are truncated within a range, such that... .
8. The method according to claim 1, characterized in that: In step seven, the multi-discrete parameter adjustment action includes at least the selection of the shadow direction category and discrete incremental instructions for each rendering control parameter to be optimized. The discrete incremental instructions for each rendering control parameter to be optimized take values from the set {-1, 0, +1}, and the j-th parameter is updated according to the following rules: in, For the first The discrete adjustment actions corresponding to each parameter. Step size, This refers to the range of parameter values.
9. The method according to claim 1, characterized in that: In step seven, the policy optimization algorithm is a near-end policy optimization algorithm, and the following constraint mechanism is introduced during the policy update process, all of which must be satisfied: (1) Restricted target strategy Compared to the old strategy The divergence of makes in, Indicates parameters Parameterized target policy in state The probability distribution of the following actions. This indicates the old policy used for constraints in the state. The probability distribution of the following actions. This indicates the preset divergence threshold. (2) Introduce an entropy regularization term into the objective function, such that... in, Let represent the objective function for policy optimization with constraints. This represents the policy loss term constructed based on the sampled data. This represents the expectation operation; Represents entropy operation. This represents the weight coefficient of the entropy regularization term. (3) Prune the update magnitude of the value function so that in, Indicates parameters Parameterized value function for state The value estimate, Represents the old value function with respect to the state The value estimate, This indicates that the input is clipped to a range. The clipping operator inside, This represents the pruning threshold of the value function. (4) Prune the gradient norm of the parameter update so that in, This represents the gradient vector of the objective function with respect to the parameters. This indicates the preset gradient clipping threshold.
10. The method according to claim 1, characterized in that: During training, the reward value, detection feedback index, and perceptual consistency index are recorded for each round or training cycle. Synthetic images and annotation information are stored in the corresponding directory according to the training cycle to achieve traceability of the training process.