Small target detection method based on multi-agent reinforcement learning
By employing a multi-agent reinforcement learning approach, combined with image feature extraction and enhancement processing, the robustness and accuracy issues of small target detection under multi-transformation imaging conditions are addressed, achieving highly efficient detection results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-03-03
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies struggle to effectively address the robustness and accuracy issues of small target detection under varying imaging conditions, especially in aerial photography and remote sensing images. Small targets have low feature recognition and are easily obscured by complex backgrounds, making it difficult to balance detection accuracy and recall.
A multi-agent reinforcement learning approach is adopted, which preprocesses the image to be detected through the agent action parameter model, combines image feature extraction and fusion by the first agent, image enhancement processing by the second agent, dynamically matches imaging conditions, and optimizes the output of the detection model.
It significantly improves the accuracy and stability of small target detection, adapts to varying imaging conditions, and enhances the robustness and efficiency of detection.
Smart Images
Figure CN122066933A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision and target detection technology, and in particular to a small target detection method based on multi-agent reinforcement learning. Background Technology
[0002] High-altitude aerial photography and remote sensing imagery technologies have irreplaceable application value in several key fields, and small target detection is the core of these technologies' implementation. However, small target detection in specific application scenarios has long faced a dual technical bottleneck, severely restricting detection performance.
[0003] The main technical bottlenecks include, firstly, image quality degradation caused by complex imaging conditions: during aerial photography and remote sensing imaging, the image quality is easily affected by various factors such as severe fluctuations in illumination, atmospheric haze scattering, flight platform vibration, and image compression distortion, which further exacerbates the reduction in the recognizability of small target features and significantly reduces the robustness of traditional detection methods; secondly, the detection difficulty caused by the characteristics of small targets themselves: small targets account for a very low percentage of pixels in the image (for example, in a 512×512 pixel image, the percentage is less than 32×32 pixels), the feature information is extremely sparse, the edge contours are blurred, and they are easily obscured by complex backgrounds (such as vegetation, building textures, and clouds), making it difficult to balance detection accuracy and recall, resulting in prominent issues of missed detections and false detections.
[0004] Existing technologies mainly fall into two categories. The first category involves image enhancement, such as denoising, dehazing, and contrast adjustment, with the core objective of improving the basic quality of the input image. However, its limitation lies in its poor adaptability, as it is mostly designed for specific image degradation scenarios (such as single haze or fixed noise) and cannot dynamically match the varied imaging conditions in real-world applications. The second category involves detection model algorithm optimization, including multi-scale feature fusion, attention mechanism embedding, and detection head improvement, with the core objective of enhancing the model's adaptability to low-quality images. Its limitation lies in the lack of synergy; simply stacking preprocessing enhancement and model optimization techniques can easily lead to performance redundancy (such as over-enhancement introducing noise) and performance degradation (such as conflict between the targets of feature fusion and image enhancement), making it difficult to balance detection robustness and inference efficiency.
[0005] Therefore, it is evident that applying the two solutions mentioned above alone is insufficient to effectively adapt to the diverse application scenarios in reality, resulting in unsatisfactory small target detection results. Summary of the Invention
[0006] Purpose of the invention: This invention provides a small target detection method based on multi-agent reinforcement learning, which aims to solve the problem that existing single image enhancement or algorithm detection models are difficult to effectively cope with the ever-changing application scenarios of small target detection.
[0007] Technical Solution: This invention provides a small target detection method based on multi-agent reinforcement learning, comprising: inputting the image to be detected into an agent action parameter model, and outputting action parameters and decision vectors as the basis for the execution of the agent execution model; the agent execution model includes a first agent and a second agent, wherein the first agent is used for image feature extraction and fusion of global and local features of the image; the second agent is used for various image enhancement processing of the image; the agent execution model processes the image to be detected according to the action parameters and the decision vector; the actions of the agent execution model determined by the action parameters include: the importance weighting degree of the image features extracted by the first agent, the fusion ratio of the global and local features of the image to be detected by the first agent, the intensity of the image enhancement processing performed by the second agent, and the order of the image enhancement processing performed by the second agent; the actions of the agent execution model determined by the decision vector include: executing one or both of the first agent or the second agent, executing the first agent and the second agent serially, in parallel, or iteratively; inputting the output of the agent execution model into a target detection model, and outputting a target recognition result.
[0008] Specifically, global features are extracted from the image to be detected; local features are extracted from image regions within bounding boxes of multiple target regions in the image to be detected; the global and local features are converted into a unified size and dimension and then concatenated, and the concatenated feature is used as a state vector input to the agent action parameter model.
[0009] Specifically, after the agent action parameter model outputs action vectors, the corresponding action vectors are used to calculate action parameters according to the corresponding preset mapping formula, so that the obtained action parameters fall within the target parameter range; the value range of all action vectors is [-1, 1].
[0010] Specifically, the action parameters include: the channel attention factor of the first agent, the fusion ratio of the global and local features of the first agent, and the denoising parameters, brightness parameters, contrast parameters, saturation parameters, sharpening parameters, and order parameters of the second agent.
[0011] Specifically, the actions of the agent executing the model, determined by the action parameters, include: based on the magnitude of the channel attention factor, the first agent determines the degree to which it amplifies or suppresses the image features extracted from the corresponding channel; based on the magnitude of the fusion ratio, the first agent performs a weighted fusion of global and local features; based on the magnitudes of the denoising parameters, brightness parameters, contrast parameters, saturation parameters, and sharpening parameters, the second agent determines the intensity of denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening; and based on the sequence parameters, the second agent determines one or more of denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening, as well as the execution order.
[0012] Specifically, when executing the first and second intelligent agents in sequence, the decision vector determines whether to execute the first or second intelligent agent first.
[0013] Specifically, when the first agent and the second agent are executed in parallel, the output of the first agent and the output of the second agent are concatenated to obtain the output result of the agent execution model.
[0014] Specifically, the feedback iterative execution of the first and second intelligent agents includes: iteratively executing a detection step when the termination condition is not met; the detection step includes: inputting the image to be detected into the intelligent agent action parameter model, inputting the output of the intelligent agent action parameter model into the intelligent agent execution model, processing the image to be detected based on the output of the intelligent agent action parameter model, inputting the output of the intelligent agent execution model into the target detection model, and then feeding back the output of the target detection model to the intelligent agent action parameter model; the termination condition includes: the deviation between the output results of the target detection model for a consecutive preset number of times is less than a standard threshold.
[0015] Specifically, the training process of the agent action parameter model includes: inputting labeled training images into the agent action parameter model; processing the training images according to the output of the agent action parameter model; inputting the output of the agent execution model into the target detection model; comparing the output of the target detection model with the labels of the training images; and feeding back the comparison results to the agent action parameter model.
[0016] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by using the agent action parameter model to pre-detect the image to be detected, the more suitable image preprocessing method for the image to be detected is determined, and the agent execution model executes the corresponding image preprocessing method, which effectively improves the accuracy of small target detection. Attached Figure Description
[0017] Figure 1A schematic diagram illustrating the steps of the small target detection method based on multi-agent reinforcement learning provided by the present invention; Figure 2 This is a schematic diagram of the execution logic of the intelligent agent execution model provided by the present invention. Detailed Implementation
[0018] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0019] See Figure 1 This is a schematic diagram illustrating the steps of the small target detection method based on multi-agent reinforcement learning provided by the present invention.
[0020] In this embodiment of the invention, the image to be detected is input into the agent action parameter model, and the action parameters and decision vectors are output as the basis for the agent's execution model.
[0021] In practice, the image to be detected, which is an aerial image or a remote sensing image, can be pre-processed with format parsing and normalization (mapping pixel values to the [0,1] range), and optionally perform local cropping and scaling operations (to adapt to the input size of the detection network) to output standardized image data.
[0022] In this embodiment of the invention, the input of the image to be detected into the agent action parameter model specifically includes: extracting global features of the image to be detected; extracting local features of the image from the image regions within the bounding boxes of multiple target regions on the image to be detected; converting the global features and local features of the image into a unified size and dimension and then concatenating them, and then inputting the concatenated part as a state vector into the agent action parameter model.
[0023] In practical implementation, in order to enable the agent action parameter model to more directly collect and analyze the image features of the image to be detected, the processed feature map can be directly input into the agent action parameter model.
[0024] In practice, the state vector is obtained by concatenating and normalizing two parts of features. The first part is the original image, whose features are extracted by the detection network backbone and then subjected to adaptive pooling (Pool64(•)). The second part is the local image, which is scaled and then has its features extracted by the same detection network backbone and subjected to adaptive pooling (Pool64(•)). The local image is the portion of the target region selected by the bounding box. The selection of the bounding box within the target region can typically be done in two ways: either the bounding box fills the entire image to be detected, or preliminary target detection is performed beforehand, and the bounding box within the target region selects the image area where the target has been detected. Both the original image and the local image can be converted into 3×64×64-dimensional features and input into the network using transform in deep learning frameworks such as PyTorch.
[0025] In practical implementation, the agent action parameter model preferably adopts a convolutional policy network (CnnPolicy) as the core architecture, which can be trained using a near-end policy optimization algorithm to learn the mapping relationship between "state vector → action parameters and decision vector".
[0026] See Figure 2 This is a schematic diagram of the execution logic of the intelligent agent execution model provided by the present invention.
[0027] In this embodiment of the invention, the agent execution model includes a first agent A1 and a second agent A2, wherein the first agent is used for image feature extraction and fusion of global and local features of the image; and the second agent is used for various image enhancement processing of the image.
[0028] In practice, the first intelligent agent is mainly used for feature adjustment, while the second intelligent agent is mainly used for image enhancement processing.
[0029] In this embodiment of the invention, the intelligent agent execution model processes the image to be detected based on the action parameters and the decision vector.
[0030] In this embodiment of the invention, the actions of the agent executing the model determined by the action parameters include: the importance weighting degree of the image features extracted by the first agent when extracting the image to be detected, the fusion ratio of the global and local features of the image to be detected by the first agent, the intensity of the image enhancement processing performed by the second agent, and the order in which the second agent performs the image enhancement processing.
[0031] In this embodiment of the invention, after the intelligent agent action parameter model outputs action vectors, the corresponding action vectors are used to calculate action parameters according to the corresponding preset mapping formula, so that the obtained action parameters fall within the target parameter range; the value range of all action vectors is [-1, 1].
[0032] In this embodiment of the invention, the action parameters include: the channel attention factor of the first agent, the fusion ratio of the global features and local features of the first agent, and the denoising parameters, brightness parameters, contrast parameters, saturation parameters, sharpening parameters, and order parameters of the second agent.
[0033] In specific implementation, the range of values for various types of vector parameters output by the intelligent agent action parameter model is consistent (in this embodiment, it is [-1,1]). However, the magnitude of the parameter values has different effects on different calculation programs. For example, the range of values for the denoising parameter in the denoising process and the brightness parameter in the brightness adjustment process are obviously not the same. Therefore, in order to adapt to their respective calculation programs, after the intelligent agent action parameter model outputs the action vector, the corresponding action vector is calculated according to the corresponding preset mapping formula to obtain the action parameters.
[0034] In practice, the mapping relationship is as follows: The channel attention factor of the first agent is α, the value of α is [0.5, 1.5], the action vector is A[0], and the linear mapping formula is: α = 1.0 + 0.5×A[0]; The fusion ratio of the first agent is β, the value range of β is [0,1], the action vector is A[1], and the linear mapping formula is: β = 0.5×(A[1] + 1); The second agent's denoising parameter δ has a range of [0, 0.2], and its action vector is A[2]. The linear mapping formula is: δ = 0.1×(A[2] + 1); The brightness parameter b of the second agent has a value range of [-0.2, 0.2], and the action vector A[3] has a linear mapping formula of: b = 0.2×A[3]; The contrast parameter c of the second agent has a range of [0.8, 1.2], and the action vector A[4] has a linear mapping formula of: c = 1.0 + 0.2×A[4]; The saturation parameter s of the second agent has a range of [0.8, 1.2], and the action vector A[5] has a linear mapping formula of: s = 1.0 + 0.2×A[5]; The sharpening parameter r of the second agent has a range of [0,2], and the action vector A[6] has a linear mapping formula of: r = A[6] + 1.
[0035] Mapping logic description: The continuous correspondence between the action amplitude and the parameter range is achieved through linear scaling. For example, when A[0]=1, α=1.5 to enhance the response of the small target channel, and when A[0]=-1, α=0.5 to suppress the response of the background channel.
[0036] In this embodiment of the invention, the actions of the agent executing the model, determined by the action parameters, include: a first agent determining the degree to amplify or suppress the image features extracted from the corresponding channel based on the magnitude of the channel attention factor; a first agent performing weighted fusion of global and local features based on the magnitude of the fusion ratio; a second agent determining the intensity of denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening based on the magnitudes of the denoising parameters, brightness parameters, contrast parameters, saturation parameters, and sharpening parameters; and a second agent determining one or more of denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening, and the execution order, based on the sequence parameter.
[0037] In practice, the sequence parameter determines which of the five image enhancement processes—denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening—are executed, which are not executed, or all of them are executed, as well as the corresponding execution order.
[0038] In practice, channel attention adjustment refers to applying a scaling factor α to the aligned feature map by channel. When α>1, the response and importance weight of the features extracted by the relevant channels are amplified; when α<1, the response and importance weight of the features extracted by the relevant channels are suppressed.
[0039] In practice, global and local features are fused and spatially aligned (preferably by upsampling high-level features or downsampling low-level features to the same size), and the number of channels is aligned using a 1×1 convolution; then, weighted fusion is performed according to the fusion ratio β. The fusion result can be adaptively pooled using Pool64(•) to obtain a uniform size representation, which is used for state feedback or subsequent detection.
[0040] In practice, noise reduction is achieved by controlling the Gaussian kernel standard deviation σ through the parameter δ∈[0,0.2].
[0041] In practice, brightness adjustment is achieved by adjusting the image brightness using the parameter b∈[−0.2,0.2], and the clip function can be used to limit the output to the [0,1] interval.
[0042] In practice, contrast adjustment is achieved by adjusting the image contrast using the parameter c∈[0.8,1.2]. First, the average pixel value μ of the image is calculated, and then the output can be limited using the clip function.
[0043] In practice, saturation adjustment involves first converting the RGB image to the HSV color space, adjusting the saturation channel using the parameter s∈[0.8,1.2], and then converting it back to the RGB space.
[0044] In practice, sharpening is adjusted by controlling the sharpening intensity through the parameter r∈[0,2], and sharpening is achieved based on the difference between the original image and the blurred image.
[0045] In this embodiment of the invention, the actions of the agent execution model determined by the decision vector include: executing one or both of the first agent or the second agent, executing serially, executing in parallel, or executing the first agent and the second agent iteratively.
[0046] In practical implementation, the decision vector can determine whether only the first agent or only the second agent is executed. If both the first and second agents need to be executed according to the decision vector, it is further determined whether to execute them sequentially, in parallel, or through feedback iterative execution.
[0047] In this embodiment of the invention, when the first intelligent agent and the second intelligent agent are executed sequentially, the decision vector determines whether the first intelligent agent or the second intelligent agent should be executed first.
[0048] In practice, when executing sequentially, if the first agent should be executed first, then the execution order is the first agent followed by the second agent, and vice versa.
[0049] In this embodiment of the invention, when the first agent and the second agent are executed in parallel, the output of the first agent and the output of the second agent are concatenated to obtain the output result of the agent execution model.
[0050] In practice, during parallel execution, the first agent executes in parallel with the second agent in the image branch on the feature branch, and then they merge at the detection entry point after completion.
[0051] In this embodiment of the invention, the feedback iterative execution of the first and second intelligent agents includes: iteratively executing a detection step when the termination condition is not met; the detection step includes: inputting the image to be detected into the intelligent agent action parameter model, inputting the output of the intelligent agent action parameter model into the intelligent agent execution model, processing the image to be detected by the intelligent agent execution model according to the output of the intelligent agent action parameter model, inputting the output result of the intelligent agent execution model into the target detection model, and then feeding back the output result of the target detection model to the intelligent agent action parameter model; the termination condition includes: the deviation between the output results of the target detection model for a consecutive preset number of times (which can be set according to the actual application scenario) is less than a standard threshold; and the computational load reaches the overhead budget.
[0052] In practical implementation, the deviation between the output results of the target detection model for a consecutive preset number of times specifically refers to the deviation between all the detected target regions.
[0053] In this embodiment of the invention, a labeled training image is input into an agent action parameter model. The agent execution model processes the training image based on the output of the agent action parameter model. The output of the agent execution model is input into a target detection model. The output of the target detection model is compared with the label of the training image, and the comparison result is fed back to the agent action parameter model.
[0054] In practice, during the training of the agent action parameter model, the training can be judged as completed based on whether the detection results meet the preset requirements (e.g., preset small target AP≥75%, small target recall Recsmall≥80%, single frame false positives FP≤3, and false negatives FN≤2).
[0055] In practice, by training the agent action parameter model as described above, it is possible to prioritize the use of the second agent to improve imaging quality in scenarios dominated by image degradation; prioritize the use of the first agent to enhance the representation of small targets in scenarios dominated by feature sparsity; and enable the feedback interaction mode to iteratively optimize detection stability in scenarios with dual complexity.
[0056] In this embodiment of the invention, the output of the intelligent agent execution model is input into the target detection model, and the target recognition result is output.
[0057] In practice, the small target detection results (including predicted bounding boxes, categories, and confidence levels) are recorded by the log audit module, along with core information such as action parameters, decision vectors, detection metrics, and agent action parameter model update parameters, thus completing one detection process.
[0058] In practice, the "feature adjustment" and "image preprocessing enhancement" functions are separated into two independent and collaborative intelligent agents. A closed-loop self-optimization mechanism is built based on reinforcement learning to achieve adaptive processing strategy scheduling for images of different quality and different scene distributions, ensuring the stability and controllability of the detection output, and achieving a significant improvement in the stability, consistency and accuracy of small target detection.
Claims
1. A small target detection method based on multi-agent reinforcement learning, characterized in that, include: The image to be detected is input into the agent action parameter model, and the output is the action parameters and decision vector that serve as the basis for the execution of the agent execution model. The agent execution model includes a first agent and a second agent, wherein the first agent is used for image feature extraction and the fusion of global and local features of the image; the second agent is used for various image enhancement processing. The intelligent agent execution model processes the image to be detected based on the action parameters and the decision vector; The actions of the agent executing the model, determined by the action parameters, include: the importance weighting of image features when the first agent extracts the image to be detected, the fusion ratio when the first agent fuses the global and local features of the image to be detected, the intensity of the image enhancement processing performed by the second agent, and the order in which the second agent performs the image enhancement processing. The actions of the agent execution model determined by the decision vector include: executing one or both of the first agent or the second agent, executing serially, executing in parallel, or executing the first agent and the second agent iteratively; The output of the agent's execution model is input into the target detection model, and the target recognition result is output.
2. The small target detection method based on multi-agent reinforcement learning according to claim 1, characterized in that, The output yields action parameters and decision vectors that serve as the basis for the agent's execution model, including: After the action parameter model of the intelligent agent outputs action vectors, the corresponding action vectors are used to calculate the action parameters according to the corresponding preset mapping formula, so that the obtained action parameters fall within the target parameter range; the value range of all action vectors is [-1,1].
3. The small target detection method based on multi-agent reinforcement learning according to claim 1, characterized in that, The action parameters include: the channel attention factor of the first agent, the fusion ratio of the global and local features of the first agent, and the denoising parameters, brightness parameters, contrast parameters, saturation parameters, sharpening parameters, and order parameters of the second agent.
4. The small target detection method based on multi-agent reinforcement learning according to claim 3, characterized in that, The actions of the agent, determined by the action parameters, include: based on the magnitude of the channel attention factor, the first agent determines the degree to which it amplifies or suppresses the image features extracted from the corresponding channel; based on the magnitude of the fusion ratio, the first agent performs a weighted fusion of global and local features; based on the magnitudes of the denoising parameters, brightness parameters, contrast parameters, saturation parameters, and sharpening parameters, the second agent determines the intensity of denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening; and based on the sequence parameter, the second agent determines one or more of denoising, brightness adjustment, contrast adjustment, saturation adjustment, and sharpening, as well as the execution order.
5. The small target detection method based on multi-agent reinforcement learning according to claim 1, characterized in that, When executing the first and second agents in sequence, the decision vector determines whether to execute the first or second agent first.
6. The small target detection method based on multi-agent reinforcement learning according to claim 1, characterized in that, When the first agent and the second agent are executed in parallel, the output of the first agent and the output of the second agent are concatenated to obtain the output result of the agent execution model.
7. The small target detection method based on multi-agent reinforcement learning according to claim 1, characterized in that, The feedback iterative execution of the first and second agents includes: If the termination condition is not met, continue the detection steps iteratively. The detection steps include: inputting the image to be detected into the agent action parameter model, inputting the output of the agent action parameter model into the agent execution model, processing the image to be detected according to the output of the agent action parameter model, inputting the output of the agent execution model into the target detection model, and then feeding back the output of the target detection model to the agent action parameter model. The termination condition includes: the deviation between the output results of the target detection model for a consecutive preset number of times is less than a standard threshold.
8. The small target detection method based on multi-agent reinforcement learning according to claim 1, characterized in that, The training process of the agent action parameter model includes: The labeled training images are input into the agent action parameter model. The agent execution model processes the training images based on the output of the agent action parameter model. The output of the agent execution model is input into the target detection model. The output of the target detection model is compared with the labels of the training images, and the comparison result is fed back to the agent action parameter model.