Dense target detection method based on comparative learning characterization and reinforcement learning decision

Abstract

The invention provides a dense target detection method based on comparative learning characterization and reinforcement learning decision, which is suitable for dense animal individual detection in a livestock breeding scene. The method comprises the following steps: constructing a dense target detection model comprising a feature encoder, a reinforcement learning decision module and a detection head; the training process is divided into two stages: in the contrast learning characterization stage, an anchor sample, a positive sample and a negative sample are constructed, a first loss function is utilized to optimize a feature encoder, and an optimal feature encoder is obtained; in the reinforcement learning decision-making stage, the feature vector is used as an input state, a decision-making module dynamically selects a threshold lowering, maintaining or improving action, a detection head outputs a detection result in combination with the feature and the action, and parameters of the decision-making module are optimized through a reward function. During detection, after features of a to-be-detected image are extracted through the feature encoder, the optimized decision module selects the optimal action, and finally a detection result is output. The method has relatively high robustness and accuracy.

Description

Technical Field

[0001] This application belongs to the interdisciplinary field of computer vision and intelligent decision-making, specifically involving target detection and recognition technology in dense scenes, and specifically involving a dense target detection method based on contrastive learning representation and reinforcement learning decision-making. Background Technology

[0002] In traditional animal husbandry, pig counting faces practical problems such as low efficiency and large errors in manual counting, and high costs associated with wearable sensor counting due to easily detached tags. While computer vision-based automatic counting technology has advantages such as being non-contact and low-cost, it faces two technical bottlenecks in intensive farming scenarios: first, dense occlusion of pigs, varied postures, and uneven lighting lead to unstable feature extraction, making it difficult for mainstream detection methods to distinguish overlapping individuals; second, similar appearances and dynamic movements of individuals exacerbate detection confusion.

[0003] Existing technologies have significant limitations in addressing the aforementioned challenges:

[0004] In terms of feature enhancement, although the discriminative power of features can be improved by introducing attention modules (such as the SE module), the group interaction relationship is not modeled, and the performance is limited in highly dense occlusion scenarios.

[0005] In the direction of lightweight design, the number of parameters is reduced by optimizing the anchor frame size, but this comes at the cost of sacrificing the detection accuracy of small targets, such as an 8% decrease in the average accuracy (AP@0.5) at the intersection-union threshold of 50%.

[0006] In terms of dynamic optimization, relying on human experience to design threshold adjustment rules (such as loss functions with fixed parameters) lacks environmental adaptability and cannot dynamically optimize detection strategies according to real-time scenarios.

[0007] Given that existing pig detection methods share a common drawback—isolated optimization of feature extraction or decision-making mechanisms without forming a collaborative framework—they suffer from "semantic discontinuity," making them ill-suited for the complex challenges of densely dynamic scenarios. Therefore, constructing an intelligent detection framework that combines occlusion robustness with decision-making flexibility, overcoming the inherent contradiction between "feature representation accuracy" and "adaptive decision-making capability," is of great significance. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this application provides a dense target detection method based on contrastive learning representation and reinforcement learning decision-making, which has strong robustness and accuracy.

[0009] The technical solution provided in this application is as follows:

[0010] Firstly, this application provides a dense target detection method based on contrastive learning representation and reinforcement learning decision-making, the method comprising:

[0011] A dense object detection model is constructed and trained, the dense object detection model including a feature encoder, a reinforcement learning decision module, and a detection head;

[0012] The training-intensive object detection model includes a contrastive learning representation stage and a reinforcement learning decision stage:

[0013] In the contrastive learning representation stage, the original sample image is used as the anchor sample to generate corresponding positive and negative samples; the positive sample and the anchor sample contain the same individual (same individual); the negative sample and the anchor sample contain different individuals (different individuals).

[0014] The anchor sample, positive sample, and negative sample are input into the feature encoder to obtain the feature vectors of the anchor sample, positive sample, and negative sample, respectively.

[0015] Input the feature vectors of anchor samples, positive samples, and negative samples into the first loss function to obtain the corresponding first loss function values;

[0016] The parameters of the feature encoder are optimized based on the first loss function value to obtain the optimal feature encoder;

[0017] In the reinforcement learning decision-making stage, the feature vector output by the optimal feature encoder is used as the input state of the reinforcement learning decision-making module. The reinforcement learning decision-making module selects the optimal action from the action space based on the input state. The detection head outputs the corresponding detection result based on the feature vector output by the optimal feature encoder and the optimal action. The reward function value corresponding to the detection result is calculated. The parameters of the reinforcement learning decision-making module are optimized based on the reward function value to obtain the optimal reinforcement learning decision-making module. The action space includes: lowering the threshold, maintaining the threshold, and raising the threshold.

[0018] Dense object detection is performed based on a trained dense object detection model: the image to be detected is input into the optimal feature encoder to obtain the feature vector corresponding to the image to be detected; the feature vector corresponding to the image to be detected is used as the input state of the optimal reinforcement learning decision module, the optimal reinforcement learning decision module selects the best action from the action space based on the input state, and the detection head outputs the detection result of the image to be detected based on the feature vector and the best action.

[0019] In the above-mentioned technical solution of this application, the feature encoder can generate robust feature vectors through comparative learning training.

[0020] The parameters of the reinforcement learning decision module are optimized using a reinforcement learning decision module. It should be understood that this threshold is a detection confidence threshold, used to determine whether the target box is valid: if the confidence of the predicted box is greater than or equal to the threshold, it is retained; if the confidence of the predicted box is less than the threshold, it is filtered out; all the remaining predicted boxes constitute the retrieval results.

[0021] In this application, the parameters of the feature encoder are frozen during the reinforcement learning decision-making stage. The feature representation of the feature encoder is optimized by the reward signal feedback and policy gradient update generated by the reinforcement learning decision-making module, which indirectly optimizes the feature discriminative power and forms a collaborative optimization closed loop.

[0022] In one possible implementation, the original sample image is a dense animal image. The detection result is the detection result of individual dense animals.

[0023] In one possible implementation, generating corresponding positive and negative samples using the original sample image as the anchor sample includes:

[0024] Using the original sample image as the anchor sample, geometric enhancement is performed on the anchor sample to generate positive samples, and a hard example mining strategy is used to generate negative samples.

[0025] In one possible implementation, the geometric enhancement process includes: affine transformation (scaling), cyclic displacement, random occlusion, and Gaussian noise enhancement with a scaling factor.

[0026] The affine transformation adjusts the feature value amplitude by multiplying by a random factor (e.g., 0.6-1.4). The cyclic shift randomly rolls along the temporal dimension (1 to L-1 positions, where L is the sequence length, i.e., the temporal window length, and here the sequence value is the feature vector sequence of multiple frames of video images); the random occlusion randomly sets some features to zero with a certain retention rate (e.g., 70% retention rate) to achieve local occlusion; the Gaussian noise enhancement adds a certain standard deviation (e.g., standard deviation) to the noise level. Random noise.

[0027] The term "difficult case mining" refers to screening for negative samples with high confusion.

[0028] The hard case mining strategy includes cross-instance temporal sampling.

[0029] Cross-instance temporal sampling refers to selecting negative samples from unrelated individuals at different time points. The hard example mining strategy involves filtering out high-confusion negative samples from the negative samples obtained through cross-instance temporal sampling.

[0030] In one possible implementation, the cross-instance temporal sampling ensures that the temporal interval with the anchor sample is ≥6 frames.

[0031] In one possible implementation, the ratio of positive to negative samples can be controlled to 1:5.

[0032] In one possible implementation, the first loss function is a triplet loss function, the formula of which is:

[0033] ;

[0034] In the formula, The value of the triplet loss function. The feature vector of the anchor sample. and These are the feature vectors of the positive and negative samples, respectively. express and The cosine distance between them. express and The cosine distance between them To adjust the marginal values ​​of inter-class and intra-class distance intervals.

[0035] in, Intra-class distance, This represents the distance between classes.

[0036] In one possible implementation, during the contrastive learning representation phase, the intra-class distance is constrained to be ≤0.3 and the inter-class distance to be ≥0.8.

[0037] By setting constraints such as intra-class distance ≤ 0.3 and inter-class distance ≥ 0.8, we can enhance the consistency of characteristics of the same individual and improve the distinguishability of different individuals.

[0038] In one possible implementation, set This can force the inter-class distance to be 0.2 or more greater than the inter-class distance.

[0039] In one possible implementation, the feature vector is an L2-normalized 256-dimensional feature vector, i.e. ,in Represents the eigenvector of the th The element value of a dimension; This represents the L2 norm.

[0040] The core of L2 normalization lies in dividing the vector by its L2 norm (Euclidean norm), so that the length of the processed vector becomes 1.

[0041] In one possible implementation, during the reinforcement learning decision-making phase, the action space is defined as three discrete actions: lowering the threshold, maintaining the threshold, and raising the threshold. The threshold adjustment step size is 0.02, that is, lowering the threshold is done by decreasing it by 0.02 (-0.02) each time, raising the threshold is done by increasing it by 0.02 (+0.02) each time, and maintaining the threshold means that the threshold change is 0.

[0042] In one possible implementation, the threshold range is constrained to [0.1, 0.9].

[0043] Execution after threshold adjustment threshold The range is constrained to [0.1, 0.9], and when the threshold... When the value reaches the 0.1 or 0.9 boundary, it will automatically adjust to 0.01 in the opposite direction.

[0044] In one possible implementation, Gaussian noise with σ=0.01 is also added to the threshold with a 10% probability.

[0045] In one possible implementation, the reward function is a multi-objective reward function, with the following formula:

[0046] ;

[0047] in, For the reward function value, A reward will be given for the mAP50-95 index of the test results. Rewards will be given for the accuracy of the test results count. The threshold is used as a reasonable penalty; α, β, and γ are weighting coefficients.

[0048] In a multi-objective reward function, Used to improve detection accuracy Used to ensure the accuracy of quantity statistics Used to keep the threshold within a reasonable range.

[0049] In one possible implementation, α=0.3, β=0.6, and γ=0.1.

[0050] In one possible implementation, the parameters of the reinforcement learning decision module are optimized based on the reward function value, including: using the PPO algorithm, gradient pruning strategy, and cosine annealing learning rate adjustment strategy to optimize the parameters of the reinforcement learning decision module.

[0051] The gradient pruning is used to constrain the gradient of the loss function corresponding to the reinforcement learning decision module within the range [-0.3, 0.3].

[0052] The training of the dense target detection model also includes a validation phase, in which an early stopping mechanism is employed.

[0053] For example, an early stopping mechanism is triggered when the validation set mAP50-95 does not improve for 20 consecutive rounds, and the optimal model weights are preserved.

[0054] In one possible implementation, the first loss function is a weighted loss function:

[0055] ;

[0056] in, This represents the weighted loss function value. This indicates the loss from reinforcement learning. , As weighting coefficients, in one possible implementation, .

[0057] In one possible implementation, the feature encoder comprises a 3-layer fully connected network, employing the GELU activation function and Dropout regularization, and integrating a channel attention module in the last layer to generate feature channel weights through the fully connected layers, thereby enhancing selectivity for occlusion-sensitive features.

[0058] In one possible implementation, the reinforcement learning decision module includes a feature extraction module, a policy network, and a value network. The feature extraction module can employ a fully connected structure for dimensionality reduction of the feature vector, such as mapping the feature vector from 256 dimensions to 128 dimensions (256→128 nodes). The policy network includes fully connected layers and activation layers. The fully connected layers are used to obtain a three-dimensional vector (256→3 nodes) based on the dimensionality-reduced feature vector. The activation layers are used to calculate the action probability distribution based on the three-dimensional vector, outputting the probability of each threshold adjustment action (-0.02 / 0 / +0.02) for action decision-making. The value network includes fully connected layers and activation layers. The fully connected layers are used to obtain a one-dimensional vector (256→3 nodes) based on the dimensionality-reduced feature vector. The activation layers are used to evaluate the state value based on the one-dimensional vector, predicting the long-term reward value of the current input state (feature vector). .

[0059] The activation layer can use the ReLU activation function.

[0060] The parameters (such as weights) of the policy network and value network are initialized using orthogonal initialization.

[0061] The parameters of the policy network can be updated based on gradients, where the gradient is... or ,in Represents the parameters of the policy network (e.g., the weights of the fully connected layers from 256 to 128). Indicates the loss in reinforcement learning. For parameters The gradient; The error function of the policy network is represented. Expressing the request For parameters gradient, , This indicates the expectation value. This indicates that when the parameter is When in input state Select action The probability is obtained based on the output of the activation layer of the policy network; Indicates the input state Select action The advantage value is equal to the reward function value minus the long-term return value of the value network output.

[0062] The parameters of the value network can be updated based on gradients, and the gradient calculation formula is as follows: ,in Parameters representing the value network, The error function representing the value network. This indicates the expectation value. Expressing the request For parameters gradient, This indicates that when the parameter is The current input state obtained by time prediction The expected value of long-term returns.

[0063] This invention employs a collaborative framework combining contrastive learning representation and reinforcement learning decision-making, effectively resolving the inherent contradiction between "feature representation accuracy" and "decision-adaptive capability." By learning the feature distribution of targets in complex scenes and their decision-making patterns under dynamic environments from dense target image data, it achieves high-precision detection of dense targets, suitable for detecting densely packed individual animals in livestock farming scenarios. This invention can provide crucial support for target detection and intelligent management in dense scenes, and significantly contribute to the intelligent upgrading of fields such as livestock farming. It has the following advantages:

[0064] (1) High detection accuracy: Through geometric enhancement of the feature encoder and optimization of triplet loss, the feature discrimination of occluded and similar targets is effectively improved. Combined with the dynamic strategy adjustment of reinforcement learning, the detection accuracy in dense scenes is significantly improved.

[0065] (2) Strong environmental adaptability: The reinforcement learning decision module can adaptively adjust the detection parameters according to dynamic environmental changes such as lighting and occlusion, thus solving the problem of insufficient generalization ability of fixed parameter models;

[0066] (3) Collaborative optimization is efficient: Through the feature-decision closed-loop system, the bidirectional feedback of contrastive learning and reinforcement learning can be realized, which can not only enhance the robustness of features, but also improve the flexibility of decision-making. Attached Figure Description

[0067] Figure 1 This is a diagram illustrating the overall architecture of a dense target detection method based on contrastive learning representation and reinforcement learning decision-making according to an embodiment of the present invention.

[0068] Figure 2 This is a diagram of a contrastive learning decision architecture according to an embodiment of the present invention;

[0069] Figure 3 This is a reinforcement learning decision graph according to an embodiment of the present invention. Detailed Implementation

[0070] To enable those skilled in the art to better understand the present application, the technical solution of the present application will be further described in detail below with reference to the embodiments and accompanying drawings.

[0071] To better understand this application, the relevant terms will be explained first.

[0072] PPO stands for Proximal Policy Optimization. The PPO algorithm is a reinforcement learning algorithm designed to improve performance while maintaining stability and efficiency in complex tasks.

[0073] GELU, short for Gaussian Error Linear Unit, is a smooth activation function based on the Gaussian error function. It has the advantages of continuous derivatives and a near-Gaussian distribution, which helps to improve model performance and convergence speed.

[0074] mAP50-95, short for Average Precision at IoU Thresholds 50% to 95%, is the average precision value at IoU Thresholds 50% to 95%.

[0075] This invention discloses a dense target detection method based on contrastive learning representation and reinforcement learning decision-making. The method employs a two-stage optimization strategy: the first stage is contrastive representation learning, which involves constructing a feature encoder based on geometric feature enhancement, generating positive and negative sample pairs using affine transformation, and employing a triplet loss function to learn feature representations robust to targets (such as pig morphology) and interference factors such as occlusion; the second stage is reinforcement decision optimization, which designs a dynamic action space including frame fine-tuning and false positive suppression, and optimizes the detection strategy by driving the PPO algorithm through reward function optimization, outputting the detection results. This application effectively addresses the challenges posed by occlusion, similar appearance, and dynamic environments in dense target detection, improving detection performance. It is suitable for target detection tasks (such as dense pig herd detection) in dense scenes with severe occlusion, individual similarity, and dynamic environmental changes, such as livestock farming, and can achieve high-precision feature extraction and adaptive decision optimization for dense targets.

[0076] Specific embodiments according to this application will now be described with reference to the accompanying drawings.

[0077] Example 1:

[0078] This application provides a dense target detection method based on contrastive learning representation and reinforcement learning decision-making, including:

[0079] A dense object detection model is constructed and trained, the dense object detection model including a feature encoder, a reinforcement learning decision module, and a detection head;

[0080] The training-intensive object detection model includes a contrastive learning representation stage and a reinforcement learning decision stage:

[0081] In the contrastive learning representation stage, the original sample image is used as the anchor sample to generate corresponding positive and negative samples; the positive sample and the anchor sample contain the same individual (same individual); the negative sample and the anchor sample contain different individuals (different individuals).

[0082] The anchor sample, positive sample, and negative sample are input into the feature encoder to obtain the feature vectors of the anchor sample, positive sample, and negative sample, respectively.

[0083] Input the feature vectors of anchor samples, positive samples, and negative samples into the first loss function to obtain the corresponding first loss function values;

[0084] The parameters of the feature encoder are optimized based on the first loss function value to obtain the optimal feature encoder;

[0085] In the reinforcement learning decision-making stage, the feature vector output by the optimal feature encoder is used as the input state of the reinforcement learning decision-making module. The reinforcement learning decision-making module selects the optimal action from the action space based on the input state. The detection head outputs the corresponding detection result based on the feature vector output by the optimal feature encoder and the optimal action. The reward function value corresponding to the detection result is calculated. The parameters of the reinforcement learning decision-making module are optimized based on the reward function value to obtain the optimal reinforcement learning decision-making module. The action space includes: lowering the threshold, maintaining the threshold, and raising the threshold.

[0086] Dense object detection is performed based on a trained dense object detection model: the image to be detected is input into the optimal feature encoder to obtain the feature vector corresponding to the image to be detected; the feature vector corresponding to the image to be detected is used as the input state of the optimal reinforcement learning decision module, the optimal reinforcement learning decision module selects the best action from the action space based on the input state, and the detection head outputs the detection result of the image to be detected based on the feature vector and the best action.

[0087] In the above-mentioned technical solution of this application, the feature encoder can generate robust feature vectors through comparative learning training.

[0088] The parameters of the reinforcement learning decision module are optimized using a reinforcement learning decision module. It should be understood that this threshold is a detection confidence threshold, used to determine whether the target box is valid: if the confidence of the predicted box is greater than or equal to the threshold, it is retained; if the confidence of the predicted box is less than the threshold, it is filtered out; all the remaining predicted boxes constitute the retrieval results.

[0089] In this application, the parameters of the feature encoder are frozen during the reinforcement learning decision-making stage. The feature representation of the feature encoder is optimized by the reward signal feedback and policy gradient update generated by the reinforcement learning decision-making module, which indirectly optimizes the feature discriminative power and forms a collaborative optimization closed loop.

[0090] In some embodiments, the original sample image is an image containing a dense concentration of animal individuals. The detection result is the detection result of the dense concentration of animal individuals.

[0091] In some embodiments, generating corresponding positive and negative samples using the original sample image as the anchor sample includes:

[0092] Using the original sample image as the anchor sample, geometric enhancement is performed on the anchor sample to generate positive samples, and a hard example mining strategy is used to generate negative samples.

[0093] In some embodiments, the geometric enhancement process includes: affine transformation (scaling) with a scaling factor, cyclic displacement, random occlusion, and Gaussian noise enhancement.

[0094] The affine transformation adjusts the feature value amplitude by multiplying by a random factor (e.g., 0.6-1.4). The cyclic shift randomly rolls along the temporal dimension (1 to L-1 positions, where L is the sequence length, i.e., the temporal window length, and here the sequence value is the feature vector sequence of multiple frames of video images); the random occlusion randomly sets some features to zero with a certain retention rate (e.g., 70% retention rate) to achieve local occlusion; the Gaussian noise enhancement adds a certain standard deviation (e.g., standard deviation) to the noise level. Random noise.

[0095] The above scheme can generate feature representations that are robust to interference factors such as occlusion and deformation through a self-supervised learning mechanism.

[0096] The term "difficult case mining" refers to screening for negative samples with high confusion.

[0097] The hard case mining strategy includes cross-instance temporal sampling.

[0098] Cross-instance temporal sampling refers to selecting negative samples from unrelated individuals at different time points. The hard example mining strategy involves filtering out high-confusion negative samples from the negative samples obtained through cross-instance temporal sampling.

[0099] In some embodiments, the cross-instance temporal sampling ensures a temporal interval of ≥6 frames with the anchor sample.

[0100] In some embodiments, the ratio of positive to negative samples can be controlled to 1:5.

[0101] In some embodiments, the first loss function is a triplet loss function, the formula of which is:

[0102] ;

[0103] In the formula, The value of the triplet loss function. The feature vector of the anchor sample. and These are the feature vectors of the positive and negative samples, respectively. express and The cosine distance between them. express and The cosine distance between them To adjust the marginal values ​​of inter-class and intra-class distance intervals.

[0104] in, Intra-class distance, This represents the distance between classes.

[0105] By combining geometric augmentation and temporal constraints, diverse sample pairs can be generated. Using triplet marginal loss as the optimization objective, and combining data augmentation and positive / negative sample strategies, feature granularity can be upgraded.

[0106] In some embodiments, during the contrastive learning representation stage, the intra-class distance is constrained to be ≤0.3 and the inter-class distance to be ≥0.8.

[0107] By setting constraints such as intra-class distance ≤ 0.3 and inter-class distance ≥ 0.8, the intra-class compactness and inter-class separability of the feature space can be enhanced, that is, the consistency of features of the same individual can be enhanced and the distinguishability of different individuals can be improved.

[0108] In some embodiments, setting This can force the inter-class distance to be 0.2 or more greater than the inter-class distance.

[0109] In some embodiments, the feature vector is a 256-dimensional feature vector normalized by L2, i.e. ,in Represents the eigenvector of the th The element value of a dimension; This represents the L2 norm.

[0110] The core of L2 normalization lies in dividing the vector by its L2 norm (Euclidean norm), so that the length of the processed vector becomes 1.

[0111] In some embodiments, during the reinforcement learning decision-making stage, the action space is defined as three discrete actions: lowering the threshold, maintaining the threshold, and raising the threshold. The threshold adjustment step size is 0.02, that is, lowering the threshold is done by decreasing it by 0.02 (-0.02) each time, raising the threshold is done by increasing it by 0.02 (+0.02) each time, and maintaining the threshold is done by changing the threshold by 0.

[0112] In some embodiments, the threshold range is constrained to [0.1, 0.9].

[0113] Execution after threshold adjustment threshold The range is constrained to [0.1, 0.9], and when the threshold... When the value reaches the 0.1 or 0.9 boundary, it will automatically adjust to 0.01 in the opposite direction.

[0114] In some embodiments, Gaussian noise with σ=0.01 is also added to the threshold with a 10% probability.

[0115] In some embodiments, the reward function is a multi-objective reward function, with the following formula:

[0116] ;

[0117] in, For the reward function value, A reward will be given for the mAP50-95 index of the test results. Rewards will be given for the accuracy of the test results count. The threshold is used as a penalty for reasonableness; α, β, and γ are weighting coefficients that can be determined empirically.

[0118] In a multi-objective reward function, Used to improve detection accuracy Used to ensure the accuracy of quantity statistics Used to keep the threshold within a reasonable range.

[0119] In some embodiments, ;

[0120] ;in The target count in the sample detection results. This is the actual target count for the sample.

[0121] .

[0122] In some embodiments, α=0.3, β=0.6, γ=0.1.

[0123] In some embodiments, the parameters of the reinforcement learning decision module are optimized based on the reward function value, including: optimizing the parameters of the reinforcement learning decision module using the PPO algorithm, gradient pruning strategy, and cosine annealing learning rate adjustment strategy.

[0124] Gradient clipping is an optimization technique that prevents gradient explosion during training by limiting the maximum value of the gradient vector.

[0125] The gradient vector is a vector composed of the partial derivatives of a multivariate function with respect to all model parameters, pointing in the direction of the fastest growth of the multivariate function value. In this application, for example, the multivariate function of the policy network can be a reinforcement learning loss or error function, and the model parameters are the parameters of the policy module, such as the weights of the 256→128 fully connected layer in the policy module.

[0126] For the gradient vector The gradient clipping formula is: .

[0127] This gradient pruning operation can constrain the gradient of the loss function corresponding to the reinforcement learning decision module to the range [-0.3, 0.3].

[0128] In the cosine annealing learning rate adjustment strategy, the initial learning rate is 3e-4, and the decay limit is 1e-5.

[0129] The training of the dense target detection model also includes a validation phase, in which an early stopping mechanism is employed.

[0130] For example, an early stopping mechanism is triggered when the validation set mAP50-95 does not improve for 20 consecutive rounds, and the optimal model weights are preserved.

[0131] The above scheme is the Enhanced-GRPO algorithm (Enhanced Gradient-Regularized Proximal Optimization) designed in this application. Based on the PPO algorithm, it introduces gradient clipping and cosine annealing learning rate, combined with an early stopping mechanism, which can improve the stability of policy updates and convergence efficiency.

[0132] In some embodiments, the first loss function employs a weighted loss function:

[0133] ;

[0134] in, This represents the weighted loss function value. This indicates the loss from reinforcement learning. , As weighting coefficients, in some embodiments, .

[0135] By using a weighted loss function, a two-way feedback between features and decisions can be achieved. The high-dimensional features generated by contrastive learning provide semantically rich state inputs for reinforcement learning, while the reward signal of reinforcement learning drives gradient backpropagation to the feature encoder, prompting the model to prioritize learning difficult example features (such as differences in overlapping regions), forming a co-evolutionary closed loop of "feature extraction - policy optimization - feature feedback".

[0136] like Figure 1 and Figure 2 As shown, in some embodiments, the feature encoder includes a backbone network (such as ResNet34), a GELU activation function, a multi-head attention module (e.g., heads=8), and a projection head. The backbone network processes the input image to construct a 768-dimensional latent feature space. The GELU activation function is used to enhance nonlinear representation capabilities, and an attention mechanism is combined to focus on key local features. Finally, the 768-dimensional latent features are output by the projection head as a 256-dimensional robust feature vector normalized by L2, encoding high-level semantics such as target morphology and occlusion patterns.

[0137] In some embodiments, the feature encoder comprises a 3-layer fully connected network, employing the GELU activation function and Dropout regularization, and integrating a channel attention module in the last layer to generate feature channel weights through the fully connected layers, thereby enhancing the selectivity for occlusion-sensitive features.

[0138] In some embodiments, the reinforcement learning decision module (PPO policy network) includes a feature extraction module, a policy network, and a value network. The feature extraction module may employ a fully connected structure for dimensionality reduction of the feature vector, such as mapping the feature vector from 256 dimensions to 128 dimensions (256→128 nodes). The policy network (Actor) includes fully connected layers and activation layers. The fully connected layers are used to obtain a three-dimensional vector (256→3 nodes) based on the dimensionality-reduced feature vector. The activation layers are used to calculate the action probability distribution based on the three-dimensional vector, outputting the probability of each threshold adjustment action (-0.02 / 0 / +0.02) for action decision-making. The value network (Critic) includes fully connected layers and activation layers. The fully connected layers are used to obtain a one-dimensional vector (256→3 nodes) based on the dimensionality-reduced feature vector. The activation layers are used to evaluate the state value based on the one-dimensional vector, predicting the long-term reward value of the current input state (feature vector). .

[0139] The activation layer can use the ReLU activation function.

[0140] The parameters (such as weights) of the policy network and value network are initialized using orthogonal initialization.

[0141] The parameters of the policy network can be updated based on gradients, where the gradient is... or in Represents the parameters of the policy network (e.g., the weights of the fully connected layers from 256 to 128). Indicates the loss in reinforcement learning. For parameters The gradient; This represents the error function of the policy network. Expressing the request For parameters gradient, , This indicates the expectation value. This indicates that when the parameter is When in input state Select action The probability is obtained based on the output of the activation layer of the policy network; Indicates the input state Select action The advantage value is equal to the reward function value minus the long-term return value of the value network output.

[0142] The parameters of the value network can be updated based on gradients, and the gradient calculation formula is as follows: ,in Parameters representing the value network, The error function representing the value network. This indicates the expectation value. Expressing the request For parameters gradient, This indicates that when the parameter is The current input state obtained by time prediction The expected value of long-term returns.

[0143] This application constructs a reinforcement learning decision-making module based on dynamic policy optimization, modeling the optimization of detection parameters as a Markov decision process, and implementing dynamic policy adjustment through the PPO algorithm. This includes: designing a state-action space, where the state space uses a 256-dimensional feature vector output from contrastive learning (after global standardization); and defining the action space as three discrete options (lowering the threshold by -0.02, maintaining the threshold, and increasing the threshold by +0.02), with the threshold range constrained within [0.1, 0.9], and adding Gaussian noise (σ=0.01) with a 10% probability to avoid local optima. A multi-objective weighted reward mechanism is adopted to balance detection accuracy, counting accuracy, and threshold rationality constraints.

[0144] This application achieves a closed-loop improvement of feature representation and decision-making strategy through a collaborative optimization mechanism between the reinforcement learning decision module and the feature encoder. Specifically, regarding feature-state coupling, the feature vector output by the feature encoder is used as the fixed state input of the reinforcement learning decision module, where the feature encoder parameters are frozen during the reinforcement learning training phase to ensure the stability of the state representation. Regarding reward-loss feedback, by employing a weighted loss function, the reward signal of the PPO policy can be backpropagated to the feature encoder. Backpropagation to the feature encoder prioritizes optimizing the feature discrimination power of difficult examples such as occlusion and deformation, forming a collaborative closed loop of "feature optimization → policy improvement → feature feedback".

[0145] The effectiveness of the embodiments of this application will be verified through experiments below.

[0146] This experiment was conducted under the Windows 11 operating system, with the following hardware configuration: 11th Gen Intel(R) Core(TM) i5-11300H @ 3.10GHz CPU, RTX 3090 (24GB) GPU; the software used was the PyTorch 1.11.0 deep learning framework, CUDA 11.3 acceleration library, Python 3.8 programming language, and dependent libraries such as stable_baselines3==1.7.0 and gymnasium==0.29.1.

[0147] Stable_baselines3 refers to the reinforcement learning algorithm implementation framework, version 1.7.0.

[0148] Gymnasium refers to the reinforcement learning environment interface standard, version number 0.29.1.

[0149] The pig farm dataset used in this experiment includes aerial images of densely packed pigs taken from actual farming scenes and pig farms in Zhuzhou City, Hunan Province, covering images of pigs under different poses, densities, and lighting conditions. Image labels are in YOLO format .txt files, accurately recording the bounding box coordinates of each pig in the image, providing a detailed and accurate data foundation for model training and testing.

[0150] After preparing the dataset, a contrastive learning pre-training process is performed, which includes the following steps:

[0151] Step S1: Use a 3-layer GELU fully connected network with an attention module to output 256-dimensional features normalized by L2 and perform hyperspherical spatial projection.

[0152] Step S2: The loss function adopted is the triplet loss function (margin=0.2), and the optimization objective is: The discriminative power of features is improved by constraining intra-class distance to ≤0.3 and inter-class distance to ≥0.8.

[0153] Step S3: Data augmentation implements five augmentation strategies using the contrastive dataset. The scaling factor of the affine transformation is 0.6-1.4 to improve geometric robustness. The retention rate of random occlusion is 70% to simulate dense occlusion. Gaussian noise with σ=0.15 is used to enhance feature stability. In temporal sampling, positive samples are 5 consecutive frames and negative samples are spaced ≥6 frames apart to distinguish similar individuals. Rotation augmentation simulates viewpoint changes through random rolling displacement (1-state length).

[0154] After completing the contrastive learning pre-training, reinforcement learning training is performed to improve the model's decision-making and adaptive capabilities in dynamic scenarios. The specific process includes the following steps:

[0155] Step s1: The algorithm improvement adopts the Enhanced-GRPO algorithm based on PPO. Key optimizations include dynamic learning rate (initial value of 3e-4, which is decayed to 1e-5 through cosine annealing warm restarts), gradient pruning (the gradient is limited to the range of [-0.3, 0.3] through enhanced gradient regularization near-end optimization callback to prevent policy update oscillations) and early stopping mechanism (if the validation set mAP50-95 does not improve for 20 consecutive rounds, training is terminated, reducing computational resource consumption by 30%). The policy network uses a Compatible Feature Extractor to load contrastive learning pre-trained features. The network architecture is pi=[256,128] and vf=[256, 128], and the activation function is ReLU to ensure efficient coupling between semantic features and decision network.

[0156] Step s2: Monitoring and stability optimization. Enhanced GRPO Callback dynamically adjusts the update step size based on the reward gradient (accelerating convergence when performance is improved and maintaining stability when there are fluctuations), and mAP metric progress callback (Map ProgressCallback) records the mAP50-95 metric in real time, generates training curves and persists them to a CSV file, which is convenient for tracing the policy optimization process.

[0157] In the performance verification experiments, YOLOv8, which incorporates a sliding window Transformer (Swin-Transformer) module, was selected as the comparison model. This model replaces the traditional CNN backbone network by integrating the Swin-Transformer module and simultaneously optimizes the feature fusion network and anchor box design, significantly enhancing the target detection performance in dense scenes. This experiment used the same pig farm dataset from Zhuzhou City, Hunan Province, and an RTX 3090 GPU hardware environment. YOLOv8 was trained using the standard configuration (input image size (imgsz) = 320×320, batch size (batch) = 16, training epochs (epochs) = 300). Experimental results show that the mAP50-95 of the proposed method fluctuated by 0.80±0.05 in 5 repeated experiments, while the YOLOv8 method achieved 0.58, a significant improvement over the comparison model, verifying the superiority of the proposed method in dense pig detection tasks.

[0158] Example 2:

[0159] This embodiment provides an electronic device, including: a memory and a processor;

[0160] The memory is used to store computer programs;

[0161] The processor is configured to invoke the computer program to execute the method as described in Embodiment 1.

[0162] Example 3:

[0163] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is run on an electronic device, it causes the electronic device to perform the method described in Embodiment 1.

[0164] Example 4:

[0165] This embodiment provides a computer program product, including a computer program that, when run on an electronic device, causes the electronic device to perform the method described in Embodiment 1.

[0166] The specific implementation of the system, electronic device, computer-readable storage medium, and computer program product provided in this application can be referred to the specific embodiments of the above methods, and will not be repeated here.

[0167] The technical content of the above embodiments can be referred to each other. For the same or similar technical features, appropriate omissions have been made in some embodiments to avoid repeated descriptions.

[0168] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0169] The above description of the embodiments of this application is only a part of the embodiments of this application, and is used to enable those skilled in the art to implement or use the content of this application, and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A dense target detection method based on contrastive learning representation and reinforcement learning decision-making, characterized in that, The method includes: A dense object detection model is constructed and trained, the dense object detection model including a feature encoder, a reinforcement learning decision module, and a detection head; The training-intensive object detection model includes a contrastive learning representation stage and a reinforcement learning decision stage: In the contrastive learning representation stage, the original sample image is used as the anchor sample to generate corresponding positive and negative samples; the positive sample and the anchor sample contain the same individuals; the negative sample and the anchor sample contain different individuals; the anchor sample, positive sample, and negative sample are respectively input into the feature encoder to obtain the feature vectors of the anchor sample, positive sample, and negative sample; the feature vectors of the anchor sample, positive sample, and negative sample are input into the first loss function to obtain the corresponding first loss function value; the parameters of the feature encoder are optimized based on the first loss function value to obtain the optimal feature encoder; In the reinforcement learning decision-making stage, the feature vector output by the optimal feature encoder is used as the input state of the reinforcement learning decision-making module. The reinforcement learning decision-making module selects the optimal action from the action space based on the input state. The detection head outputs the corresponding detection result based on the feature vector output by the optimal feature encoder and the optimal action. The reward function value corresponding to the detection result is calculated. The parameters of the reinforcement learning decision-making module are optimized based on the reward function value to obtain the optimal reinforcement learning decision-making module. The action space includes: lowering the threshold, maintaining the threshold, and raising the threshold. Dense object detection is performed based on a trained dense object detection model: the image to be detected is input into the optimal feature encoder to obtain the feature vector corresponding to the image to be detected; the feature vector corresponding to the image to be detected is used as the input state of the optimal reinforcement learning decision module, the optimal reinforcement learning decision module selects the best action from the action space based on the input state, and the detection head outputs the detection result of the image to be detected based on the feature vector and the best action.

2. The method according to claim 1, characterized in that, The step of generating corresponding positive and negative samples using the original sample image as the anchor sample includes: using the original sample image as the anchor sample, performing geometric enhancement processing on the anchor sample to generate positive samples, and generating negative samples through a hard example mining strategy.

3. The method according to claim 1, characterized in that, The first loss function uses the triplet loss function, and the formula for the triplet loss function is as follows: ； In the formula, The value of the triplet loss function. The feature vector of the anchor sample. and These are the feature vectors of the positive and negative samples, respectively. express and The cosine distance between them. express and The cosine distance between them To adjust the marginal values ​​of inter-class and intra-class distance intervals.

4. The method according to claim 3, characterized in that, The reward function is a multi-objective reward function, and the formula is as follows: ； in, For the reward function value, A reward will be given for the mAP50-95 index of the test results. Rewards will be given for the accuracy of the test results count. The threshold is used as a reasonable penalty; α, β, and γ are weighting coefficients.

5. The method according to claim 4, characterized in that, The first loss function uses a weighted loss function: ； in, This represents the weighted loss function value. This indicates the loss from reinforcement learning. , As weighting coefficients, in some embodiments, .

6. The method according to any one of claims 1 to 5, characterized in that, During the reinforcement learning decision-making phase, the threshold adjustment step size is 0.02, and the threshold range is constrained to [0.1, 0.9].

7. The method according to any one of claims 1 to 5, characterized in that, The parameters of the reinforcement learning decision module are optimized based on the reward function value, including: using the PPO algorithm, gradient pruning strategy and cosine annealing learning rate adjustment strategy to optimize the parameters of the reinforcement learning decision module.

8. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is configured to invoke the computer program to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1 to 7.

Images