Cross-domain battlefield target recognition method and system based on deep reinforcement learning

Abstract

The present invention provides a battlefield target cross-domain recognition method and system based on deep reinforcement learning, which relates to the field of target recognition technology. The method comprises the following steps: based on a source domain battlefield environment and a target domain battlefield environment, respectively obtaining a labeled source domain target image and an unlabeled target domain target image, extracting source domain image features and target domain image features through a deep convolutional neural network model; based on an adversarial domain adaptation network, mapping to the same feature space through adversarial training, learning a common feature representation, and obtaining a target domain image domain adaptation feature by reducing distribution differences; inputting the target domain image domain adaptation feature into a deep reinforcement learning module, learning an optimal classification strategy for a target domain battlefield target recognition task through interactive exploration between an intelligent agent and a reinforcement learning environment including a state space, an action space, and a reward function, classifying the target domain image domain adaptation feature based on the optimal classification strategy, and outputting a battlefield target cross-domain recognition result.

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and in particular to a battlefield target cross-domain recognition method and system based on deep reinforcement learning. Background Art

[0002] With the continuous evolution of modern warfare, the battlefield environment is becoming increasingly complex and changeable.

[0003] Accurate and real-time identification of target objects on the battlefield is of great significance for battlefield situation awareness, command decision-making and weapon control.

[0004] Traditional target recognition methods are mainly used for training and identifying targets in specific environments. When faced with new battlefield environments, they are often difficult to apply directly. They require re-collecting target data in the environment and conducting model training, and their adaptability and generalization capabilities are poor.

[0005] Existing cross-domain target recognition methods based on deep learning still have some shortcomings. They ignore the optimization of the decision-making level and require a large amount of target domain data for training, which is difficult to meet in practical applications.

[0006] In summary, the present invention combines the advantages of deep learning and reinforcement learning, and realizes adaptive recognition of target domain images by constructing a Markov decision process and designing a deep reinforcement learning model. It not only considers the alignment at the feature level, but also realizes the adaptability at the decision level through strategy optimization, thereby improving the performance and robustness of cross-domain recognition and being able to solve the problems in the prior art. Summary of the invention

[0007] The embodiments of the present invention provide a method and system for cross-domain identification of battlefield targets based on deep reinforcement learning, which can solve the problems in the prior art.

[0008] According to a first aspect of the embodiments of the present invention,

[0009] A battlefield target cross-domain recognition method based on deep reinforcement learning is provided, including:

[0010] Based on the source domain battlefield environment and the target domain battlefield environment, labeled source domain target images and unlabeled target domain target images are obtained respectively, and the source domain image features and the target domain image features are extracted through a deep convolutional neural network model;

[0011] Based on an adversarial domain adaptation network, the source domain image features and the target domain image features are mapped to the same feature space through adversarial training, common feature representation is learned, and domain adaptation features of the target domain image are obtained by reducing distribution differences;

[0012] The target domain image domain adaptation features are input into the deep reinforcement learning module. Through the interactive exploration of the intelligent agent and the reinforcement learning environment containing state space, action space and reward function, the optimal classification strategy for the target domain battlefield target recognition task is learned, and the target domain image domain adaptation features are classified based on the optimal classification strategy, and the cross-domain recognition results of battlefield targets are output.

[0013] In an optional embodiment,

[0014] The deep convolutional neural network model includes:

[0015] Using ResNet-50 as the backbone network, a deep deformable convolutional neural network is constructed. The position offset is learned by setting an additional convolutional layer at the initial position, and the convolutional layers in the third, fourth, and fifth residual blocks of ResNet-50 are replaced with deformable convolutional layers. The position of the convolution sampling is adaptively adjusted to obtain the original feature map.

[0016] After each of the deformable convolutional layers, a spatial attention module and a channel attention module are connected in parallel, wherein the spatial attention module learns the spatial importance of each position of the original feature map through global average pooling and a fully connected layer to obtain a spatial attention weight; the channel attention module learns the channel importance of different feature channels through global average pooling, a fully connected layer and a sigmoid activation function to obtain a channel attention weight, and the spatial attention weight and the channel attention weight are multiplied with the original feature map to obtain a weighted feature map;

[0017] After the third, fourth, and fifth residual blocks of the deep deformable convolutional neural network, a multi-scale feature map of the corresponding level is extracted based on the weighted feature map, and the size of the multi-scale feature map is adjusted by adaptive upsampling or adaptive downsampling to keep the size consistent. Multi-scale feature fusion is performed by element-by-element addition to obtain image features.

[0018] In an optional embodiment,

[0019] Based on the adversarial domain adaptation network, the source domain image features and the target domain image features are mapped to the same feature space through adversarial training, and common feature representation is learned. By reducing the distribution difference, the domain adaptation features of the target domain image are obtained, including:

[0020] The adversarial domain adaptation network receives source domain image features and target domain image features through a pre-built feature mapper, and maps them to a common feature space respectively, to obtain source domain common space features and target domain common space features;

[0021] The source domain common space features and the target domain common space features are received through a pre-built domain confusion feature propagation module, and the source domain common space features and the target domain common space features are respectively input into two independent fully connected layers to obtain the source domain initial confusion features and the target domain initial confusion features, and propagate alternately between the source domain and the target domain. After multiple iterations, the final source domain confusion features and the final target domain confusion features are obtained, wherein in each iteration, the source domain confusion features and the target domain confusion features corresponding to each iteration are obtained by weighted summation and activation function;

[0022] The source domain confusion feature and the target domain confusion feature constitute a confusion feature, which is input into a pre-built domain discriminator to determine whether the confusion feature is a source domain confusion feature or a target domain confusion feature, and iteratively optimize the domain discriminator by maximizing the domain discrimination loss;

[0023] By using a pre-built feature classifier, receiving source domain confusion features, annotating and predicting the source domain image features, and iteratively optimizing the feature classifier by minimizing the classification loss in combination with the real annotations corresponding to the source domain image features;

[0024] Construct a domain confusion loss, iteratively optimize the feature generator by minimizing the source domain discriminant loss of the source domain confusion feature in the domain discriminator and the target domain discriminant loss of the target domain confusion feature in the domain discriminator;

[0025] Jointly optimizing the feature generator, the domain confusion feature propagation module and the domain discriminator to collaboratively learn the domain confusion feature representation; fixing the parameters of the feature generator and the domain confusion feature propagation module, and training the feature classifier based on the source domain confusion feature; iterating alternately until the preset convergence condition is met, and obtaining the trained feature generator and the domain confusion feature propagation module;

[0026] The target domain image features are input into the trained feature generator to obtain the mapped target domain features. After multiple iterations of the domain confusion feature propagation module, the target domain confusion features are obtained to determine the domain adaptation features of the target domain image.

[0027] In an optional embodiment,

[0028] The domain discrimination loss is formulated as follows: ;

[0029] in, L d represents the domain discrimination loss function, E represents the expectation operator, f s represents the source domain confusion feature, p (·) represents the distribution of the corresponding feature, D (·) represents the output result of the domain discriminator,f t represents the target domain confusion feature, λ gp represents the weight of the gradient penalty term, represents samples interpolated between the source and target domain confusion features, Express The gradient of

[0030] The domain confusion loss is formulated as follows:

[0031] ;

[0032] in, L c represents the domain confusion loss function, x s represents the source domain image features, G (·) represents the feature generator output, x t represents the target domain image features, λ ce represents the weight parameter of the conditional entropy regularization term, x represents the joint representation of source domain and target domain image features, p ( y | G ( x )) indicates a given feature G ( x ) under the target label y The conditional probability distribution of H (·) represents the conditional entropy output result.

[0033] In an optional embodiment,

[0034] The target domain image domain adaptation features are input into the deep reinforcement learning module, and the optimal classification strategy for the target domain battlefield target recognition task is learned through interactive exploration between the intelligent agent and the reinforcement learning environment including the state space, action space and reward function. The target domain image domain adaptation features are classified based on the optimal classification strategy, and the cross-domain recognition results of battlefield targets are output, including:

[0035] Construct a reinforcement learning environment, model the battlefield target cross-domain recognition task as a Markov decision process, define the state space, action space and reward function, the state space is the domain adaptation feature of the target domain image, each image corresponds to a state, the state dimension is equal to the domain adaptation feature dimension, the action space is the battlefield target category, each action corresponds to a target category, the reward function is determined based on the consistency between the agent's predicted category label and the true label. If the prediction is correct, a positive reward is given, otherwise a negative reward or zero reward is given;

[0036] Construct a deep reinforcement learning model, determine the corresponding policy network, value function network and target network, the policy network takes the target domain image domain adaptation feature as input, outputs the probability distribution of the corresponding action, selects the action by random sampling, the value function network takes the target domain image domain adaptation feature as input, outputs the estimated value function of the state, the target network is composed of corresponding copies of the policy network and the value function network, calculates the stable target value, and copies the current network parameters to the target network according to the preset period;

[0037] Start the iterative process, select actions based on the current decision-making strategy, interact with the reinforcement learning environment, obtain the next state and immediate reward, repeat the iterative process, and collect multiple rounds of interaction experience data; use the interaction experience data to optimize the policy network parameters through the policy gradient algorithm, update the decision strategy with the goal of maximizing the cumulative reward; at the same time, use the temporal difference algorithm to update the value function network parameters, and use the target network to calculate the value function target value;

[0038] Until the preset iteration rounds are reached, the optimal classification strategy is obtained;

[0039] The trained strategy network is used to classify the domain adaptation features of the target domain image, select the action with the highest probability as the predicted category label, and output the cross-domain recognition results of battlefield targets.

[0040] In an optional embodiment,

[0041] The policy network parameters are optimized through the policy gradient algorithm to maximize the cumulative reward. The updated decision strategy includes:

[0042] Use the policy gradient algorithm to update the policy network parameters. The formula is as follows:

[0043] ;

[0044] in, θ represents the policy network parameters, α represents the learning rate of the policy gradient algorithm, represents the gradient of the policy network parameters, E represents the expectation operator, a Indicates action, u Indicates the status, Indicates that the current policy network is in a given state u Next select action a The probability of Indicates that the previous policy network is in a given state u Next select action a The probability of A represents the advantage function, clip(·) represents the clipping function, Indicates the clipping range threshold.

[0045] In an optional embodiment,

[0046] Using the temporal difference algorithm to update the value function network parameters and using the target network to calculate the target value of the value function includes:

[0047] Use the temporal difference algorithm to update the value function network parameters. The formula is as follows:

[0048] ;

[0049] in, φ represents the network parameters of the value function, β represents the learning rate in the temporal difference algorithm, represents the gradient of the network parameters of the value function, N represents the total number of samples, n Indicates the sample number, M represents the maximum time step of the complete sequence of each sample, m represents a time step in the complete sequence, Indicates n Samples in m The instant reward at the time step, γ represents the discount factor, Indicates n Samples in m+ The state of 1 time step, represents the target network value function, Indicates n Samples in m The state of the time step, Represents the current network value function.

[0050] According to a second aspect of the embodiments of the present invention,

[0051] A battlefield target cross-domain recognition system based on deep reinforcement learning is provided, including:

[0052] The first unit is used to obtain a labeled source domain target image and an unlabeled target domain target image based on a source domain battlefield environment and a target domain battlefield environment, respectively, and extract source domain image features and target domain image features through a deep convolutional neural network model;

[0053] The second unit is used to map the source domain image features and the target domain image features to the same feature space through adversarial training based on an adversarial domain adaptation network, learn common feature representation, and obtain the target domain image domain adaptation features by reducing distribution differences;

[0054] The third unit is used to input the target domain image domain adaptation features into a deep reinforcement learning module, learn the optimal classification strategy for the target domain battlefield target recognition task through interactive exploration between the intelligent agent and the reinforcement learning environment containing state space, action space and reward function, classify the target domain image domain adaptation features based on the optimal classification strategy, and output the cross-domain recognition results of battlefield targets.

[0055] According to a third aspect of the embodiments of the present invention,

[0056] An electronic device is provided, comprising:

[0057] processor;

[0058] a memory for storing processor-executable instructions;

[0059] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0060] A fourth aspect of the embodiments of the present invention is:

[0061] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0062] In an embodiment of the present invention, a deformable convolution layer is added so that the network can adaptively adjust the sampling position of the convolution operation, enhance the network's adaptability to target deformation and position change, and improve the effect of feature extraction; the additional convolution layer is used to learn the position offset of the convolution kernel, so that the convolution operation can be adaptively adjusted according to the input data, so that the network can better capture the spatial deformation characteristics of the target and improve the quality of feature representation; through the feature mapper and the domain confusion feature propagation module, the image features of the source domain and the target domain are mapped to a common feature space.

[0063] This mapping capability helps to reduce the feature distribution differences between the source domain and the target domain and improve the effect of domain adaptation; through multiple iterations of the domain confusion feature propagation module, the confusion features of the source domain and the target domain are effectively fused, so that the final generated target domain confusion features can better represent the data features of the target domain, which helps to improve the representation ability and classification accuracy of the target domain data; using the domain adaptation features of the target domain image as the state representation can effectively reduce the distribution differences between the source domain and the target domain, help improve the reinforcement learning model's understanding and response capabilities to the target domain, and improve the performance of the task; modeling the battlefield target cross-domain recognition task as a Markov decision process, clarifying the definitions of the state space, action space and reward function, and the modeling method can effectively guide the intelligent agent to learn the optimal action strategy in a complex environment, thereby improving the execution efficiency and accuracy of the task. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a process flow of a battlefield target cross-domain recognition method based on deep reinforcement learning according to an embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of the structure of a battlefield target cross-domain recognition system based on deep reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0067] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0068] The technical solution of the present invention is described in detail below with specific embodiments.

[0069] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0070] Figure 1 FIG. 1 is a flow chart of a battlefield target cross-domain recognition method based on deep reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0071] S101. Based on the source domain battlefield environment and the target domain battlefield environment, respectively obtain a labeled source domain target image and an unlabeled target domain target image, and extract the source domain image features and the target domain image features through a deep convolutional neural network model;

[0072] The source domain specifically refers to the domain where the training data is located, which usually contains a large amount of labeled sample data. In a battlefield environment, the source domain contains target images collected and labeled in a known battlefield environment. These images and their corresponding labels are used to train the model so that it learns the characteristics and patterns of a specific task.

[0073] The target domain specifically refers to the domain to which the model will be applied, which usually lacks or has no labeled data at all.

[0074] In a battlefield setting, the target domain contains target images collected in unknown or new battlefield environments.

[0075] These images are not labeled, so the model needs to have the ability to process and recognize objects under unsupervised conditions.

[0076] In this embodiment, by extracting image features from the source domain and the target domain through a deep convolutional neural network, the source domain knowledge can be transferred to the target domain, thereby enhancing the adaptability of the model in different battlefield environments. By utilizing unlabeled target domain images and through domain adaptation technology, the dependence on labeled data is reduced, so that the model can still maintain good performance in an environment where data is scarce.

[0077] In an optional embodiment, the deep convolutional neural network model includes:

[0078] Using ResNet-50 as the backbone network, a deep deformable convolutional neural network is constructed. The position offset is learned by setting an additional convolutional layer at the initial position, and the convolutional layers in the third, fourth, and fifth residual blocks of ResNet-50 are replaced with deformable convolutional layers. The position of the convolution sampling is adaptively adjusted to obtain the original feature map.

[0079] After each of the deformable convolutional layers, a spatial attention module and a channel attention module are connected in parallel, wherein the spatial attention module learns the spatial importance of each position of the original feature map through global average pooling and a fully connected layer to obtain a spatial attention weight; the channel attention module learns the channel importance of different feature channels through global average pooling, a fully connected layer and a sigmoid activation function to obtain a channel attention weight, and the spatial attention weight and the channel attention weight are multiplied with the original feature map to obtain a weighted feature map;

[0080] After the third, fourth, and fifth residual blocks of the deep deformable convolutional neural network, a multi-scale feature map of the corresponding level is extracted based on the weighted feature map, and the size of the multi-scale feature map is adjusted by adaptive upsampling or adaptive downsampling to keep the size consistent. Multi-scale feature fusion is performed by element-by-element addition to obtain image features.

[0081] ResNet-50 is selected as the backbone network as the basis of deep deformable convolutional neural network.

[0082] An additional convolutional layer is set at the initial position of ResNet-50 to learn the position offset of the convolution kernel. By adding the convolutional layer, the network can adaptively adjust the sampling position of the convolution operation.

[0083] The ordinary convolutional layers in the third, fourth, and fifth residual blocks of ResNet-50 are replaced with deformable convolutional layers.

[0084] The deformable convolution layer adaptively adjusts the position of the convolution sampling according to the learned position offset, so that the network can better adapt to the deformation and position changes of the target.

[0085] In the replaced residual block, the convolution operation becomes a deformable convolution, and the other parts remain unchanged.

[0086] After each deformable convolutional layer, the spatial attention module and channel attention module are introduced in parallel to strengthen the network's attention to important features.

[0087] In the spatial attention module, the feature map of each channel is converted into a numerical value by performing a global average pooling operation on the feature map.

[0088] Then, a fully connected layer is used to process these values, learn the spatial importance of each position in the feature map, and obtain the spatial attention weight.

[0089] The spatial attention weights reflect the importance of different locations in the feature map.

[0090] In the channel attention module, the feature map of each channel is converted into a numerical value by performing a global average pooling operation on the feature map.

[0091] Then, these values ​​are processed using a fully connected layer and a sigmoid activation function to learn the channel importance of different feature channels and obtain the channel attention weights.

[0092] The channel attention weights reflect the importance of different feature channels.

[0093] Multiply the obtained spatial attention weights and channel attention weights with the corresponding original feature maps to obtain the weighted feature maps.

[0094] This weighted operation can make the network pay more attention to important feature positions and channels, and improve the quality of feature representation.

[0095] After the third, fourth, and fifth residual blocks of the deep deformable convolutional neural network, multi-scale feature maps of the corresponding levels are extracted respectively.

[0096] Multi-scale feature maps contain target information at different scales, which helps to improve the robustness and generalization ability of the network.

[0097] For the extracted multi-scale feature maps, adaptive upsampling or adaptive downsampling is used to adjust their sizes to keep them consistent.

[0098] Adaptive upsampling increases the size of feature maps by interpolation, and adaptive downsampling reduces the size of feature maps by pooling operations, ultimately making feature maps of different scales have the same size.

[0099] The resized multi-scale feature maps are added element by element to achieve multi-scale feature fusion.

[0100] Multi-scale feature fusion can comprehensively utilize target information at different scales and improve the representation ability of the network.

[0101] Through multi-scale feature fusion, the final image feature representation is obtained as the output of the deep deformable convolutional neural network.

[0102] This image feature contains the spatial information, scale information and channel information of the target, which can be used for subsequent target detection, classification or recognition tasks.

[0103] In this embodiment, a deformable convolution layer is added so that the network can adaptively adjust the sampling position of the convolution operation, enhance the network's adaptability to target deformation and position changes, and improve the effect of feature extraction; the additional convolution layer is used to learn the position offset of the convolution kernel, so that the convolution operation can be adaptively adjusted according to the input data, so that the network can better capture the spatial deformation characteristics of the target and improve the quality of feature representation; by extracting multi-scale feature maps in different residual blocks and adaptively upsampling or downsampling them, feature maps of different scales can maintain a consistent size, and the target information at different scales is integrated to improve the robustness and generalization ability of the network; the spatial attention module learns the importance of each position of the feature map, and the channel attention module learns the importance of different feature channels, so that the network can pay more attention to important feature positions and channels, thereby further improving the quality of feature representation.

[0104] S102. Based on an adversarial domain adaptation network, the source domain image features and the target domain image features are mapped to the same feature space through adversarial training, common feature representation is learned, and domain adaptation features of the target domain image are obtained by reducing distribution differences;

[0105] In this embodiment, by mapping to the same feature space, the feature distributions of the source domain and the target domain are made closer, and the distribution differences between the domains are reduced; by learning a common feature representation, the representations of the images of the source domain and the target domain in the feature space are made more consistent, and the consistency of the features is improved, which helps to obtain better performance in the target domain; by reducing the distribution differences between the source domain and the target domain, the performance of the model in cross-domain tasks is improved, especially in the target domain, and classification and recognition tasks can be performed more accurately.

[0106] In an optional embodiment, based on an adversarial domain adaptation network, the source domain image features and the target domain image features are mapped to the same feature space through adversarial training, and common feature representations are learned. By reducing distribution differences, domain adaptation features of the target domain image are obtained, including:

[0107] The adversarial domain adaptation network receives source domain image features and target domain image features through a pre-built feature mapper, and maps them to a common feature space respectively, to obtain source domain common space features and target domain common space features;

[0108] The source domain common space features and the target domain common space features are received through a pre-built domain confusion feature propagation module, and the source domain common space features and the target domain common space features are respectively input into two independent fully connected layers to obtain the source domain initial confusion features and the target domain initial confusion features, and propagate alternately between the source domain and the target domain. After multiple iterations, the final source domain confusion features and the final target domain confusion features are obtained, wherein in each iteration, the source domain confusion features and the target domain confusion features corresponding to each iteration are obtained by weighted summation and activation function;

[0109] The source domain confusion feature and the target domain confusion feature constitute a confusion feature, which is input into a pre-built domain discriminator to determine whether the confusion feature is a source domain confusion feature or a target domain confusion feature, and iteratively optimize the domain discriminator by maximizing the domain discrimination loss;

[0110] By using a pre-built feature classifier, receiving source domain confusion features, annotating and predicting the source domain image features, and iteratively optimizing the feature classifier by minimizing the classification loss in combination with the real annotations corresponding to the source domain image features;

[0111] Construct a domain confusion loss, iteratively optimize the feature generator by minimizing the source domain discriminant loss of the source domain confusion feature in the domain discriminator and the target domain discriminant loss of the target domain confusion feature in the domain discriminator;

[0112] Jointly optimizing the feature generator, the domain confusion feature propagation module and the domain discriminator to collaboratively learn the domain confusion feature representation; fixing the parameters of the feature generator and the domain confusion feature propagation module, and training the feature classifier based on the source domain confusion feature; iterating alternately until the preset convergence condition is met, and obtaining the trained feature generator and the domain confusion feature propagation module;

[0113] The target domain image features are input into the trained feature generator to obtain the mapped target domain features. After multiple iterations of the domain confusion feature propagation module, the target domain confusion features are obtained to determine the domain adaptation features of the target domain image.

[0114] A feature mapper is pre-built to map source domain image features and target domain image features into a common feature space; source domain image features are input into the feature mapper, and after a series of convolutions, poolings and non-linear transformations, mapped source domain common space features are obtained; target domain image features are input into the feature mapper, and after the same convolutions, poolings and non-linear transformations, mapped target domain common space features are obtained.

[0115] A domain confusion feature propagation module is pre-built to generate domain confusion features; source domain common space features are input into a fully connected layer in the domain confusion feature propagation module, and after non-linear transformation, the source domain initial confusion features are obtained; target domain common space features are input into another fully connected layer in the domain confusion feature propagation module, and after non-linear transformation, the target domain initial confusion features are obtained; in the domain confusion feature propagation module, the source domain initial confusion features and the target domain initial confusion features are propagated iteratively for multiple times; in each iteration, the source domain confusion features and the target domain confusion features obtained in the previous iteration are weightedly summed with the source domain initial confusion features and the target domain initial confusion features, respectively, and then the source domain confusion features and the target domain confusion features corresponding to the current iteration are obtained after an activation function; after multiple iterations, the final source domain confusion features and the target domain confusion features are obtained.

[0116] A domain discriminator is pre-built to determine whether the input confusion features come from the source domain or the target domain. The source domain confusion features and the target domain confusion features are input into the domain discriminator. Through a series of convolution, pooling, and fully connected layers, the domain discrimination results are obtained, and the domain discrimination loss is calculated, that is, the loss of the source domain confusion features being judged as the source domain in the domain discriminator and the loss of the target domain confusion features being judged as the target domain in the domain discriminator.

[0117] By maximizing the domain discrimination loss, the gradient ascent algorithm is used to iteratively optimize the parameters of the domain discriminator so that it can discriminate the domain attributes of the confusing features as accurately as possible.

[0118] A feature classifier is pre-built to predict the annotations of source domain image features.

[0119] The source domain confusion features are input into the feature classifier, and through a series of convolution, pooling and fully connected layers, the annotation prediction results of the source domain image are obtained. The classification loss, that is, the difference between the predicted annotation and the true annotation of the source domain image feature, is calculated. By minimizing the classification loss, the gradient descent algorithm is used to iteratively optimize the parameters of the feature classifier so that it can accurately predict the annotation of the source domain image.

[0120] Construct the domain confusion loss, that is, the loss of the source domain confusion features being judged as the source domain in the domain discriminator and the loss of the target domain confusion features being judged as the target domain in the domain discriminator; by minimizing the domain confusion loss, use the gradient descent algorithm to iteratively optimize the parameters of the feature generator (including the feature mapper and the domain confusion feature propagation module) so that it can generate domain-indistinguishable confusion features.

[0121] The feature generator, domain confusion feature propagation module and domain discriminator are jointly optimized to collaboratively learn the domain confusion feature representation through alternating training. The parameters of the feature generator and the domain confusion feature propagation module are fixed, and the feature classifier is trained using the source domain confusion features to improve its classification performance on the source domain data. The process is performed alternately until the preset convergence conditions are met, preferably until the maximum number of iterations is reached. If the loss function changes less than a certain threshold, the iteration can be terminated early to obtain the trained feature generator and domain confusion feature propagation module.

[0122] The target domain image features are input into the trained feature generator, and after passing through the feature mapper, the mapped target domain features are obtained. The mapped target domain features are input into the trained domain confusion feature propagation module, and after multiple iterative propagations, the final target domain confusion features are obtained. The target domain confusion features are used as domain adaptation features of the target domain image for subsequent target domain image classification, detection or other tasks.

[0123] In this embodiment, the image features of the source domain and the target domain are mapped into a common feature space through a feature mapper and a domain confusion feature propagation module.

[0124] This mapping capability helps to reduce the difference in feature distribution between the source domain and the target domain, and improves the effect of domain adaptation; through multiple iterations of the domain confusion feature propagation module, the confusion features of the source domain and the target domain are effectively fused, so that the finally generated target domain confusion features can better represent the data features of the target domain, which helps to improve the representation ability and classification accuracy of the target domain data; optimizing the training of the domain discriminator can more accurately judge the difference between the source domain and target domain features, which helps to enhance the learning effect of the feature generator and the confusion feature propagation module; after fixing the parameters of the feature generator and the confusion feature propagation module, the feature classifier is trained using the optimized source domain confusion features, and the domain adaptation technology is effectively used to improve the classification accuracy of the source domain data; the alternating training method can effectively improve the convergence and stability of the domain adaptation model, which helps to obtain more reliable and reproducible domain adaptation effects in practical applications.

[0125] In an optional embodiment, the domain discrimination loss is formulated as follows: ;

[0126] in, L d represents the domain discrimination loss function, E represents the expectation operator, f s represents the source domain confusion feature, p (·) represents the distribution of the corresponding feature, D (·) represents the output result of the domain discriminator, ft represents the target domain confusion feature, λ gp represents the weight of the gradient penalty term, represents samples interpolated between the source and target domain confusion features, Express The gradient of

[0127] The domain confusion loss is formulated as follows:

[0128] ;

[0129] in, L c represents the domain confusion loss function, x s represents the source domain image features, G (·) represents the feature generator output, x t represents the target domain image features, λ ce represents the weight parameter of the conditional entropy regularization term, x represents the joint representation of source domain and target domain image features, p ( y | G ( x )) indicates a given feature G ( x ) under the target label y The conditional probability distribution of H (·) represents the conditional entropy output result.

[0130] In the formula, the domain discrimination loss function is used to train the domain discriminator so that it can effectively distinguish the source domain confusion features from the target domain confusion features, and is composed of three parts: the first part measures the expected output of the domain discriminator for the source domain confusion features through the expectation operation; the second part also measures the expected output of the domain discriminator for the target domain confusion features through the expectation operation; the third part is the gradient penalty term, which is used to enhance the gradient regularization of the domain discriminator;

[0131] The domain confusion loss function is used to optimize the feature generator and the confusion feature propagation module to generate more domain-indistinguishable confusion features. It consists of three parts: the first part measures the expected output of the confusion features generated by the domain discriminator for the source domain image features through the expectation operation; the second part measures the expected output of the confusion features generated by the domain discriminator for the target domain image features through the expectation operation; the third part is the conditional entropy regularization term, which is used to maximize the conditional entropy generated by the target domain confusion features to enhance their distinguishability.

[0132] According to the formula, by minimizing the domain discrimination loss function, the domain discriminator is trained to effectively distinguish the confusion features of the source domain and the target domain, which helps to reduce the feature distribution difference between the source domain and the target domain and achieve domain adaptation; the gradient penalty term helps to improve the stability and convergence speed of training by forcing the gradient of the domain discriminator output to be close to 1; by minimizing the domain confusion loss function, the feature generator and the domain confusion feature propagation module are optimized to generate confusion features that are more similar and more domain-indistinguishable between the source domain and the target domain; the conditional entropy regularization term helps to improve the conditional entropy of the target domain confusion feature generation, enhancing its distinguishing ability and generalization ability in classification or other tasks.

[0133] S103. Input the target domain image domain adaptation features into the deep reinforcement learning module, and learn the optimal classification strategy for the target domain battlefield target recognition task through the interactive exploration of the intelligent agent and the reinforcement learning environment containing the state space, action space and reward function. Classify the target domain image domain adaptation features based on the optimal classification strategy, and output the cross-domain recognition results of battlefield targets.

[0134] The intelligent agent specifically refers to an entity that can perceive the environment, make decisions and perform actions in reinforcement learning.

[0135] It learns based on the state of the environment and optimizes its behavior through interaction with the environment to obtain the maximum cumulative reward. It learns the optimal classification strategy for the battlefield target recognition task in the target domain and uses reinforcement learning methods to achieve the classification and recognition task of the target domain image.

[0136] The interactive exploration specifically refers to the process in which the agent learns the best strategy through interaction with the environment in reinforcement learning, which is called interactive exploration.

[0137] The agent selects an action based on the current state at each time step, observes the reward from the environment after executing the action, and updates its strategy to optimize future decisions. The agent explores and learns the optimal classification strategy by interacting with the reinforcement learning environment that contains the state space (domain adaptation features of the target domain image), the action space (battlefield target categories), and the reward function to achieve the cross-domain recognition task of the target domain image.

[0138] In this embodiment, the optimal classification strategy for the target domain battlefield target recognition task is learned by interactive exploration between the intelligent agent and the reinforcement learning environment including the state space, action space and reward function.

[0139] Reinforcement learning enables the intelligent agent to gradually optimize its behavior strategy according to the reward signal of the environment to achieve accurate target recognition in a complex battlefield environment; based on the learned optimal classification strategy, the domain adaptation features of the target domain image that has been domain adapted are classified, which not only improves the classification accuracy of the target domain image, but also enhances the model's ability to recognize various battlefield targets in the target domain.

[0140] In an optional embodiment, the target domain image domain adaptation features are input into a deep reinforcement learning module, and the optimal classification strategy for the target domain battlefield target recognition task is learned through interactive exploration between the intelligent agent and the reinforcement learning environment including the state space, action space and reward function, and the target domain image domain adaptation features are classified based on the optimal classification strategy, and the battlefield target cross-domain recognition results are outputted, including:

[0141] Construct a reinforcement learning environment, model the battlefield target cross-domain recognition task as a Markov decision process, define the state space, action space and reward function, the state space is the domain adaptation feature of the target domain image, each image corresponds to a state, the state dimension is equal to the domain adaptation feature dimension, the action space is the battlefield target category, each action corresponds to a target category, the reward function is determined based on the consistency between the agent's predicted category label and the true label. If the prediction is correct, a positive reward is given, otherwise a negative reward or zero reward is given;

[0142] Construct a deep reinforcement learning model, determine the corresponding policy network, value function network and target network, the policy network takes the target domain image domain adaptation feature as input, outputs the probability distribution of the corresponding action, selects the action by random sampling, the value function network takes the target domain image domain adaptation feature as input, outputs the estimated value function of the state, the target network is composed of corresponding copies of the policy network and the value function network, calculates the stable target value, and copies the current network parameters to the target network according to the preset period;

[0143] Start the iterative process, select actions based on the current decision-making strategy, interact with the reinforcement learning environment, obtain the next state and immediate reward, repeat the iterative process, and collect multiple rounds of interaction experience data; use the interaction experience data to optimize the policy network parameters through the policy gradient algorithm, update the decision strategy with the goal of maximizing the cumulative reward; at the same time, use the temporal difference algorithm to update the value function network parameters, and use the target network to calculate the value function target value;

[0144] Until the preset iteration rounds are reached, the optimal classification strategy is obtained;

[0145] The trained strategy network is used to classify the domain adaptation features of the target domain image, select the action with the highest probability as the predicted category label, and output the cross-domain recognition results of battlefield targets.

[0146] Construct a reinforcement learning environment and model the battlefield target cross-domain recognition task as a Markov decision process; define the state space and use the domain adaptation features of the target domain image as the state representation.

[0147] Each target domain image corresponds to a state, and the dimension of the state is equal to the dimension of the domain adaptation feature; the action space is defined, and the category of the battlefield target is used as the action representation.

[0148] Each action corresponds to a target category, and the agent predicts the category label of the target domain image by selecting actions; a reward function is defined to determine the reward value based on the consistency between the category label predicted by the agent and the true label.

[0149] If the prediction is correct, a positive reward is given; if the prediction is wrong, a negative reward or zero reward is given.

[0150] Build a deep reinforcement learning model, including policy network, value function network and target network.

[0151] Design a policy network that takes the domain adaptation features of the target domain image as input and outputs the probability distribution of the corresponding action (target category) through a series of convolution, pooling, and fully connected layers; select the action to be performed by randomly sampling the output probability distribution of the policy network;

[0152] A value function network is designed, which takes the domain adaptation features of the target domain image as input, and outputs the estimated value function of the state through a series of convolution, pooling and fully connected layers, which represents the long-term cumulative reward estimate of the current state.

[0153] Construct a target network, including a copy of the policy network and a copy of the value function network, for calculating a stable target value; according to a preset period, preferably at a certain number of steps, copy the parameters of the current policy network and the value function network to the corresponding target network to maintain the relative stability of the target network.

[0154] The iterative process of reinforcement learning begins, and the agent interacts with the reinforcement learning environment.

[0155] Based on the current decision-making policy (represented by the policy network), the agent chooses an action to perform (predicted target category).

[0156] The selected action is applied to the reinforcement learning environment to obtain the next state (domain-adapted features of the next target domain image) and the immediate reward (reward value given according to the correctness of the prediction).

[0157] Repeat the execution to collect multiple rounds of interaction experience data, including the current state, the selected action, the reward obtained, and the next state.

[0158] Using the collected interactive experience data, the parameters of the policy network are optimized through the policy gradient algorithm.

[0159] The policy gradient algorithm aims to maximize the cumulative reward. It updates the parameters of the policy network by gradient ascent, so that the agent tends to choose actions that can obtain higher cumulative rewards. At the same time, the temporal difference algorithm is used to update the parameters of the value function network.

[0160] The temporal difference algorithm optimizes the parameters of the value function network by minimizing the difference between the current value function estimate and the target value calculated based on the target network, enabling it to more accurately estimate the long-term cumulative reward of the state.

[0161] Repeat the process until the preset number of iterations is reached to obtain a trained strategy network, that is, the optimal classification strategy.

[0162] The trained policy network is used to classify the domain-adapted features of target domain images.

[0163] The domain adaptation features of the target domain image are input into the policy network to obtain the probability distribution of the corresponding action (target category); the action with the highest probability is selected as the predicted category label, and the result of cross-domain recognition of battlefield targets is output.

[0164] The corresponding probabilities are repeatedly generated and the one with the largest probability is selected to classify all target domain images to obtain a complete cross-domain recognition result of battlefield targets.

[0165] In this embodiment, the domain adaptation features of the target domain image are used as state representations, which can effectively reduce the distribution differences between the source domain and the target domain, help improve the understanding and response capabilities of the reinforcement learning model to the target domain, and improve the performance of the task; the battlefield target cross-domain recognition task is modeled as a Markov decision process, and the definitions of the state space, action space and reward function are clarified. The modeling method can effectively guide the intelligent agent to learn the optimal action strategy in a complex environment, thereby improving the execution efficiency and accuracy of the task; a policy network and a value function network are designed, and optimized by a policy gradient algorithm and a temporal difference algorithm. The policy network adjusts the action selection strategy by maximizing the cumulative reward, and the value function network improves the accuracy of long-term decisions by optimizing the estimation of the state value, which can enable the intelligent agent to learn and improve its behavior strategy more effectively; the intelligent agent interacts with the reinforcement learning environment and adjusts its behavior through the real-time feedback reward value. This interactive learning process enables the model to improve through continuous trial and error, and gradually optimizes its understanding and classification capabilities of the target domain features.

[0166] In an optional embodiment, the policy network parameters are optimized by a policy gradient algorithm to maximize the cumulative reward, and the decision-making strategy is updated including:

[0167] Use the policy gradient algorithm to update the policy network parameters. The formula is as follows:

[0168] ;

[0169] in, θ represents the policy network parameters, α represents the learning rate of the policy gradient algorithm, represents the gradient of the policy network parameters, E represents the expectation operator, a Indicates action, u Indicates the status, Indicates that the current policy network is in a given state u Next select action a The probability of Indicates that the previous policy network is in a given state u Next select action a The probability of A represents the advantage function, clip(·) represents the clipping function, Indicates the clipping range threshold.

[0170] According to the formula, the policy gradient algorithm is used when updating the policy network parameters.

[0171] The goal of the algorithm is to maximize the expected cumulative reward.

[0172] Specifically, the difference between the new and old policies is evaluated by calculating the probability ratio of the action selected by the policy network in a given state.

[0173] The probability ratio is used to weigh an advantage function that measures the advantage of choosing that action relative to the average expected reward.

[0174] In order to stabilize the training process, the clip function is used to limit the range of the ratio to avoid excessive update steps.

[0175] Based on the formula, the policy network can be effectively guided to adjust in the direction of higher cumulative rewards, thereby improving the decision-making strategy and behavior effects of the intelligent agent.

[0176] In an optional embodiment, updating the value function network parameters using a temporal difference algorithm and using a target network to calculate the value function target value includes:

[0177] Use the temporal difference algorithm to update the value function network parameters. The formula is as follows:

[0178] ;

[0179] in, φ represents the network parameters of the value function, βrepresents the learning rate in the temporal difference algorithm, represents the gradient of the network parameters of the value function, N represents the total number of samples, n Indicates the sample number, M represents the maximum time step of the complete sequence of each sample, m represents a time step in the complete sequence, Indicates n Samples in m The instant reward at the time step, γ represents the discount factor, Indicates n Samples in m+ The state of 1 time step, represents the target network value function, Indicates n Samples in m The state of the time step, Represents the current network value function.

[0180] According to the formula, the value function network parameters are updated through a temporal difference algorithm.

[0181] The goal of the algorithm is to minimize the expected mean squared error loss, which is the square of the difference between the current network estimate of the value function and the target network estimate of the value function.

[0182] Specifically, for each sample, the algorithm calculates the immediate reward for each time step in the complete sequence and combines the discount factor, as well as the target network value function and the current network value function to calculate the square of the error.

[0183] The value function network parameters are then adjusted by performing a gradient descent update on the average error of all samples.

[0184] Based on the formula, the value function network can be effectively optimized so that it can more accurately estimate the long-term cumulative reward of the state and improve the performance and learning effect of the intelligent agent in the decision-making process.

[0185] Figure 2 FIG. 1 is a schematic diagram of the structure of a battlefield target cross-domain recognition system based on deep reinforcement learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0186] The first unit is used to obtain a labeled source domain target image and an unlabeled target domain target image based on a source domain battlefield environment and a target domain battlefield environment, respectively, and extract source domain image features and target domain image features through a deep convolutional neural network model;

[0187] The second unit is used to map the source domain image features and the target domain image features to the same feature space through adversarial training based on an adversarial domain adaptation network, learn common feature representation, and obtain the target domain image domain adaptation features by reducing distribution differences;

[0188] The third unit is used to input the target domain image domain adaptation features into a deep reinforcement learning module, learn the optimal classification strategy for the target domain battlefield target recognition task through interactive exploration between the intelligent agent and the reinforcement learning environment containing state space, action space and reward function, classify the target domain image domain adaptation features based on the optimal classification strategy, and output the cross-domain recognition results of battlefield targets.

[0189] According to a third aspect of the embodiments of the present invention,

[0190] An electronic device is provided, comprising:

[0191] processor;

[0192] a memory for storing processor-executable instructions;

[0193] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0194] A fourth aspect of the embodiments of the present invention is:

[0195] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0196] The present invention may be a method, an apparatus, a system and / or a computer program product.

[0197] The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for performing various aspects of the present invention.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A battlefield target cross-domain recognition method based on deep reinforcement learning, characterized in that: include: Based on the source domain battlefield environment and the target domain battlefield environment, labeled source domain target images and unlabeled target domain target images are obtained respectively, and the source domain image features and the target domain image features are extracted through a deep convolutional neural network model; Based on an adversarial domain adaptation network, the source domain image features and the target domain image features are mapped to the same feature space through adversarial training, common feature representation is learned, and domain adaptation features of the target domain image are obtained by reducing distribution differences; The target domain image domain adaptation features are input into a deep reinforcement learning module, and the optimal classification strategy for the target domain battlefield target recognition task is learned through interactive exploration between the intelligent agent and the reinforcement learning environment including the state space, the action space and the reward function, and the target domain image domain adaptation features are classified based on the optimal classification strategy, and the cross-domain recognition result of the battlefield target is output; Based on the adversarial domain adaptation network, the source domain image features and the target domain image features are mapped to the same feature space through adversarial training to learn The common feature representation, by reducing the distribution difference, obtains the domain adaptation features of the target domain image, including: The adversarial domain adaptation network receives source domain image features and target domain image features through a pre-built feature mapper, and maps them to a common feature space respectively, to obtain source domain common space features and target domain common space features; The source domain common space features and the target domain common space features are received through a pre-built domain confusion feature propagation module, and the source domain common space features and the target domain common space features are respectively input into two independent fully connected layers to obtain the source domain initial confusion features and the target domain initial confusion features, and propagate alternately between the source domain and the target domain. After multiple iterations, the final source domain confusion features and the final target domain confusion features are obtained, wherein in each iteration, the source domain confusion features and the target domain confusion features corresponding to each iteration are obtained by weighted summation and activation function; The source domain confusion feature and the target domain confusion feature constitute a confusion feature, which is input into a pre-built domain discriminator to determine whether the confusion feature is a source domain confusion feature or a target domain confusion feature, and iteratively optimize the domain discriminator by maximizing the domain discrimination loss; By using a pre-built feature classifier, receiving source domain confusion features, annotating and predicting the source domain image features, and iteratively optimizing the feature classifier by minimizing the classification loss in combination with the real annotations corresponding to the source domain image features; Construct a domain confusion loss, iteratively optimize the feature generator by minimizing the source domain discriminant loss of the source domain confusion feature in the domain discriminator and the target domain discriminant loss of the target domain confusion feature in the domain discriminator; Jointly optimizing the feature generator, the domain confusion feature propagation module and the domain discriminator to collaboratively learn the domain confusion feature representation; fixing the parameters of the feature generator and the domain confusion feature propagation module, and training the feature classifier based on the source domain confusion feature; iterating alternately until the preset convergence condition is met, and obtaining the trained feature generator and the domain confusion feature propagation module; The target domain image features are input into the trained feature generator to obtain the mapped target domain features. After multiple iterations of the domain confusion feature propagation module, the target domain confusion features are obtained to determine the domain adaptation features of the target domain image. The target domain image domain adaptation features are input into the deep reinforcement learning module, and the optimal classification strategy for the target domain battlefield target recognition task is learned through interactive exploration between the intelligent agent and the reinforcement learning environment including the state space, action space and reward function. The target domain image domain adaptation features are classified based on the optimal classification strategy, and the cross-domain recognition results of battlefield targets are output, including: Construct a reinforcement learning environment, model the battlefield target cross-domain recognition task as a Markov decision process, define the state space, action space and reward function, the state space is the domain adaptation feature of the target domain image, each image corresponds to a state, the state dimension is equal to the domain adaptation feature dimension, the action space is the battlefield target category, each action corresponds to a target category, the reward function is determined based on the consistency between the agent's predicted category label and the true label. If the prediction is correct, a positive reward is given, otherwise a negative reward or zero reward is given; Construct a deep reinforcement learning model, determine the corresponding policy network, value function network and target network, the policy network takes the target domain image domain adaptation feature as input, outputs the probability distribution of the corresponding action, selects the action by random sampling, the value function network takes the target domain image domain adaptation feature as input, outputs the estimated value function of the state, the target network is composed of corresponding copies of the policy network and the value function network, calculates the stable target value, and copies the current network parameters to the target network according to the preset period; Start the iterative process, select actions based on the current decision-making strategy, interact with the reinforcement learning environment, obtain the next state and immediate reward, repeat the iterative process, and collect multiple rounds of interaction experience data; use the interaction experience data to optimize the policy network parameters through the policy gradient algorithm, update the decision strategy with the goal of maximizing the cumulative reward; at the same time, use the temporal difference algorithm to update the value function network parameters, and use the target network to calculate the value function target value; Until the preset iteration rounds are reached, the optimal classification strategy is obtained; The trained strategy network is used to classify the domain adaptation features of the target domain image, select the action with the highest probability as the predicted category label, and output the cross-domain recognition results of battlefield targets.

2. The method according to claim 1, characterized in that The deep convolutional neural network model includes: Using ResNet-50 as the backbone network, a deep deformable convolutional neural network is constructed. The position offset is learned by setting an additional convolutional layer at the initial position, and the convolutional layers in the third, fourth, and fifth residual blocks of ResNet-50 are replaced with deformable convolutional layers. The position of the convolution sampling is adaptively adjusted to obtain the original feature map. After each of the deformable convolutional layers, a spatial attention module and a channel attention module are connected in parallel, wherein the spatial attention module learns the spatial importance of each position of the original feature map through global average pooling and a fully connected layer to obtain a spatial attention weight; the channel attention module learns the channel importance of different feature channels through global average pooling, a fully connected layer and a sigmoid activation function to obtain a channel attention weight, and the spatial attention weight and the channel attention weight are multiplied with the original feature map to obtain a weighted feature map; After the third, fourth, and fifth residual blocks of the deep deformable convolutional neural network, a multi-scale feature map of the corresponding level is extracted based on the weighted feature map, and the size of the multi-scale feature map is adjusted by adaptive upsampling or adaptive downsampling to keep the size consistent. Multi-scale feature fusion is performed by element-by-element addition to obtain image features.

3. The method according to claim 1, characterized in that The domain discrimination loss is formulated as follows: ; in, represents the domain discrimination loss function, represents the expectation operator, represents the source domain confusion feature, represents the distribution of the corresponding feature, represents the output result of the domain discriminator, represents the target domain confusion feature, represents the weight of the gradient penalty term, represents samples interpolated between the source and target domain confusion features, Express The gradient of The domain confusion loss is formulated as follows: ;in, represents the domain confusion loss function, represents the source domain image features, represents the feature generator output, represents the target domain image features, represents the weight parameter of the conditional entropy regularization term, represents the joint representation of source domain and target domain image features, Represents a given feature Next target label The conditional probability distribution of Represents the conditional entropy output result.

4. The method according to claim 1, characterized in that: The policy network parameters are optimized through the policy gradient algorithm to maximize the cumulative reward. The updated decision strategy includes: Use the policy gradient algorithm to update the policy network parameters. The formula is as follows: ; in, represents the policy network parameters, represents the learning rate of the policy gradient algorithm, represents the gradient of the policy network parameters, E represents the expectation operator, Indicates action, Indicates the status, Indicates that the current policy network is in a given state Next select action The probability of Indicates that the previous policy network is in a given state Next select action The probability of represents the advantage function, represents the shear function, Indicates the clipping range threshold.

5. The method according to claim 4, characterized in that Using the temporal difference algorithm to update the value function network parameters and using the target network to calculate the value function target value includes: Use the temporal difference algorithm to update the value function network parameters. The formula is as follows: ;in, represents the network parameters of the value function, represents the learning rate in the temporal difference algorithm, represents the gradient of the network parameters of the value function, represents the total number of samples, Indicates the sample number, represents the maximum time step of the complete sequence of each sample, represents a time step in the complete sequence, Indicates Samples in The immediate reward at the time step, represents the discount factor, Indicates Samples in The state of the time step, represents the target network value function, Indicates Samples in The state of the time step, Represents the current network value function.

6. A battlefield target cross-domain identification system based on deep reinforcement learning, used to implement the method described in any one of claims 1 to 5, characterized in that: include: The first unit is used to obtain a labeled source domain target image and an unlabeled target domain target image based on a source domain battlefield environment and a target domain battlefield environment, respectively, and extract source domain image features and target domain image features through a deep convolutional neural network model; The second unit is used to map the source domain image features and the target domain image features to the same feature space through adversarial training based on an adversarial domain adaptation network, learn common feature representation, and obtain the target domain image domain adaptation features by reducing distribution differences; The third unit is used to input the target domain image domain adaptation features into a deep reinforcement learning module, learn the optimal classification strategy for the target domain battlefield target recognition task through interactive exploration between the intelligent agent and the reinforcement learning environment containing state space, action space and reward function, classify the target domain image domain adaptation features based on the optimal classification strategy, and output the cross-domain recognition results of battlefield targets.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images