Unmanned aerial vehicle ground target active detection method, terminal device and storage medium

Abstract

The invention discloses an unmanned aerial vehicle ground target active detection method, a terminal device and a storage medium, and aims to improve the state representation learned by the backbone part of a strategy network by using inductive bias, thereby improving the effectiveness and generalization of the strategy learned by an intelligent agent in solving the active target detection problem. Firstly, image features which are mainly concerned by an active target detection problem should concentrate the appearance and posture of a target, the position relation between the target and surrounding ground features and other features, and the color, texture and other features of the ground features can be discarded, so that scene pre-decomposition operation is carried out on the image features before the image features are extracted; and the irrelevant features are filtered out. Secondly, the characteristics of the historical observation sequence are fused by using the GRU network; through the improvement in the two aspects, the generalization ability of state characterization can be further improved, so that an intelligent agent is helped to learn a better active target detection strategy.

Description

Technical Field

[0001] The present invention relates to target detection technology, and in particular to a method for actively detecting ground targets by unmanned aerial vehicles, a terminal device and a storage medium. Background Art

[0002] In recent years, drones have shined in applications such as traffic monitoring, post-disaster rescue, smart agriculture, and military reconnaissance with their advantages of flexibility, wide field of view, rapid deployment, and low cost, showing extraordinary performance beyond traditional means. Behind these applications, target detection is an indispensable key technology. Target detection can locate and identify targets from aerial images, providing important prior information for subsequent actions. Therefore, enhancing target detection performance is of great significance to further improve the quality of drone service assurance.

[0003] Since the rise of deep learning, deep learning methods have gradually replaced traditional methods in the field of drone target detection and become the mainstream detection method. For example, Faster-RCNN, YOLO, SSD and other methods and their variants have been widely used in the field of remote sensing target detection. However, in drone aerial images, the effect of target detection will inevitably be affected by the interference of ground objects, resulting in reduced detection performance or even failure of target detection. In response to this difficult problem, most existing methods make different adaptive improvements to the detection model according to the characteristics of the problem. Although researchers have made many efforts to improve, the ability of detectors in the task of detecting occluded targets is still far behind that of humans, and is far from a satisfactory level. Fundamentally, the quality of input model data limits the upper limit of detection performance improvement, and it may be difficult to achieve performance breakthroughs by only adaptively improving the model. Relatively speaking, considering that the drone itself has strong maneuverability, if it can autonomously change the viewing angle of the observed target to obtain images that are more conducive to target identification, it will greatly improve the upper limit of target detection performance, that is, overcome the problem of target occlusion through active perception.

[0004] Active vision is a field created by humans based on the visual mechanisms of natural organisms. It studies how to make the visual system autonomously determine the movement of the camera according to task requirements and analysis results, and obtain the corresponding image from the appropriate perspective. Since it was proposed by Krotkov and others in the 1990s, active vision has been widely used in navigation and positioning, intelligent monitoring, object recognition, target tracking and other fields. Active Object Detection (AOD) is a subfield formed by active vision for target detection. Its research purpose is to target targets with certain position information but uncertain identity information. By giving the mobile sensor platform the ability to autonomously adjust the viewing angle, it enables it to obtain visual information that is more conducive to judging the target identity, thereby improving the target detection effect. Figure 1As shown in the figure, the UAV cannot determine the target identity information with certainty due to occlusion and other reasons in certain viewing angles. At this time, if the UAV wants to complete the AOD task, it needs to solve two problems: 1) how to find the best observation angle to reliably eliminate the uncertainty of the target identity information; 2) how to optimize the motion trajectory to minimize the motion cost. It is worth noting that AOD is different from target-driven visual navigation (TDVN). For the latter, the agent usually plans the view based on video data, while for the former, the agent uses image data. In addition, in the AOD task, the target needs to appear in the initial observation. On the contrary, in the initial stage of TDVN, the target may not be within the field of view of the agent, but it needs to appear in the final observation.

[0005] However, existing AOD research mainly focuses on indoor robot applications, and there is a lack of AOD research from the perspective of outdoor drones. In addition, from a methodological perspective, existing DRL-based AOD methods have two shortcomings. First, in the representative works mentioned above, the AOD problem is usually modeled as a Markov decision process (MDP), and then the target observation image from a single perspective is directly used as the state of the agent. However, since the observation image is only a projection of the three-dimensional real world, a single observation cannot fully contain all the information about the state of the agent. Therefore, the information extracted from a single observation is not enough to support the agent to learn the optimal active observation strategy. On the other hand, in the existing DRL-based AOD methods, the agent obtains rewards in the interaction with the environment, which is used as a supervisory signal to promote the continuous optimization of the agent's policy network. However, the reward value is a weak supervisory signal, which is difficult to help the agent learn the strategy efficiently. Especially when the training data is limited, the state representation learned by the backbone of the policy network may not capture the key features related to the task, resulting in poor generalization of the agent's strategy. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method for active detection of ground targets by unmanned aerial vehicles, a terminal device and a storage medium in view of the shortcomings of the existing technology, so as to improve the effectiveness and generalization of the strategy learned by the intelligent agent in solving the problem of active target detection.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for actively detecting ground targets by a UAV, comprising the following steps:

[0008] S1. Initialize the agent strategy network model π train , experience cache pool, round counter is reset to zero;

[0009] S2. Set t = 0 and initialize the drone position p t , get the corresponding drone observation image o t , initialize the hidden state h t ;

[0010] S3, observing the image o of the drone t The image to be processed is cropped to obtain the image to be processed, the scene is pre-decomposed for the image to be processed, and the feature extraction is performed on the image after the scene pre-decomposition, and the extracted features are compared with the hidden state h t After GRU fusion, the state s is obtained t , update the hidden state h t , if t>0, then the interaction experience (s t-1 ,a t-1 ,r t-1 ,s t ) is stored in the experience cache pool;

[0011] S4, if the amount of experience data in the experience buffer pool is greater than the threshold thre pool , then a batch of data is extracted from the experience buffer pool to the agent strategy network model π train Conduct training;

[0012] S5, based on state s t , the agent decision network gives the action output a at the current moment t , training environment feedback reward value r t , the drone is based on a t Move to new position p t+1 , get the new target observation image o t+1 ;

[0013] S6, the value of t increases by 1, if t reaches the maximum time step T max , then jump to step S2, otherwise jump to step S3. When the number of rounds is greater than the set constant, the trained agent strategy network model is obtained.

[0014] The method of the present invention further comprises:

[0015] S7. Use the trained intelligent agent strategy network model to perform target detection on the UAV observation image to obtain the target observation image.

[0016] In the present invention, the agent strategy network model adopts Dauling deep Q network.

[0017] In step S2 of the present invention, the scene is pre-decomposed for the image to be processed, and the specific implementation process of feature extraction for the image after the scene pre-decomposition includes: using the SAM model and the target wireframe as a prompt to segment the image to be processed, using a mask to cover the surrounding objects of the target, and extracting features from the image covered by the mask through an encoder; using the SAM model and the target wireframe as a prompt to segment the image to be processed, using a mask to cover the surrounding objects of the target, and extracting features from the image covered by the mask through an encoder; splicing the extracted feature vector with the vector representation of the target wireframe coordinates and the vector representation of the drone observation azimuth to obtain the features corresponding to the single-view observation, that is, to obtain the extracted features.

[0018] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.

[0019] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon; the computer program / instruction implements the steps of the above method when executed by a processor.

[0020] As an inventive concept, the present invention also provides a computer program product, including a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the above method are implemented.

[0021] Compared with the prior art, the beneficial effect of the present invention is that the present invention uses inductive bias to improve the state representation learned by the backbone of the strategy network, thereby improving the effectiveness and generalization of the strategy learned by the agent in solving the problem of active target detection. Specifically, the inductive bias includes two aspects, which are specifically reflected in the scene pre-decomposition in step S3 and the feature fusion operation based on the GRU network. First, the image features that the active target detection problem mainly focuses on should focus on the characteristics of the target's appearance, posture, and the positional relationship between the target and the surrounding objects, while the color, texture and other features of the objects can be discarded. Therefore, the present invention proposes to perform a scene pre-decomposition operation on the image before feature extraction, thereby filtering out these irrelevant features. Secondly, due to the partially observable characteristics, the state representation of the agent needs to be extracted from the historical observation sequence, rather than relying solely on observations from a single perspective. Therefore, the present invention proposes to use the GRU network to fuse the features of the historical observation sequence. Through these two improvements, the generalization ability of the state representation can be further improved, thereby helping the agent learn a better active target detection strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the motivation for active target detection by drones in a scene;

[0023] Figure 2 This is a forward reasoning flow chart of the method according to an embodiment of the present invention;

[0024] Figure 3 This is a visual comparison picture sequence of the detection results of the embodiment of the present invention;

[0025] Figure 4 Comparison of the evaluation results of the three AOD methods; (a) reward value change curve, (b) recognition rate change curve, (c) motion distance change curve. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Example 1

[0028] The process of UAV performing AOD (active target detection) tasks can be summarized as follows: the UAV, as an intelligent agent, makes decisions based on raw observations, finds the best observation angle with minimal motion and computational cost, and reliably eliminates the uncertainty of target identity information.

[0029] We model the AOD problem as a POMDP, usually with a seven-tuple express, represents the state set of the agent, corresponds to the useful features abstracted from the drone observation data, Represents a set of actions, Represents the actions that the agent can take, including action type a type and the range of motion a range Two parts, a type ∈{forward,backward,left,right,up,down,stop}, a range ∈{1,2,...,A}. Represents a set of observations. T(s'|s,a) represents the state transfer function, Ω(ο|s,a) is the observation function, which represents the probability that the agent is in state s and takes action a, and the environment feeds back observation o to the agent. The original observation o=(I,b), I represents the aerial image acquired by the drone, and b represents the wireframe information of the target (assumed to be obtained by the tracking algorithm or given manually). γ is the discount factor when calculating the return value. R(s,a) represents the reward function, and its design should take into account the target recognition result, the number of decisions, and the length of the motion trajectory. We define the reward function as:

[0030]

[0031] when a type = stop or t = T max -1, the variable flag = 1, indicating the end of a round, otherwise the agent needs to continue making decisions, reco = 1 means the target is correctly identified, and reco = 0 means the identification is wrong. ξ1, ξ2 are reward constants greater than 0, and σ is a coefficient used to control the effect of the action amplitude on the reward function. Its value depends on a type ,Right now:

[0032]

[0033] On this basis, the following optimization problem is solved by RL algorithm:

[0034]

[0035] Among them, π(a t |s t ) represents the agent strategy, Π represents the set of strategies, π* represents the optimal strategy, and ρ(s) represents the distribution of the initial state. r(s) represents a round (assuming that a round contains at most T max The expected return value within s as the initial state within time steps can be calculated by the following formula:

[0036]

[0037] To solve the optimal strategy, we first need to estimate the state from the observed data. Existing work usually only estimates the state based on the current observations. t To estimate the state s t , but due to the incomplete observation characteristics of a single perspective, the state obtained in this way is not enough to support the agent to learn the optimal strategy, because it is impossible to distinguish multiple states by relying on a single observation, that is, a single observation cannot uniquely determine a state. Instead, we adopt the observation sequence o1, o2, ..., o t The estimated state s tThis improves the accuracy of the agent’s state description and thus improves the performance of the learned strategy.

[0038] Since the training set and test set in the UEVAVD dataset contain multiple environment settings, the agent network needs to be trained and tested in multiple environment instances. In order to formally and quantitatively describe the zero-shot generalization (ZSG) performance of the agent on the UEVAVD dataset, we introduce the contextual Markovian decision process (CMDP) based on POMDP modeling. CMDP is a POMDP that introduces the concept of context, using tuples Compared with the general POMDP modeling representation In terms of context space, Context variables refers to the location of the target. In addition, we call is the hidden state space, which is different from the original state space The relationship is p(c) represents the distribution of context variables. At the same time, it is stipulated that within a round, the context variables of the agent's environment remain unchanged, that is, if c1≠c2, then T((s'2,c2)|(s1',c1),a)=0.

[0039] For the context set CMDP The expected return of strategy π on it is:

[0040]

[0041] in represents the CMDP under the given context c. Next, let They are respectively the training context set and the test context set. The agent The corresponding training environment set interacts and continuously updates the strategy until the expected return converges to the optimal result, and the final Right now:

[0042]

[0043] We expect Also in The corresponding test environment set performs well, even if As high as possible. In addition to observing We also use the indicator GenGap(π) to measure the zero-shot generalization ability of the strategy. For , the calculation formula is:

[0044]

[0045] The smaller the GenGap(π), the smaller the performance difference between the strategy in the training and test environments. In addition, we need to observe If under the same conditions, The higher the value of and the smaller the GenGap(π), the stronger the generalization performance of the strategy.

[0046] Based on the above problem modeling, Figure 2 The figure shows the flowchart of our proposed DRL-based AOD method. Compared with previous similar works, the innovation of this method is to use inductive bias to improve the state representation learned by the backbone of the policy network, thereby improving the effectiveness and generalization of the strategy learned by the agent in solving the AOD problem. The inductive bias here includes two aspects. First, the image features that the AOD problem mainly focuses on should focus on the characteristics of the target's appearance, posture, and the positional relationship between the target and the surrounding objects, while the color, texture and other features of the objects can be discarded. Second, due to the partially observable characteristics, the state representation of the agent needs to be extracted from the historical observation sequence, rather than relying solely on observations from a single perspective.

[0047] Figure 2 The upper middle part presents the abstract process of the entire AOD task at the macro level. The agent continuously optimizes the viewing angle of the observed target through multiple time-step decisions, and sends the target slice at the final ideal viewing angle to the classifier for classification. Each time step includes five steps: imaging, scene pre-decomposition, feature extraction, feature fusion, and decision-making. First, the action a before the drone is installed t Move to a new observation position, image the scene where the target is located and crop the image of the area around the target. Then, use the SAM model to segment the obtained image based on the target wireframe as a hint, and use a mask to cover the objects around the target. The grayscale value of the mask of each object is randomly distributed in the range of 0 to 1. In this way, in addition to the target itself, the observed image only contains the contour features of the surrounding objects and excludes their color, texture and other features. Then, the encoder extracts features from the simplified image, and splices the proposed features with the target wireframe coordinates, the drone observation azimuth and other information to obtain the feature g corresponding to the single-view observation. t To improve the sample efficiency of training, the encoder here is usually pre-trained. Then, we use GRU to fuse the feature vectors corresponding to previous observations in order to learn the state representation s containing only the features related to the AOD task from the historical observation sequence. t On this basis, we continue to use the s-based tThe decision-making method is to use the Dualing Deep Q Network (DualingDQN) to estimate the state value, and also let it output the advantages of various possible actions in two dimensions (action type and action amplitude), and obtain the value of each action combination by adding them up, and then select the optimal action a* as a t+1 Output. The specific steps are shown in the following table:

[0048] Table 1 Process of active target detection method based on reinforcement learning enhanced by inductive bias

[0049]

[0050]

[0051] The key to the intelligent agent performing the AOD task is to understand the three-dimensional scene where the target is located from the two-dimensional observation image, including the appearance, posture, and positional relationship between the target and the surrounding objects. However, the color, texture and other features of the objects around the target are not important for the completion of the AOD task. Therefore, in order to make the state representation more concise and compact and reduce the overfitting phenomenon in the policy learning process, it is necessary to retain the important features and exclude the unimportant or even interfering features. To achieve this goal, feature filtering can be performed at the feature layer or image layer. However, if the former method is adopted, the encoder must first learn the object-centric representation (OCR) and then filter out irrelevant features from it. This process is cumbersome, difficult to implement and not very interpretable. SAM, or segmentation model, provides researchers with a new way to segment objects. It generates high-quality object masks based on input prompts such as points or boxes, and can also directly generate masks for all objects in the image. Its powerful segmentation ability and zero-sample generalization ability are greatly promoting the development of various fields of computer vision. Therefore, we perform feature filtering at the image layer. Before the observed image I enters the policy network, we use SAM to pre-decompose it.

[0052] Specifically, first, in order to improve the segmentation performance of SAM on the UEVAVD dataset, we took out some training data from the UEVAVD training set and fine-tuned the pre-trained SAM. Then, in the training and testing phases, the target is segmented based on the given target wireframe and the original image. According to the segmentation results, it is not difficult to obtain a circle of coordinate points outside the target outline. With these points as hints, combined with the original image, the mask of the objects around the target can be obtained. Next, these masks are randomly assigned grayscale values ​​in the range of 0 to 1 while keeping the original slices of the target unchanged. The resulting simplified image is as follows: Figure 2This can promote the state representation learned by the backbone of the agent policy network to focus on the appearance, posture, and positional relationship between the target and the surrounding objects, while being insensitive to the color, texture and other features of the objects around the target.

[0053] As mentioned above, the agent needs to understand the three-dimensional scene where the target is located from the two-dimensional observation image. In previous work, the state representation s was extracted only by single-view observation. t , so we get s t It may not support the agent to learn the optimal strategy because of some observation characteristics. Single-view observation is only a projection of the three-dimensional world on a two-dimensional plane, and cannot fully contain all the information required for the agent to make decisions. For example, when the agent determines the amplitude of movement, it needs the depth information of the target and surrounding objects in the scene, which a single RGB image cannot provide. On the contrary, if it relies on image sequences, it can capture their depth information from the relative position changes of each object, thereby helping the agent to make effective decisions. Therefore, we use the GRU network to store the agent's observation history in a memory-based manner to help extract a complete and meaningful state representation.

[0054] The effects of the embodiments of the present invention are described below through experiments.

[0055] First, the training set and test set of the UEVAVD dataset are used for training and testing the agent policy, respectively. Next, we select a ResNet18 network pre-trained on ImageNet as a classifier and fine-tune it using unoccluded multi-view images of five targets. Since the proposed method is an improved version of MAP, it is named IBE-MAP, compared with the baseline methods MAP and Memo-MAP, which is a modified version of MAP that combines a memory module to extract state representations from observation sequences, but does not include a scene pre-decomposition stage compared to IBE-MAP. The comparison results between them will verify the effectiveness of our innovation.

[0056] Since the purpose of the AOD task is to find the best observation angle in the shortest time or motion path to eliminate the uncertainty of the target identity, the experiment uses indicators such as reward value, recognition rate, number of decisions and path length to evaluate the performance of different AOD methods on the UEVAVD dataset. The number of decisions refers to the total number of decisions made by the drone from the beginning of receiving the observation image to the end of the movement in a single round, and the path length refers to the total length of the drone's motion trajectory from the beginning to the end of the AOD task in that round.

[0057] We focus on the performance of the agent’s policy during the test phase and the generalization gap GenGap(π). The overall path length is determined by the number of decision steps and the path length of each decision step.

[0058] By default, we assume a maximum decision step T max is 4. The maximum action range A is set to 4, and the discount factor γ in the reward calculation is set to 0.9. t 、h t and t The dimensions of are 544, 544, and 512 respectively. In the reward function, ξ1 = 0.5, ξ2 = 0.1. Let the upper bound C of the coefficient σ be 0.02, so that it can be adjusted to balance the movement cost and reward value. In the training phase, we use the Adam optimizer with a learning rate of 0.0001 and a total of 300,000 training rounds.

[0059] Figure 3 An intuitive strategy comparison result is given in . Given the same initial observation in the test environment, the three strategies perform differently within one round. The random strategy arbitrarily gives action instructions, thus failing the AOD task. The MAP strategy successfully helps the UAV platform avoid obstacles and obtain correct recognition results. However, it deviates significantly from the optimal path. In contrast, the IBE-MAP strategy guides the UAV to a better viewing angle with a lower movement cost. Once the agent finds enough viewing angles for target recognition, it makes a decision to stop early, thereby improving the efficiency of performing the AOD task.

[0060] Next, we Figure 4 The overall comparison results of the three AOD methods are shown in . The figure shows the changes in their performance in terms of rewards, recognition rates, and movement distances during the training and testing phases. To obtain these colored curves, we first run the program under five random seeds to obtain five sets of fluctuation curves, and smooth them using sliding window averages. Then, a tool called Seaborn is used for visualization, where the solid line in the shaded area represents the mean of the five smoothed curves, and the shaded area represents the fluctuation range of the data, expressed as a confidence interval (confidence level is 0.95). At the end of each training round, the intermediate state strategy is evaluated in the test environment. As the number of training rounds increases, the reward curves under different strategies all increase, reflecting that these strategies can help the agent obtain better recognition results at a lower movement cost.

[0061] At the same time, we can see that the policy derived by the IBEMAP method has the strongest generalization ability, as its reward value surpasses the other two methods at test time after convergence, and its generalization gap is the smallest. Although the original MAP performs better during training, its generalization gap is the largest, reflecting the overfitting problem. By incorporating historical observations into the state representation, the policy derived by Memo-MAP is slightly better than MAP, with a shorter move distance and a smaller generalization gap. The above results verify the effectiveness of the scene pre-decomposition and memory-based state estimation innovations.

[0062] Next, we will analyze two hyperparameters in the reward function: thre and C, to understand their impact on policy performance. All evaluation results are obtained by testing fully trained policies on the difficult test set, and the best results are indicated in bold. thre refers to the threshold for judging whether the classifier correctly identifies the target, that is, when the classifier score is higher than thre, it is considered a successful classification. Obviously, thre may affect the reward function and thus affect the learned policy. Table 2 gives the policy evaluation results under different thresholds. We can see that as thre increases, the reward decreases, the path length increases, and the impact on recognition accuracy is small. The reason is that the larger the threshold, the agent is forced to move more frequently to find the "perfect" perspective, resulting in a longer path length and lower rewards. Since there are five targets, the smallest thre is 0.2, and we found that the policy under this configuration performs best.

[0063] Table 2 Changes in agent strategy performance under different classification threshold conditions

[0064] thre=0.2 thre=0.4 thre=0.6 thre=0.8 Return Value -0.059±0.006 -0.072±0.021 -0.102±0.011 -0.178±0.011 Accuracy 0.610±0.015 0.602±0.022 0.605±0.009 -0.603±0.015 Path length 4.065±0.169 4.160±0.171 4.211±0.152 4.505±0.190

[0065] In the reward function, the coefficient σ is used to control the impact of the action range on the reward value, thereby balancing the movement cost and accuracy gain. By adjusting its upper limit C, we can observe the changing trend of the policy test performance, as shown in Table 3. When C = 0, we only care about whether the newly obtained imaging result can be recognized, regardless of how far the drone has moved to obtain the result. In other words, there is no further constraint on the action range of each step at this time, so the agent can explore the environment more freely, thereby obtaining better recognition accuracy and the longest path length. As C increases, the constraints on the action amplitude become stricter, the path length is shortened, and the recognition rate decreases. Obviously, there is a trade-off between accuracy and path length, so the setting of C depends on the relative importance of these two factors when performing the AOD task.

[0066] Table 3. Changes in the performance of the agent strategy with the hyperparameter C

[0067] C＝0 C＝0.02 C＝0.04 C＝0.06 Return Value 0.000±0.004 -0.059±0.006 -0.119±0.010 -0.146±0.019 Accuracy 0.616±0.007 0.610±0.015 0.577±0.005 -0.575±0.025 Path length 4.717±0.160 4.065±0.169 3.509±0.104 3.147±0.076

[0068] Example 2

[0069] Embodiment 2 of the present invention provides a terminal device corresponding to the above-mentioned embodiment 1. The terminal device may be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0070] The terminal device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in the above-mentioned embodiment 1.

[0071] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0072] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0073] Example 3

[0074] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to the above embodiment 1, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the above embodiment 1 are implemented.

[0075] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0076] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0079] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0080] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for actively detecting ground targets by a UAV, characterized in that: The following steps are involved: S1. Initialize the agent strategy network model π train , experience cache pool, round counter is reset to zero; S2. Set t = 0 and initialize the drone position p t , get the corresponding drone observation image o t , initialize the hidden state h t ; S3, observing the image o of the drone t The image to be processed is cropped to obtain the image to be processed, the scene is pre-decomposed for the image to be processed, and the feature extraction is performed on the image after the scene pre-decomposition, and the extracted features are compared with the hidden state h t After GRU fusion, the state s is obtained t , update the hidden state h t , if t>0, then the interaction experience (s t-1 ,a t-1 ,r t-1 ,s t ) is stored in the experience cache pool; S4, if the amount of experience data in the experience buffer pool is greater than the threshold thre pool , then a batch of data is extracted from the experience buffer pool to the agent strategy network model π train Conduct training; S5, based on state s t , the agent decision network gives the action output a at the current moment t , training environment feedback reward value r t , the drone is based on a t Move to new position p t+1 , get the new target observation image o t+1 ; S6, the value of t increases by 1, if t reaches the maximum time step T max , then jump to step S2, otherwise jump to step S3. When the number of rounds is greater than the set constant, the trained agent strategy network model is obtained.

2. The method for active detection of ground targets by unmanned aerial vehicles according to claim 1, characterized in that: Also includes: S7. Use the trained agent strategy network model to perform active target detection on the UAV observation image to obtain the target observation image.

3. The method for active detection of ground targets by unmanned aerial vehicles according to claim 1, characterized in that: The agent strategy network model adopts the Dauling deep Q network.

4. The method for active detection of ground targets by unmanned aerial vehicles according to claim 1, characterized in that: In step S2, the scene pre-decomposition of the image to be processed is performed, and the specific implementation process of feature extraction of the image after the scene pre-decomposition includes: The SAM model is used to segment the image to be processed with the target wireframe as a prompt, and a mask is used to cover the surrounding objects of the target. The encoder is used to extract features of the masked image, and the extracted feature vector is spliced ​​with the vector representation of the target wireframe coordinates and the vector representation of the drone observation azimuth to obtain the features corresponding to the single-view observation, that is, the extracted features.

5. The method for active detection of ground targets by unmanned aerial vehicles according to claim 4, characterized in that: The grayscale value of the mask of the feature is randomly distributed in the range of 0 to 1.

6. A terminal device comprising a memory, a processor and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

8. A computer program product comprising a computer program / instructions; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method of any one of claims 1 to 5 are implemented.

Images