Unmanned aerial vehicle active detection method for ground target, terminal device and storage medium

Abstract

The application discloses a kind of unmanned aerial vehicle to ground target active detection method, terminal equipment and storage medium, utilize induction bias improvement strategy network's main part learned state representation, to improve the effectiveness and generalization of agent learned strategy when solving active target detection problem.First, the image features that active target detection problem mainly concerns should concentrate on the characteristics of target appearance, posture and the position relationship between target and surrounding ground objects, etc., and the features such as color and texture of ground objects can be discarded, so the application proposes to perform scene pre-decomposition operation on the image features before feature extraction, and then filter out these irrelevant features.Second, the application proposes to use GRU network to fuse the features of historical observation sequence.Through these two improvements, the generalization ability of state representation can be further improved, thereby helping the agent to learn better active target detection strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to target detection technology, in particular to an unmanned aerial vehicle active target detection method, a terminal device and a storage medium. BACKGROUND

[0002] In recent years, unmanned aerial vehicles have shown extraordinary performance beyond traditional means in applications such as traffic monitoring, post-disaster rescue, smart agriculture, and military reconnaissance, due to their advantages such as maneuverability, wide field of view, rapid deployment, and low cost. Behind these applications, target detection is an indispensable key technology. Through target detection, targets can be located and identified from aerial images, providing important prior information for subsequent actions. Therefore, enhancing target detection performance is of great significance to further improving the quality of unmanned aerial vehicle service support.

[0003] Since the deep learning boom, deep learning methods have gradually replaced traditional methods and become the mainstream detection means in the field of unmanned aerial vehicle target detection. For example, methods such as Faster-RCNN, YOLO, SSD, and their variants have been widely used in remote sensing target detection. However, in unmanned aerial vehicle aerial images, the effect of target detection will inevitably be affected by the interference of ground cover, leading to a decrease in detection performance or even failure of target detection. To solve this difficult problem, existing methods mostly make different adaptive improvements to the detection model according to the characteristics of the problem. Although researchers have made many efforts to improve, the ability of the detector on the target detection task with occlusion is still far from satisfactory. Fundamentally, the quality of the input model data limits the upper limit of the detection performance improvement, and only adaptive improvement of the model may not be able to obtain performance breakthrough. Relatively speaking, considering that unmanned aerial vehicles have strong maneuverability, if they can change the viewing angle of the observed target to obtain images more conducive to identifying the target, it will greatly improve the performance upper bound of target detection, i.e. overcoming the difficulty of target occlusion through active perception.

[0004] Active vision is a field created by humans based on the visual mechanism of organisms in nature, which studies how the visual system autonomously decides the movement of the camera according to the task requirements and analysis results, and obtains the corresponding images from the appropriate viewing angle. Since Krotkov et al. proposed it in the 1990s, active vision has been widely applied in navigation positioning, intelligent monitoring, object recognition, target tracking, and other fields. Active target detection (AOD) is a sub-field of active vision for target detection, whose research purpose is to determine the position information but not the identity information of the target, and to improve the target detection effect by giving the mobile sensing platform the ability to autonomously adjust the viewing angle to obtain more favorable visual information for judging the target identity. For example, Figure 1As shown, the UAV cannot reliably determine the target identity information due to occlusion and other reasons at certain viewing angles. At this time, if the UAV is to complete the AOD task, it needs to solve two problems: 1) how to find the best observation viewing angle to reliably eliminate the uncertainty of the target identity information; 2) how to optimize the motion trajectory to minimize the motion cost. It is worth noting that AOD is different from target-driven visual navigation (TDVN). For the latter, the agent usually plans the viewing angle based on video data, while for the former, the agent uses image data. In addition, in the AOD task, the target needs to appear in the initial observation, while in the initial stage of TDVN, the target may not be in the agent's field of view, but needs to appear in the final observation.

[0005] However, existing AOD research mainly focuses on indoor robot applications, and there is a lack of AOD related research for outdoor UAVs. In terms of methods, existing DRL-based AOD methods have two shortcomings. First, in the above representative works, the AOD problem is usually modeled as a Markovian Decision Process (MDP), and then the target observation image at a single viewing angle is directly used as the state of the agent. However, since the observation image is only a projection of the three-dimensional real world, the entire information of the state in which the agent is located cannot be completely contained in a single observation, so the information extracted from a single observation is not sufficient to support the agent to learn the optimal active observation strategy. On the other hand, in existing DRL-based AOD methods, the agent obtains rewards in the interaction with the environment, which is used as a supervision signal to promote the continuous optimization of the agent's policy network. However, the reward value is a weak supervision signal, which is difficult to help the agent efficiently learn the policy, especially in the case of limited training data, the state representation learned by the backbone of the policy network may not capture the key features related to the task, resulting in poor generalization ability of the agent's policy. SUMMARY

[0006] The technical problem to be solved by the present application is to provide a UAV active detection method for ground targets, a terminal device and a storage medium to improve the effectiveness and generalization of the learned policy of the agent in solving the active target detection problem.

[0007] To solve the above technical problems, the technical solution adopted by the present application is: a UAV active detection method for ground targets, comprising the following steps:

[0008] S1, initialize the agent policy network model π train , the experience cache pool, and the round counter are set to zero;

[0009] S2. Set t=0, initialize the drone position p. t Obtain the corresponding UAV observation image. t Initialize the hidden state h t ;

[0010] S3, Observe the image of the UAV. t Cropping is performed to obtain the image to be processed. Scene pre-decomposition is performed on the image to be processed. Feature extraction is performed on the image after scene pre-decomposition. The extracted features are then compared with the hidden state h. t After GRU fusion, state s is obtained. t Update the hidden state h t If t > 0, then the interactive experience (s) will be... t-1 ,a t-1 ,r t-1 ,s t Stored in the experience cache pool;

[0011] S4. If the number of experience data in the experience cache pool is greater than the threshold, then... pool Then, a batch of data is extracted from the experience cache pool to analyze the agent policy network model π. train Conduct training;

[0012] S5, based on state s t The agent decision network outputs the action 'a' at the current moment. t Training environment feedback reward value r t The drone is based on a t Move to the new position p t+1 New target observation images are obtained. t+1 ;

[0013] S6, increment the value of t by 1, if t reaches the maximum time step T. max If the number of rounds is greater than the set constant, then proceed to step S2; otherwise, proceed to step S3. When the number of rounds is greater than the set constant, the trained agent policy network model is obtained.

[0014] The method of the present invention further includes:

[0015] S7. Use the trained agent policy network model to perform target detection on the UAV observation images to obtain target observation images.

[0016] In this invention, the agent policy network model adopts a Dauling deep Q-network.

[0017] In step S2, the image to be processed is pre-decomposed in a scene, and a specific implementation process of feature extraction of the image pre-decomposed in the scene includes: using a SAM model to segment the image to be processed with a target line frame as a prompt, using a mask to cover ground objects around the target, using an encoder to extract features of the image covered by the mask, using the SAM model to segment the image to be processed with the target line frame as the prompt, using the mask to cover the ground objects around the target, using the encoder to extract features of the image covered by the mask, splicing the extracted feature vectors with vector representation of the target line frame coordinates and vector representation of a UAV observation azimuth angle to obtain features corresponding to single-view observation, that is, to obtain the extracted features.

[0018] As an inventive concept, the present application also provides a terminal device comprising a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.

[0019] As an inventive concept, the present application also provides a computer readable storage medium having a computer program / instruction stored thereon; the computer program / instruction is executed by a processor to implement the steps of the above method.

[0020] As an inventive concept, the present application also provides a computer program product comprising a computer program / instruction; the computer program / instruction is executed by a processor to implement the steps of the above method.

[0021] Compared with the prior art, the present application has the beneficial effects that: the present application uses inductive bias to improve the state representation learned by the backbone part of the strategy network, thereby improving the effectiveness and generalization of the strategy learned by the agent in solving the active target detection problem. Specifically, the inductive bias includes two aspects, which are embodied in the scene pre-decomposition in the S3 step and the feature fusion operation based on the GRU network. First, the image features mainly concerned in the active target detection problem should focus on the characteristics of the appearance, posture of the target itself and the positional relationship between the target and the surrounding ground objects, and the features such as color and texture of the ground objects can be discarded, so the present application proposes to perform a scene pre-decomposition operation on the image features before extracting them, and then filter out these irrelevant features. Second, due to the partial observable characteristics, the state representation of the agent needs to be extracted from the historical observation sequence, not just rely on the observation under a single view, so the present application proposes to fuse the features of the historical observation sequence using the GRU network. Through the improvement of the two aspects, the generalization ability of the state representation can be further improved, thereby helping the agent to learn a better active target detection strategy. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 Motivation diagram for active target detection of a UAV in a scene

[0023] Figure 2 The forward reasoning flow chart of the embodiment method of the present application;

[0024] Figure 3 The detection result visualization comparison picture sequence form of the embodiment of the present application;

[0025] Figure 4 The evaluation result comparison chart of the three AOD methods; (a) return value change curve, (b) recognition rate change curve, (c) motion distance change curve. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in combination with the drawings in the embodiment of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0027] Embodiment 1

[0028] The process of the UAV performing the AOD (active target detection) task can be summarized as follows: the UAV as an intelligent agent makes decisions based on the original observation to find the best observation angle with the minimum motion and calculation cost, and reliably eliminates the uncertainty of the target identity information.

[0029] We model the AOD problem as a POMDP, which is usually represented by a seven-tuple , represents the state set of the agent, corresponding to the useful features abstracted from the UAV observation data, represents the action set, represents the action that the agent can take, including the action type a type and the action amplitude a range two parts, a type ∈{forward,backward,left,right,up,down,stop}, a range ∈{1,2,...,A}. Let T(s'|s,a) represent the set of observations. T(s'|s,a) represents the state transition function, and Ω(ο|s,a) is the observation function, representing the probability that the environment will provide observation o to the agent after the agent is in state s and takes action a. The original observation o = (I,b), where I represents the aerial image acquired by the UAV, and b represents the wireframe information of the target (assuming it is obtained by a tracking algorithm or given manually). γ is the discount factor when calculating the reward value. R(s,a) represents the reward function, whose design should take into account the target recognition result, the number of decisions, and the length of the trajectory. We define the reward function as:

[0030]

[0031] when a type =stop or t=T max When the value is -1, the variable flag = 1, indicating the end of a round; otherwise, the agent needs to continue making decisions. reco = 1 indicates the target was correctly identified, and reco = 0 indicates an incorrect identification. ξ1 and ξ2 are reward constants greater than 0, and σ is a coefficient used to control the influence of the action magnitude on the reward function; its value depends on a. type ,Right now:

[0032]

[0033] Based on this, the following optimization problem is solved using the RL algorithm:

[0034]

[0035] Wherein, π(a t |s t ) represents the agent's policy, Π represents the set of policies, π* represents the optimal policy, and ρ(s) represents the distribution of the initial state. r(s) represents a round (assuming a round contains at most T elements). max The expected return value with an initial state of s within (n time steps) can be calculated using the following formula:

[0036]

[0037] To find the optimal policy, the state must first be estimated from the observation data. Existing work usually only relies on the current observations. t To estimate state s t However, due to the incomplete observation characteristic of a single perspective, the resulting state is insufficient to support the agent in learning the optimal policy, because a single observation cannot distinguish multiple states; that is, a single observation cannot uniquely determine a state. Instead, we adopt a sequence of observations o1, o2, ..., o t Estimated state s tIn this way, the accuracy of the agent state description is improved, and the performance of the learned strategy is improved.

[0038] Since the training set and the test set in the UEVAVD dataset contain multiple environment settings, the agent network needs to be trained and tested in multiple environment instances. To formally and quantitatively describe the zero-shot generalization (ZSG) performance of the agent on the UEVAVD dataset, we introduce a contextual Markovian decision process (CMDP) based on POMDP modeling. The CMDP is a POMDP that introduces the concept of context. It is represented by the tuple , which is compared to the general POMDP modeling representation . The main difference is that the context space is introduced. The context variable refers to the location of the target. In addition, we call the hidden state space, which is related to the original state space .

[0039] For a CMDP with a context set , the expected return value of the strategy π on it is:

[0040]

[0041] where represents the CMDP under the given context c. Next, let be the training context set and the test context set, respectively. The agent updates the strategy by interacting with the training environment set corresponding to until the expected return converges to the optimal result, obtaining the final , that is:

[0042]

[0043] We expect to also perform well on the test environment set corresponding to , that is, to make as high as possible. In addition to observing the value of , we also use the GenGap(π) index to measure the zero-shot generalization capability of the strategy. For​ And the formula is:

[0044]

[0045] The smaller the GenGap(π) is, the smaller the difference between the performance of the policy in the training and testing environment is. In addition, we need to observe the value of If the value of is higher and the GenGap(π) is smaller, it means that the generalization performance of the policy is stronger.

[0046] Based on the above problem modeling, Figure 2 The flow chart of the AOD method based on DRL proposed by us is shown in FIG. 1. Compared with the prior art of the same type, the innovation of the method is to improve the state representation learned by the main part of the policy network by using inductive bias, so as to improve the effectiveness and generalization of the policy learned by the agent when solving the AOD problem. The inductive bias here includes two aspects. First, the image features mainly concerned by the AOD problem should focus on the characteristics of the appearance, posture and position relationship of the target and the surrounding objects, and the color, texture and other features of the objects can be discarded. Second, due to the partially observable characteristics, the state representation of the agent needs to be extracted from the historical observation sequence, not just rely on the observation under a single perspective.

[0047] Figure 2 The upper half of FIG. 1 presents the abstract process of the entire AOD task at the macro level. The agent continuously optimizes the observation angle of the target by making decisions at multiple time steps, and sends the target slice under the ideal observation angle to the classifier for classification. Each time step includes imaging, scene pre-decomposition, feature extraction, feature fusion and decision making. First, the UAV moves to a new observation site according to the action a t of the previous time step, images the scene where the target is located, and crops the image of the target surrounding area from it. Then, the SAM model is used to segment the obtained image based on the target wireframe as a prompt, and the target surrounding objects are covered by masks. The gray value of each mask of the surrounding objects is randomly distributed in the range of 0 to 1, so that the observation image contains only the contour features of the surrounding objects except the target itself, and the color, texture and other features of the surrounding objects are excluded. Then, the simplified image is feature-extracted by the encoder, and the extracted features are spliced with the target wireframe coordinates, UAV observation azimuth and other information to obtain the single-perspective observation corresponding feature g t . To improve the efficiency of the training samples, the encoder here is usually pre-trained. Then, we use GRU to fuse the feature vectors corresponding to the previous observations in order to learn the state representation s t from the historical observation sequence which only contains the features related to the AOD task. On this basis, we follow the s tThe method for making decisions is that in addition to estimating the state value by using a Dauling deep Q network (Dualing DQN), the Dualing DQN is also caused to output the advantages of various possible actions in two dimensions (action type and action amplitude), the values of various action combinations are obtained in a summation manner, and then the optimal action a* is selected from the values as a t+1 The output. The specific method steps are shown in the following table:

[0048] Table 1 summarizes the bias-enhanced reinforcement learning active object detection method process

[0049]

[0050]

[0051] The key for the agent to perform the AOD task is to understand the three-dimensional scene where the target is located from the two-dimensional observation image, which specifically includes the appearance, pose of the target, and the positional relationship between the target and the surrounding terrain. The color and texture of the surrounding terrain of the target are not important for the completion of the AOD task. Therefore, in order to make the state representation more concise and compact and to reduce the overfitting phenomenon in the policy learning process, important features need to be retained and unimportant or even interfering features need to be excluded. To achieve this purpose, feature filtering can be performed at the feature layer or the image layer. However, if the former way is adopted, the encoder needs to learn an object-centric representation (OCR) centered on the target first, and then irrelevant features need to be filtered from the OCR. This process is tedious, difficult to implement, and not strongly interpretable. SAM, a segmentation model, provides researchers with a new way of object segmentation. It can generate high-quality object masks according to point or frame input prompts, and can also directly generate masks for all objects in the image. The strong segmentation capability and zero-shot generalization capability of SAM are greatly promoting the development of various fields of computer vision. Therefore, we perform feature filtering at the image layer. Before the observation image I enters the policy network, we perform a pre-decomposition operation on it by using SAM.

[0052] Specifically, first, to improve the segmentation performance of SAM on the UEVAVD dataset, we fine-tune the pre-trained SAM by taking part of the training data in the UEVAVD training set. Subsequently, the target is segmented based on the given target frame and the original image during the training and testing stages. According to the segmentation result, it is not difficult to obtain a circle of coordinate points outside the target contour. By using these points as prompts and combining the original image, the mask of the surrounding terrain of the target can be obtained. Next, we randomly assign gray values in the range of 0 to 1 to these masks while keeping the original slice of the target unchanged. The obtained simplified image is as follows: Figure 2As shown in the middle. In this way, the state representation learned by the backbone of the agent policy network can focus on the appearance and pose of the target, and the position relationship between the target and the surrounding objects, and be insensitive to the color and texture of the surrounding objects of the target.

[0053] As mentioned earlier, the agent needs to understand the three-dimensional scene where the target is located from the two-dimensional observation image. In the past work, only single-view observation is used to extract the state representation s t , and s t may not support the agent to learn the optimal strategy, because part of the observation characteristics, single-view observation is only a projection of the three-dimensional world in a two-dimensional plane, and cannot comprehensively contain all the information required for decision-making of the agent. For example, the agent needs the depth information of the target and the surrounding objects in the scene when deciding the movement amplitude, and a single RGB image cannot provide it. On the contrary, if relying on image sequences, the depth information of the objects can be captured from the relative position changes of the objects, thereby helping the agent to make effective decisions. Therefore, we use the GRU network to store the observation history of the agent in a memory-based manner, to help extract complete and meaningful state representations.

[0054] The following experiments illustrate the effectiveness of the embodiment scheme of the present application.

[0055] First, the training set and the test set of the UEVAVD dataset are respectively used for training and testing of the agent policy. Next, we select a ResNet18 network pre-trained on ImageNet as the classifier, and use the unoccluded multi-view images of five targets to fine-tune it. Since the method of the present application is a modified version of the MAP method, it is named IBE-MAP. Compared with the baseline method MAP and Memo-MAP, the latter is a modified version of MAP, which combines a memory module to extract state representations from observation sequences, but does not include the scene pre-decomposition stage compared with IBE-MAP. The comparison between them will verify the effectiveness of our innovation.

[0056] Since the purpose of the AOD task is to find the best observation view to eliminate the uncertainty of the target identity in the shortest time or movement path, the experiment uses the reward value, recognition rate, decision-making times and path length as indexes to evaluate the performance of different AOD methods on the UEVAVD dataset. The decision-making times refer to the total number of decisions made by the UAV from the beginning of receiving the observation image to the end of the movement in a single round, and the path length refers to the total length of the UAV movement trajectory from the beginning to the end of the AOD task in the round.

[0057] We mainly focus on the performance of the agent policy in the test stage and the generalization gap GenGap(π). The total path length is determined by the number of decision steps and the path length of each decision step.

[0058] By default, we assume the maximum decision step T max is 4. The maximum action range A is set to 4, and the discount factor γ in the reward calculation is set to 0.9. t The dimensions of g t , h t , and s are 544, 544, and 512, respectively. In the reward function, ξ1= 0.5 and ξ2= 0.1. Let the upper bound C of the coefficient σ be 0.02, which can be adjusted to balance the motion cost and the reward value. During the training phase, we use the Adam optimizer with a learning rate of 0.0001 and a total of 300,000 training episodes.

[0059] Figure 3 A visual comparison of the strategies is given in Fig. 5. Given the same initial observation in the test environment, the three strategies behave differently within one episode. The random strategy gives arbitrary action commands, thus failing the AOD task. The MAP strategy successfully helps the UAV platform avoid obstacles and obtain the correct recognition result. However, it deviates significantly from the optimal path. In contrast, the IBE-MAP strategy guides the UAV to reach a better viewing angle with lower movement cost. Once the agent finds a viewing angle sufficient for target recognition, it makes the decision to stop early, thus improving the efficiency of executing the AOD task.

[0060] Next, we show the overall comparison results of the three AOD methods in Fig. 6. This figure demonstrates their performance variations about the reward, recognition rate, and movement distance during the training and testing phases. To obtain these color curves, we first run the program under five random seeds to obtain five sets of fluctuating curves and smooth them using a sliding window average. Then, we visualize them using a tool named Seaborn, where the solid lines in the shaded areas represent the mean of the five smoothed curves, and the shaded areas represent the fluctuation range of the data, expressed in the confidence interval (confidence level of 0.95). At the end of each training round, the intermediate-state strategies are evaluated in the test environment. As the training episodes increase, the reward curves under different strategies all rise, reflecting that these strategies can help the agent obtain better recognition results with lower movement cost.

[0061] Meanwhile, we can see that the strategy derived by the IBE-MAP method has the strongest generalization ability, as its reward value surpasses the other two methods in the test after convergence, and its generalization gap is the smallest. Although the original MAP performs better during training, its generalization gap is the largest, reflecting the overfitting problem. By incorporating historical observations into the state representation, the strategy derived by the Memo-MAP outperforms the MAP, with shorter movement distance and smaller generalization gap. The above results verify the effectiveness of the innovations of scene pre-decomposition and memory-based state estimation.

[0062] Next, we analyze the two hyperparameters in the reward function: thre and C, to understand their impact on the policy performance. All the evaluation results are obtained by testing the well-trained policy on the difficulty test set, and the best result is shown in bold. thre refers to the threshold to determine whether the classifier correctly identifies the target, i.e., when the classifier score is higher than thre, it is considered a successful classification. Obviously, thre can affect the reward function and thus the learned policy. Table 2 shows the policy evaluation results under different thresholds. We can see that as thre increases, the return decreases and the path length increases, while the impact on the identification accuracy is small. The reason is that the larger the threshold, the more frequently the agent is forced to move to find the "perfect" perspective, resulting in longer path length and lower return. Since there are five targets, the smallest thre is 0.2, and we find that the policy performs best under this configuration.

[0063] Table 2 Changes in the performance of the agent's policy under different classification threshold conditions

[0064] thre = 0.2 thre = 0.4 thre = 0.6 thre = 0.8 Return value -0.059±0.006 -0.072±0.021 -0.102±0.011 -0.178±0.011 Accuracy 0.610±0.015 0.602±0.022 0.605±0.009 -0.603±0.015 Path length 4.065±0.169 4.160±0.171 4.211±0.152 4.505±0.190

[0065] In the reward function, the coefficient σ is used to control the impact of the action range on the reward value, thereby balancing the movement cost and accuracy gain. By adjusting its upper limit C, we can observe the trend of policy test performance, as shown in Table 3. When C = 0, we only focus on whether the newly obtained imaging result can be identified, without considering how far the UAV has moved to obtain that result. In other words, at this time, there is no further constraint on the action range for each step, so the agent can explore the environment more freely, thereby achieving better identification accuracy and the longest path length. As C increases, the constraint on the action amplitude becomes more stringent, and the path length shortens while the identification rate decreases. Obviously, there is a trade-off between accuracy and path length, so the setting of C depends on the relative importance of these two factors when performing the AOD task.

[0066] Table 3 Changes in the performance of the agent's policy with the hyperparameter C

[0067] C＝0 C＝0.02 C＝0.04 C＝0.06 Return value 0.000±0.004 -0.059±0.006 -0.119±0.010 -0.146±0.019 Accuracy 0.616±0.007 0.610±0.015 0.577±0.005 -0.575±0.025 Path length 4.717±0.160 4.065±0.169 3.509±0.104 3.147±0.076

[0068] Embodiment 2

[0069] Embodiment 2 of the present application provides a terminal device corresponding to the above-mentioned embodiment 1, which can be a processing device for a client, such as a mobile phone, a notebook computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0070] The terminal device of the embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program stored in the memory to implement the steps of the method of embodiment 1.

[0071] In some implementations, the memory can be a high-speed random access memory (RAM), and can also include a non-volatile memory, such as at least one disk memory.

[0072] In some implementations, the processor can be a central processing unit (CPU), a digital signal processor (DSP), or various types of general-purpose processors, without limitation.

[0073] Embodiment 3

[0074] Embodiment 3 of the present application provides a computer-readable storage medium corresponding to the above-mentioned embodiment 1, which stores a computer program / instruction. The computer program / instruction is executed by the processor to implement the steps of the method of embodiment 1.

[0075] The computer-readable storage medium can be a tangible device that maintains and stores instructions for use by an instruction execution device. The computer-readable storage medium can be, for example but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0076] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0077] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions described in the flowcharts and / or block diagrams.Figure 1 apparatuses that implement functions specified in one or more flowcharts and / or blocks. Figure 1

[0078] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide a process for implementing the functions specified in the flowcharts Figure 1 apparatuses that implement functions specified in one or more flowcharts and / or blocks. Figure 1

[0079] Although the preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims be construed to include all such modifications and variations as fall within the scope of the application.

[0080] Obviously, various modifications and changes are possible in the application without departing from the spirit and scope of the application. Accordingly, it is intended that the application embrace all modifications and changes that fall within the scope of the claims and their equivalents.​​

Claims

1. A method for active detection of ground targets by a UAV, characterized in that, The method comprises the following steps: S1, initialize the agent policy network model π train , the experience cache pool, the round counter is zeroed; S2, set t = 0, initialize the position of the UAV p t , obtain the observation image o corresponding to the UAV t , initialize the hidden state h t ; S3, observing the unmanned aerial vehicle image o t cropping to obtain a to-be-processed image, performing scene pre-decomposition on the to-be-processed image, performing feature extraction on the image after scene pre-decomposition, and storing the extracted features and the hidden state h t obtaining a state s t after GRU fusion t updating the hidden state h t-1 , if t>0, storing the interaction experience (s t-1 , a t-1 , r t , s ) in an experience cache pool. S4, if the number of experience data in the experience cache pool is greater than a threshold thre pool , a batch of data is extracted from the experience cache pool to train the agent policy network model π train ; S5, based on state s t The agent decision network outputs the action 'a' at the current moment. t Training environment feedback reward value r t The drone is based on a t Move to the new position p t+1 New target observation images are obtained. t+1 ; S6, the value of t is added by 1, if t reaches the maximum time step T max then jump to step S2, otherwise go to step S3, when the number of rounds is greater than the set constant, the trained agent policy network model is obtained.

2. The method of claim 1, wherein, Further comprising: S7, using the trained agent policy network model to perform active target detection on the UAV observation image to obtain a target observation image.

3. The method of claim 1, wherein, The agent policy network model adopts a Dauling deep Q network.

4. The method of claim 1, wherein, In step S2, the specific implementation process of performing scene pre-decomposition on the to-be-processed image and performing feature extraction on the image after scene pre-decomposition comprises: The SAM model is used to segment the to-be-processed image with the target wireframe as a prompt, the mask is used to cover the surrounding objects of the target, the image after being covered by the mask is subjected to feature extraction by an encoder, the extracted feature vector is spliced with the vector representation of the target wireframe coordinates and the vector representation of the UAV observation azimuth angle to obtain the features corresponding to single-view observation, i.e., the extracted features.

5. The method of claim 4, wherein, The gray values of the mask of the object are randomly distributed in the range of 0 to 1.

6. A terminal device comprising a memory, a processor, and a computer program stored on the memory; characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-5.

7. A computer readable storage medium having stored thereon computer programs / instructions; characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method of any one of claims 1-5.

8. A computer program product comprising computer programs / instructions; characterized in that, The computer program / instruction is executed by the processor to implement the steps of the method of any one of claims 1-5.

Images