Active tracking method for cross-scene of unmanned aerial vehicle in open world and related equipment

Abstract

The invention discloses an open world unmanned aerial vehicle cross-scene active tracking method and related equipment, and the method comprises the steps: creating various open world environments, and obtaining an open world unmanned aerial vehicle active tracking system; creating a tracker and a tracking target, and designing a target management scheme; packaging the active tracking system into a universal interface to adapt to a current mainstream reinforcement learning framework; constructing an unmanned aerial vehicle active tracking model, and designing a reward function; based on the reward function, adopting a near-end strategy optimization reinforcement learning algorithm to train the unmanned aerial vehicle active tracking model; and obtaining an input image, inputting the input image into the trained unmanned aerial vehicle active tracking model, and outputting a tracking signal for controlling the unmanned aerial vehicle. The unmanned aerial vehicle active tracking agent is trained through a near-end strategy optimization algorithm and a course learning strategy, so that the agent can adapt to complex visual features of the open world. The method can be widely applied to the fields of security monitoring, automatic driving, robot service and the like.

Description

Technical Field

[0001] The present invention relates to technical fields such as deep learning, reinforcement learning, and computer vision, and in particular to an active tracking method and related equipment for open-world drones across scenarios. Background Art

[0002] With the continuous development of deep learning technology, visual tracking tasks have made significant progress. This technology has been widely used in security monitoring, autonomous driving and other fields.

[0003] The emergence of correlation filtering methods, Siamese network models, and attention-based tracking models have brought significant breakthroughs in visual tracking. These visual tracking models utilize video data and target box labels for network learning, performing training and prediction under a preset fixed viewing angle. However, while these models achieve high tracking success rates in video datasets, existing models are unable to generate control signals for the camera lens based on visual observations, limiting the practical application of current visual tracking algorithms. This limitation manifests itself in the limited camera perception range, making it incapable of adapting to highly dynamic scenes and tracking targets, hindering the practical deployment of visual tracking models.

[0004] To address this issue, researchers have proposed active visual tracking tasks and are working to improve tracking methods so that visual tracking models can generate control signals based on visual observations. One approach involves integrating the aforementioned visual tracking model with a control algorithm to introduce control signals. Another approach involves constructing a simulation environment with controllable camera viewpoints to train a model end-to-end that outputs control signals based on image information.

[0005] Currently, in the field of visual active tracking, although reinforcement learning algorithms have been preliminarily verified to be feasible in corresponding simulation environments, they still face three major limitations: (1) Lack of a unified benchmark construction method. The current simulation environment is small in scope, low in complexity, and limited in number, which cannot fully verify the capabilities of the intelligent agent. In addition, the number of trackers and targets provided by the existing environment is limited, and it cannot provide a platform for different tasks. (2) Poor adaptability to large environments with complex interference. Open-world active tracking involves large-scale, highly dynamic environments and frequent environmental interference. Direct training in complex environments often leads to slow convergence and poor tracking robustness. (3) Poor adaptability to tracking targets with diverse behaviors. Existing methods assume that the tracker adopts a fixed horizontal forward perspective, which significantly limits the tracker's perception range and causes the reward function to be unable to accurately reflect the tracking performance when the perspective changes. Summary of the Invention

[0006] In order to at least solve one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide an active tracking method and related equipment for open-world drones across scenarios.

[0007] The first technical solution adopted by the present invention is:

[0008] An active tracking method for open-world drones across scenarios includes the following steps:

[0009] Create diverse open world environments and gain access to an open world drone active tracking system;

[0010] Create common trackers and tracking targets, design target management solutions, and provide common sensor data;

[0011] Encapsulate the active tracking system into a universal interface to adapt to the current mainstream reinforcement learning framework;

[0012] Build a drone active tracking model and design a reward function;

[0013] Based on the reward function, the Proximal Policy Optimization (PPO) reinforcement learning algorithm is used to train the UAV active tracking model;

[0014] Get the input image, feed it into the trained UAV active tracking model, and output the tracking signal to control the UAV.

[0015] Furthermore, the creation of a diverse open world environment and obtaining an open world drone active tracking system includes:

[0016] Build multiple scene maps based on preset environmental scenes and weather conditions. Preset environmental scenes include lakes, towns, urban areas, villages, deserts, and farmlands. Weather conditions include sunny days, nights, foggy days, and snowy days.

[0017] For lake, city street, and downtown maps, we used the OpenStreetMap platform to capture blocks from real scenes and converted them into 3D assets in the Webots software. For village, desert, and farmland maps, we used Creo software to directly model the elements within the scene.

[0018] After obtaining the 3D assets, manually modify the current scene assets according to different scene requirements, and add other scene elements to increase the visual interference of the environment to obtain the open world drone active tracking system.

[0019] Furthermore, the trackers include two types of trackers: ground robots and drones. The tracking targets include cars, motorcycles, pedestrians, wheeled robots and legged robots, which are 24 common tracking targets in five categories.

[0020] Use SUMO (traffic simulation engine) to uniformly manage tracking targets within the map, and automatically configure target appearance, behavior, and interactions between targets;

[0021] Provides a variety of commonly used vision and motion sensors, providing a unified platform for various mission requirements.

[0022] Furthermore, encapsulating the active tracking system into a universal interface includes:

[0023] The active tracking system is encapsulated into a Gymnasium interface to achieve lightweight configuration of functions and adapt it to existing mainstream reinforcement learning frameworks.

[0024] Furthermore, the design reward function includes:

[0025] The tracking target and image are unified in the same plane for calculation, that is, the projection quadrilateral of the image on the ground is obtained; within the projection quadrilateral, the reward function is constructed according to the principle of distance attenuation from the image center. The expression of the reward function is as follows:

[0026]

[0027] Where x represents the distance between the target and the center of the image; α is the attenuation factor, which controls the attenuation of the reward function; I target To track the position of the target in the image, I clip The image cropping area is used to prevent the drone from keeping the target at the edge of the image for a long time. When the target is outside the image, the reward function is 0.

[0028] Furthermore, the training process of the UAV active tracking model consists of two stages:

[0029] The first stage is the task understanding stage, where the UAV agent tracks a target moving in a straight line without obstructions or visual interference, and understands the task goal of keeping the target in the center of the image.

[0030] The second stage is the visual generalization stage. Based on the task understanding in the first stage, the UAV intelligent agent improves its visual generalization ability in maps with complex visual features and diverse target behaviors.

[0031] Furthermore, an entropy loss term is added to the loss function of the training model:

[0032] Among them, the loss function expression of the actor is as follows:

[0033]

[0034] Where r t is the gradient update speed, is the entropy loss term, is the advantage function, ∈ is the truncation parameter, and β is the coefficient of the entropy loss term; represents mathematical expectation; clip represents the truncation function;

[0035] The loss function of the critic is expressed as follows:

[0036]

[0037] Where, represents the mathematical expectation about time t; V(·) represents the state value function; γ is the discount factor, s t is the state at time step t.

[0038] The second technical solution adopted by the present invention is:

[0039] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-mentioned active tracking method for open-world drones across scenarios.

[0040] The fourth technical solution adopted by the present invention is:

[0041] A computer-readable storage medium stores at least one instruction, at least one program, a code set, or an instruction set, which is loaded and executed by a processor to implement the above-mentioned active tracking method for open-world drones across scenarios.

[0042] The fifth technical solution adopted by the present invention is:

[0043] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above-mentioned active tracking method for open-world drones across different scenarios.

[0044] The present invention utilizes a proximal policy optimization (PPO) algorithm and a curriculum learning strategy to train a drone active tracking agent, enabling it to adapt to the complex visual features of an open world. It also designs a new reward function to address the limitations of existing models in their ability to adapt to diverse tracking targets. This invention will have a significant impact on active visual tracking technology, which is widely used in security monitoring, autonomous driving, and robotic services. It provides an important data generation platform and improves the performance and adaptability of tracking models. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 This is a flowchart of the steps of an active tracking method for open-world drones across scenarios in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0048] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise clearly defined, words such as setting, installing, and connecting should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0049] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0050] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0051] In the description of this application, "and / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0052] Explanation of terms:

[0053] SUMO: The abbreviation of Simulation of Urban Mobility, is an open source, highly portable, microscopic and continuous multimodal traffic simulation software that can handle the modeling and simulation of large-scale transportation networks.

[0054] With the continuous advancement of deep learning technology, visual tracking tasks have gained widespread application in fields such as security surveillance and autonomous driving. Deep learning models have made significant progress in this field. Emerging methods such as correlation filtering, Siamese network models, and attention-based mechanisms have brought significant breakthroughs in visual tracking. These visual tracking models utilize video data and target box labels for network learning, performing training and prediction under a fixed preset viewpoint. While these models achieve high tracking success rates in video datasets, they are unable to generate control signals for the camera lens based on visual observations, which limits the applicability of current visual tracking algorithms in real-world scenarios. To address this issue, researchers have proposed active visual tracking tasks and are working to improve tracking methods so that visual tracking models can generate control signals based on visual observations. One approach is to integrate these visual tracking models with control algorithms in series to introduce control signals. Another approach is to construct a simulation environment with controllable camera viewpoints and train models end-to-end to output control signals based on image information. However, current solutions still face three challenges: a lack of a unified benchmark construction method, poor adaptability to large environments with complex interference, and poor adaptability to tracking targets with diverse behaviors.

[0055] To address the above issues, the present invention proposes a unified open-world drone active tracking benchmark construction scheme and designs a drone active tracking algorithm based on a reinforcement learning algorithm, which can adapt to complex scene conditions and target motion. This reinforcement learning-based drone active tracking algorithm uses reinforcement learning combined with curriculum learning to address the current algorithm's poor adaptability to large-scale environments with complex interference. It also designs a new reward function to address the limitations of existing models' poor adaptability to tracking targets with diverse behaviors. This invention will have a significant impact on visual active tracking technology, which is widely used in security monitoring, autonomous driving, and robotic services. It provides an important data generation platform and improves the performance and adaptability of tracking models.

[0056] For example, a UAV active tracking simulation system was created with 24 complex open-world environments, 2 trackers, and 24 commonly used tracking targets. It can be used for cross-scene and cross-domain testing and provides a platform for various visual active tracking tasks. Then, a reward function was designed to accurately reflect the tracking performance when the UAV's perspective changes. Finally, the UAV active tracking agent was trained based on the Proximal Policy Optimization (PPO) algorithm and curriculum learning strategy.

[0057] Example 1

[0058] like Figure 1 As shown, this embodiment provides an active tracking method for open-world drones across scenarios, including the following steps:

[0059] S1. Create a diverse open world environment and obtain an open world drone active tracking system.

[0060] In some embodiments, Webots (open source robot simulation software) is used as the system base. Step S1 specifically includes the following steps:

[0061] S11. We selected six environmental scenarios: lake, town, city, village, desert, and farmland, covering most outdoor scenarios in real-world applications. We also selected four weather conditions: sunny, night, foggy, and snowy. This resulted in a total of 24 scene maps for this system.

[0062] S12. For lake, town, and downtown maps, we used OpenStreetMap to capture real-world blocks and converted them into 3D assets in Webots. For village, desert, and farmland maps, we used Creo to directly model the elements within the scene.

[0063] S13. After completing the 3D asset construction, manually modify the current scene assets according to the needs of different scenes and add other scene elements to increase the visual interference of the environment. In addition, according to different weather conditions, manually configure the scene lighting conditions, visibility, background color and other physical properties to make them conform to the characteristics of the corresponding weather.

[0064] After constructing steps S11 to S13 above, an open-world drone active tracking system is obtained. This system provides 24 highly complex scene maps, fully demonstrating the cross-scene and cross-domain adaptability of the intelligent agent.

[0065] S2. Create common trackers and tracking targets, design target management solutions, and provide common sensor data.

[0066] To accommodate a variety of mission settings, this example integrates commonly used trackers and tracking target models. Specifically, it provides trackers for ground robots and drones, as well as 24 common tracking targets across five categories: cars, motorcycles, pedestrians, wheeled robots, and legged robots.

[0067] As an implementation, a plug-and-play custom interface is provided, allowing users to build task flows using their own models and controllers. In addition to the diversity of tracked targets, the randomness and rationality of their actions are also key to modeling the complexity of reality. Therefore, this system utilizes SUMO (a traffic simulation engine) to uniformly manage tracked targets within the map, enabling automatic configuration of target appearance, behavior, and interactions between targets.

[0068] In addition to visual image information, real-world visual active tracking systems rely on multi-sensor fusion to improve system performance. Therefore, this system provides a total of six commonly used visual and motion sensors, providing a unified platform for various task requirements.

[0069] S3. Encapsulate the active tracking system into a universal interface to adapt to the current mainstream reinforcement learning framework.

[0070] To ensure the ease of use and versatility of this system, this embodiment encapsulates the above environment into a Gymnasium (an open source reinforcement learning platform that provides a standardized environment interface) interface to achieve lightweight configuration of the above functions and adapt it to existing mainstream reinforcement learning frameworks (such as Stable-baselines3 and Tianshou reinforcement learning framework).

[0071] S4. Build a drone active tracking model and design a reward function.

[0072] The current algorithm assumes the tracker's viewpoint is fixed, horizontally and forward, significantly limiting the tracker's perception range and making it less adaptable to targets with diverse behaviors. Furthermore, the current solution's reward function, designed under the assumption of a fixed forward viewpoint, fails to accurately reflect tracking performance when the tracker's viewpoint changes, significantly hindering agent learning. To train agents to adapt to diverse tracking targets, this embodiment designs a reward function at the image level to ensure that it accurately represents tracking performance even when the tracker's viewpoint changes.

[0073] To accurately reflect tracking performance, the reward function must ensure that targets farther from the image center receive lower rewards. Therefore, the tracking target and image must first be aligned on the same plane for calculations, i.e., the image's projection on the ground is obtained. Then, within the projection quadrilateral, a reward function is constructed that decays according to the distance from the image center to meet this requirement. The reward function expression is as follows:

[0074]

[0075] Where x represents the distance from the target to the center of the image. When the target is outside the image, the reward function is 0.

[0076] S5. Based on the reward function, the proximal policy optimization (PPO) reinforcement learning algorithm is used to train the drone active tracking model.

[0077] Specifically, based on the above-mentioned reward function, this embodiment proposes a method for training a drone active tracking model using a proximal policy optimization (PPO) reinforcement learning algorithm. In order to enable the tracking model to improve its adaptability to large environments with complex interference, this embodiment designs a training strategy based on curriculum learning to gradually improve the performance of the model. The training strategy is divided into two stages. The first stage is the task understanding stage. In the absence of obstructing objects and visual interference, the drone agent tracks a target moving in a straight line and understands the task goal of keeping the target in the center of the image. The second stage is the visual generalization stage. Based on the task understanding in the first stage, the drone agent improves its visual generalization ability in maps with complex visual features and diverse target behaviors.

[0078] During both phases of training, this example uses the Proximal Policy Optimization (PPO) reinforcement learning algorithm for model training. To enhance the agent's exploration capabilities, an entropy loss term is added to the loss function. This drone active tracking model consists of three components: a backbone network, an actor network, and a critic network.

[0079] Among them, the loss function expression of the actor is as follows:

[0080]

[0081] Among them, r t is the gradient update speed, is the entropy loss term, is the advantage function, ∈ is the truncation parameter, and β is the coefficient of the entropy loss term.

[0082] In addition, the loss function of the critic is expressed as follows:

[0083]

[0084] S6. Obtain an input image, input the input image into the trained UAV active tracking model, and output a tracking signal for controlling the UAV.

[0085] After platform construction from step S1 to step S3 and model training from step S4 to step S5, a UAV active tracking model has been successfully developed, which can actively generate UAV control signals based on visual observations.

[0086] In summary, the present invention proposes a multi-scenario, high-complexity UAV active tracking simulation system, provides multiple trackers, tracking targets, and provides a platform for multiple visual active tracking tasks. In addition, the invention proposes a UAV active tracking method based on reinforcement learning, which adopts reinforcement learning combined with curriculum learning to solve the problem that the current algorithm has poor adaptability to large-scale environments with complex interference, and designs a new reward function to solve the limitation of the existing model's poor adaptability to tracking targets with diverse behaviors. The present invention will have a significant impact on visual active tracking technology that is widely used in security monitoring, autonomous driving, and robotic services, and provides an important data generation platform, improving the performance and adaptability of the tracking model.

[0087] Example 2

[0088] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 1 An active tracking method for open-world drones across scenes is shown.

[0089] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0090] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0091] Since the electronic device is an electronic device corresponding to an active tracking method for open-world drones across scenarios in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0092] Example 3

[0093] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 An active tracking method for open-world drones across scenes is shown.

[0094] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0095] Since the storage medium is the storage medium corresponding to the active tracking method for open-world drones across scenes in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0096] Example 4

[0097] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to perform the steps of the method for active tracking of open-world drones across scenarios according to various exemplary embodiments of the present application described above in this specification. The executable computer program code or "code" used to perform the various embodiments may be written in a high-level programming language such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0098] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0099] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0100] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. An active tracking method for open-world drones across scenarios, characterized by: The following steps are involved: Create diverse open world environments and gain access to an open world drone active tracking system; Create trackers and tracking targets, and design target management plans; Encapsulate the active tracking system into a universal interface to adapt to the reinforcement learning framework; Build a drone active tracking model and design a reward function; Based on the reward function, the proximal policy optimization reinforcement learning algorithm is used to train the UAV active tracking model; Get the input image, feed it into the trained UAV active tracking model, and output the tracking signal to control the UAV.

2. The method for active tracking of open-world drones across scenarios according to claim 1, characterized in that: The creation of a diverse open-world environment and the acquisition of an open-world drone active tracking system include: Multiple scene maps were constructed based on preset environmental scenarios and weather conditions. Preset environmental scenarios included lakes, rural areas, urban areas, mountains, deserts, and farmlands. Weather conditions included sunny, dark, foggy, and snowy days. For lakes, rural areas, and urban area maps, blocks from the real scenes were captured using the OpenStreetMap platform and converted into 3D assets in the Webots software. For mountain, desert, and farmland maps, elements within the scenes were directly modeled using Creo software. After obtaining the 3D assets, modify the current scene assets according to different scene requirements, and add other scene elements to increase the visual interference of the environment to obtain the open world drone active tracking system.

3. The method for active tracking of open-world drones across scenarios according to claim 1, characterized in that: The trackers include two types of trackers: ground robots and drones. The tracking targets include cars, motorcycles, pedestrians, wheeled robots and legged robots, with a total of 24 common tracking targets in 5 categories. Use SUMO to uniformly manage tracking targets within the map, and automatically configure target appearance, behavior, and interactions between targets. Provides a variety of commonly used vision and motion sensors, providing a unified platform for various mission requirements.

4. The method for active tracking of open-world drones across scenarios according to claim 1, characterized in that: The active tracking system is encapsulated into a universal interface, including: The active tracking system is encapsulated into a Gymnasium interface to achieve lightweight configuration of functions and adapt it to existing mainstream reinforcement learning frameworks.

5. The method for active tracking of open-world drones across scenarios according to claim 1, characterized in that: The design reward function includes: The tracking target and image are unified in the same plane for calculation, that is, the projection quadrilateral of the image on the ground is obtained; within the projection quadrilateral, the reward function is constructed according to the principle of distance attenuation from the image center. The expression of the reward function is as follows: Where x represents the distance between the target and the center of the image; α is the attenuation factor, which controls the attenuation of the reward function; I target To track the position of the target in the image, I clip The image cropping area is used to prevent the drone from keeping the target at the edge of the image for a long time. When the target is outside the image, the reward function is 0.

6. The method for active tracking of open-world drones across scenarios according to claim 1, characterized in that: The training process of the drone active tracking model consists of two stages: The first stage is the task understanding stage, where the UAV agent tracks a target moving in a straight line without obstructions or visual interference, and understands the task goal of keeping the target in the center of the image. The second stage is the visual generalization stage. Based on the task understanding in the first stage, the UAV intelligent agent improves its visual generalization ability in maps with complex visual features and diverse target behaviors.

7. The method for active tracking of open-world drones across scenarios according to claim 1, characterized in that: Add an entropy loss term to the loss function of the training model: Among them, the actor's loss function expression is as follows: Where r t is the gradient update speed, is the entropy loss term, is the advantage function, ∈ is the truncation parameter, and β is the coefficient of the entropy loss term; represents mathematical expectation; clip represents the truncation function; The loss function of the critic is expressed as follows: Where, represents the mathematical expectation about time t; V(·) represents the state value function; γ is the discount factor, s t is the state at time step t.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises computer instructions, which are used to perform the method according to any one of claims 1 to 7 when executed by a processor.

Images