Unmanned aerial vehicle infrared target positioning system fusing multi-model KF algorithm and deep learning

Abstract

The invention provides an unmanned aerial vehicle infrared target positioning system fusing multi-model Kalman filtering and deep learning, and relates to the technical field of unmanned aerial vehicle navigation and positioning. The main body system comprises a data acquisition module, a multi-source data fusion module, a feature extraction enhancement module, a dynamic model optimization module, a target state estimation module, a positioning calculation module and a flight decision planning module, and the data acquisition module is provided with a high-resolution infrared sensor and accurately captures target infrared radiation information; a high-definition vision camera and a high-precision laser radar are integrated at the same time, environment data are collected in an omnibearing mode, and the data collection module can collect the environment data according to the environment complexity and the real-time change of the target motion state by adopting a self-adaptive sampling strategy based on fuzzy logic, particle swarm optimization and a self-adaptive threshold value. And the sampling frequency and the resolution of the sensor are dynamically and intelligently adjusted.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle navigation and positioning technology, and specifically to an unmanned aerial vehicle infrared target positioning system integrating a multi-model KF algorithm (Kalman filtering) and deep learning. Background Art

[0002] According to a UAV precise positioning system with Chinese patent number CN105467416A, it includes a flight control system, a GPS positioning system and an ultrasonic positioning system, wherein the flight control system is connected to an infrared receiver, a GPS positioning system and an ultrasonic positioning system respectively, and the infrared receiver is connected to a remote controller. Before the present invention reaches an area where precise landing is required, the GPS positioning system guides the flight control system according to manual remote control; when the area where precise landing is required is reached, the ultrasonic positioning system is automatically turned on, the ultrasonic receiver and the ultrasonic transmitter continuously transmit signals, the flight control system continuously measures the distance between them, the flight control system updates the target position, and adjusts the flight attitude at any time. Therefore, the GPS positioning system of the present invention realizes basic positioning, and the ultrasonic positioning system realizes precise positioning. The GPS positioning system and the ultrasonic positioning system can realize precise positioning of the UAV, so that the UAV can be accurately placed in a specific area.

[0003] At present, there are many drone positioning technologies on the market, among which the GPS positioning system based on satellite signals and the ultrasonic positioning system using the propagation characteristics of sound waves are more common. For example, a typical drone precision positioning system includes a flight control system, a GPS positioning system and an ultrasonic positioning system. The system switches the GPS positioning system and the ultrasonic positioning system through the switch on the remote control, uses the signal interaction between the ultrasonic transmitter and the receiver, measures the distance by measuring the time difference, and realizes positioning in combination with the triangulation positioning algorithm. For example, when the distance detection module detects that the distance between the drone and the precise landing area is less than ten meters, the flight control system turns on the ultrasonic positioning system, the ultrasonic transmitter is set at a right angle, and the controller uploads the measured distance to the flight control system through the wireless communication module, and then adjusts the flight attitude.

[0004] However, the existing technology has exposed the following problems in practical applications: 1. Although the GPS positioning system can achieve relatively accurate positioning in open spaces, it is affected by the propagation characteristics of satellite signals. In environments where signals are easily blocked or interfered, such as urban high-rise canyons, dense forests, and indoors, the positioning accuracy will drop significantly, and even signal loss will occur, resulting in the drone being unable to accurately determine its own position, seriously affecting the execution of the mission; 2. As for the ultrasonic positioning system, its range of action is relatively limited, and it is generally suitable for short-distance and small-scale precise landing positioning. In addition, ultrasonic signals are easily interfered by environmental factors during propagation, such as air flow, temperature changes, humidity fluctuations, etc. These factors will change the propagation speed and direction of sound waves, which will lead to deviations in the measured distance, making it difficult to ensure positioning accuracy; 3. Existing positioning systems often rely only on a single technology or a simple combination of technologies, lacking comprehensive consideration and adaptive capabilities for complex environments and diversified task requirements. For example, in the face of complex multi-source data fusion, dynamic environmental changes, and high-precision target positioning, existing technologies are difficult to meet the needs of efficient, accurate, and reliable positioning.

[0005] Therefore, a UAV infrared target positioning system that integrates multi-model Kalman filtering and deep learning is needed to solve the above problems. Summary of the invention

[0006] Technical issues solved

[0007] In view of the shortcomings of the prior art, the present invention provides a UAV infrared target positioning system that integrates multi-model Kalman filtering and deep learning, which solves the problems mentioned in the above background technology.

[0008] Technical Solution

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: an infrared target positioning system for unmanned aerial vehicles that integrates multi-model Kalman filtering and deep learning, including a main system, wherein the main system includes a data acquisition module, a multi-source data fusion module, a feature extraction and enhancement module, a dynamic model optimization module, a target state estimation module, a positioning solution module and a flight decision planning module, wherein the data acquisition module deploys a high-resolution infrared sensor to accurately capture the target infrared radiation information, and integrates a high-definition visual camera and a high-precision laser radar to collect environmental data in all directions, and adopts an adaptive sampling strategy based on fuzzy logic, particle swarm optimization and adaptive threshold, and dynamically adjusts the sampling frequency and resolution of the sensor according to the complexity of the environment and the motion state of the target; wherein fuzzy logic is used to evaluate the degree of fuzziness of the complexity of the environment and the motion state of the target, particle swarm optimization is used to find the optimal solution among multiple possible sampling parameter combinations, and the adaptive threshold dynamically adjusts the triggering conditions of the sampling according to the rate of change of the data, while ensuring the data quality, reducing data redundancy and processing burden, and ensuring the richness and integrity of the data.

[0010] Preferably, the multi-source data fusion module uses an adaptive fusion algorithm that integrates biological neuron coordination mechanism, tensor decomposition and deep learning attention mechanism to simulate information interaction and integration between neurons and perform feature encoding on infrared, visual and lidar data, wherein tensor decomposition decomposes multi-source data from high-dimensional space into low-dimensional subspace to extract core features of the data, and the deep learning attention mechanism dynamically adjusts the weight of each sensor data according to the dynamic changes of the environment through competitive learning and collaborative feedback to achieve deep fusion of multi-source data and provide a high-quality data foundation for subsequent processing. At the same time, cross-modal consistency constraints are introduced to ensure that the consistency and complementarity of information of various modal data are maintained during the fusion process by constructing a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information;

[0011] The feature extraction enhancement module adopts a convolutional neural network architecture that integrates quantum enhancement, generative adversarial network and residual network, and uses the superposition and entanglement characteristics of quantum states to accelerate the feature extraction process. The generator in the generative adversarial network generates simulated feature data, and the discriminator distinguishes between real and simulated features. More representative features are mined through adversarial training. The residual network solves the gradient vanishing problem in the deep network through jump connections, enhances the network's ability to extract complex features, and combines the attention mechanism with the improved void convolution technology to focus on the key features of the target, expand the receptive field, and enhance the ability to extract target features in complex scenes.

[0012] The dynamic model optimization module introduces a model optimization strategy based on generative adversarial networks (GANs), transfer learning, and genetic algorithms, wherein the generator generates simulated target and scenario data, the discriminator distinguishes between real and simulated data, and the generalization ability of the model is improved through adversarial training. The transfer learning transfers the model knowledge trained in similar scenarios to accelerate the convergence speed of the model in new scenarios. The genetic algorithm searches for the optimal model configuration in the model space by encoding, selecting, crossing, and mutating the structure and parameters of the model, thereby realizing rapid optimization and adaptive adjustment of the model.

[0013] The target state estimation module uses an algorithm that integrates reinforcement learning, adaptive Kalman filtering and particle filtering to model the target state estimation problem as a Markov decision process. Through the reward function for the drone target positioning scenario, the intelligent agent is guided to learn the optimal state estimation strategy. The adaptive Kalman filter dynamically adjusts the filter parameters according to the rapid changes in the target motion and the interference of complex environmental noise. The particle filter estimates the target state by sampling and updating the weights of a large number of particles.

[0014] The positioning solution module is based on multi-view data fusion and improved triangulation principle, combined with the real-time attitude information of the UAV, and integrates the mechanisms of dynamic coordinate transformation, error compensation and least squares fitting. By real-time monitoring of the flight attitude of the UAV and changes in environmental parameters, the coordinate system in the positioning process is dynamically adjusted, and the measurement error is compensated online. The least squares fitting is used to find the optimal positioning solution in multiple sets of measurement data, adapt to the flight attitude changes of the UAV and the dynamic interference of the environment in real time, and achieve high-precision three-dimensional positioning of the target;

[0015] The flight decision planning module adopts a dual-depth Q network architecture based on deep reinforcement learning, ant colony algorithm and A algorithm, combined with experience replay and priority experience replay mechanism, effectively reducing the overfitting problem of Q value estimation and accelerating the learning process. The deep reinforcement learning will learn the optimal flight strategy, in which the ant colony algorithm simulates the pheromone transmission in the process of ant foraging to find the potential optimal path in the path search space, and the A algorithm uses heuristic functions to quickly search for the optimal path from the current position to the target position, while considering the energy consumption of the drone, flight safety constraints and environmental dynamic factors, and planning the optimal flight trajectory in real time. Ensure that the drone can achieve long endurance and safe flight while efficiently tracking the target.

[0016] Preferably, in the multi-source data fusion module, an adaptive fusion algorithm based on biological neuron synergy mechanism, tensor decomposition and deep learning attention mechanism constructs a neuron connection weight matrix to simulate the excitation and inhibition relationship between neurons, and combines the low-dimensional feature representation after tensor decomposition and the weight distribution of the attention mechanism to achieve efficient fusion of sensor data, thereby improving the system's adaptability to complex and changing environments and the accuracy of target perception.

[0017] Preferably, the parallel computing capability of the quantum bits in the feature extraction enhancement module simultaneously explores the optimal solution direction during the parameter update process of the convolution kernel, and the adversarial training mechanism of the generative adversarial network prompts the network to mine representative features, wherein the skip connection of the residual network ensures the effective extraction of deep-level features by the network and accelerates the convergence of the model. It can also mine deep-level target features that are difficult to discover with traditional methods.

[0018] Preferably, in the dynamic model optimization module, the generative adversarial network uses adversarial training to make the data distribution generated by the generator gradually approach the real data distribution. The transfer learning reduces the training time and sample requirements of the model in the new scenario through feature mapping and knowledge transfer between the source domain and the target domain. The genetic algorithm further improves the performance and adaptability of the model by optimizing the model structure and parameters.

[0019] Preferably, the target state estimation module uses reinforcement learning to guide the agent to select the optimal estimation strategy under the target motion mode and environmental noise conditions, and the adaptive Kalman filter quickly adjusts the filter parameters, wherein the particle filter improves the estimation accuracy through particle sampling and weight update, so that the estimation error is kept within a small range, effectively improving the stability and reliability of the target state estimation.

[0020] Preferably, when considering the energy consumption of the UAV, the flight decision planning module establishes an energy model to associate factors such as flight speed, altitude, and attitude with energy consumption. When planning the flight trajectory, deep reinforcement learning, ant colony algorithm, and A algorithm are used to give priority to paths with low energy consumption. When considering flight safety constraints, environmental perception data is combined to avoid obstacles and dangerous areas to ensure the safety of UAV flight.

[0021] Preferably, when collecting data, the data acquisition module dynamically and intelligently adjusts the sampling frequency and resolution of the sensor according to the real-time changes in the complexity of the environment and the target's motion state, thereby minimizing data redundancy and processing burden while ensuring data quality.

[0022] Preferably, the multi-source data fusion module introduces a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information during the data fusion process to measure the similarity and complementarity between different modal data from various angles, ensuring that different modal data maintain a high degree of consistency and complementarity of information during the fusion process, thereby further improving the quality and reliability of the fused data.

[0023] The different modal data involved here include infrared radiation data collected by infrared sensors, which can reflect the thermal characteristic distribution of the target and play a key role in identifying the heat-generating parts and temperature differences of the target; visible light image data acquired by high-definition visual cameras can present the shape, color, texture and other appearance characteristics of the target, which helps to identify the category and details of the target from a visual perspective; distance data collected by high-precision lidar can construct three-dimensional spatial structure information of the target and its surrounding environment, and accurately depict the relative distance between the target's position and surrounding objects.

[0024] This module encodes the features of the above-mentioned different modal data by simulating the information interaction and integration between neurons. Tensor decomposition is used to decompose multi-source data from high-dimensional space into low-dimensional subspaces to extract the core features of the data. The deep learning attention mechanism dynamically adjusts the weight of each sensor data through competitive learning and collaborative feedback according to the dynamic changes of the environment, such as weather changes, lighting changes, and targets entering different scenes, to achieve deep fusion of multi-source data and provide a high-quality data foundation for subsequent processing.

[0025] At the same time, cross-modal consistency constraints are introduced, and a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information is constructed to ensure that different modal data maintain information consistency and complementarity during the fusion process. Specifically, Wasserstein distance is used to measure the difference between the distribution of different modal data to ensure the consistency of the data in the overall distribution; cosine similarity measures the similarity between different modal feature vectors from the perspective of vector angle to capture the similarity of features; mutual information evaluates the degree of information sharing between different modal data to highlight the complementarity of data.

[0026] Beneficial Effects

[0027] The present invention provides a UAV infrared target positioning system integrating multi-model Kalman filtering and deep learning. It has the following beneficial effects:

[0028] 1. The data acquisition module of the present invention adopts an adaptive sampling strategy based on fuzzy logic, particle swarm optimization and adaptive threshold, which can dynamically and intelligently adjust the sampling frequency and resolution of the sensor according to the real-time changes of environmental complexity and target motion state. This method effectively avoids data redundancy, greatly reduces the burden of subsequent processing while ensuring data quality. The multi-source data fusion module uses an adaptive fusion algorithm that integrates biological neuron synergy mechanism, tensor decomposition and deep learning attention mechanism, which can not only efficiently encode different modal data such as infrared, vision, and lidar, but also extract core features through tensor decomposition, dynamically adjust data weights in combination with deep learning attention mechanism, and achieve deep fusion. At the same time, the cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information is introduced to ensure the high consistency and complementarity of different modal data in the fusion process, provide a high-quality data foundation for subsequent modules, and significantly improve the accuracy and efficiency of the entire system in acquiring and processing target and environmental information.

[0029] 2. The feature extraction enhancement module in the present invention adopts a convolutional neural network architecture that integrates quantum enhancement, generative adversarial network and residual network, showing unique advantages. Quantum enhancement uses the superposition and entanglement characteristics of quantum states to accelerate the feature extraction process when the convolution kernel parameters are updated. The generative adversarial network mines more representative features through adversarial training of the generator and the discriminator, while the jump connection of the residual network solves the problem of gradient disappearance in the deep network and enhances the ability to extract complex features. Combined with the attention mechanism and the improved hole convolution technology, the key features of the target are further focused and the receptive field is expanded. The dynamic model optimization module introduces an optimization strategy based on generative adversarial networks, transfer learning and genetic algorithms. Generative adversarial networks improve the generalization ability of the model, transfer learning accelerates the convergence of the model in new scenarios, and the genetic algorithm searches for the optimal model configuration to achieve rapid optimization and adaptive adjustment of the model. These modules work together to enable the system to accurately extract deep-level features of the target, quickly adapt to different scenarios, and significantly improve the overall performance and adaptability of the model.

[0030] 3. The target state estimation module in the present invention uses an algorithm that integrates reinforcement learning, adaptive Kalman filtering and particle filtering to model the target state estimation problem as a Markov decision process, and guides the intelligent agent to learn the optimal estimation strategy through a specific reward function. The adaptive Kalman filter can dynamically adjust the filter parameters according to the target motion changes and environmental noise interference. The particle filter improves the estimation accuracy through particle sampling and weight updating, effectively improving the stability and reliability of the target state estimation, and keeping the estimation error within a very small range. The positioning solution module is based on multi-perspective data fusion and improved triangulation principles, combined with the real-time attitude information of the drone, using dynamic coordinate transformation, error compensation and least squares fitting mechanism, to adapt to the changes in the flight attitude of the drone and the dynamic interference of the environment in real time, and achieve high-precision three-dimensional positioning of the target. This high-precision target state estimation and positioning capability enables the system to accurately lock the target position even in a complex environment, providing a reliable basis for subsequent flight decisions.

[0031] 4. The flight decision planning module in the present invention adopts a dual-depth Q network architecture based on deep reinforcement learning, ant colony algorithm and A algorithm, combined with experience replay and priority experience replay mechanism, effectively reducing the overfitting problem of Q value estimation, accelerating the learning process, and learning the optimal flight strategy through deep reinforcement learning. The ant colony algorithm and A algorithm respectively simulate the transmission of ant foraging pheromones and use heuristic functions to find the potential optimal path in the path search space. At the same time, considering the energy consumption, flight safety constraints and environmental dynamic factors of the drone, the flight parameters and energy consumption are associated by establishing an energy model, and low-energy consumption paths are given priority when planning the flight trajectory, and obstacles and dangerous areas are avoided in combination with environmental perception data. This enables the drone to intelligently plan the optimal trajectory during flight, while efficiently tracking the target, achieving long endurance and safe flight, greatly improving the practicality and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the main system framework diagram of the present invention;

[0033] Figure 2 A flow chart for use of the system of the present invention;

[0034] Figure 3 It is a data acquisition simulation diagram of the present invention;

[0035] Figure 4 This is a normalized data processing simulation diagram of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Specific embodiment one:

[0038] like Figure 1-4 As shown, a UAV infrared target positioning system integrating multi-model Kalman filtering and deep learning includes a main system, which includes a data acquisition module, a multi-source data fusion module, a feature extraction and enhancement module, a dynamic model optimization module, a target state estimation module, a positioning solution module and a flight decision planning module. The data acquisition module deploys a high-resolution infrared sensor to accurately capture the target infrared radiation information, and integrates a high-definition visual camera and a high-precision laser radar to collect environmental data in all directions. An adaptive sampling strategy based on fuzzy logic, particle swarm optimization and adaptive threshold is adopted to dynamically adjust the sampling frequency and resolution of the sensor according to the complexity of the environment and the motion state of the target; wherein fuzzy logic is used to evaluate the complexity of the environment and the fuzziness of the motion state of the target, particle swarm optimization is used to find the optimal solution among multiple possible sampling parameter combinations, and the adaptive threshold dynamically adjusts the triggering conditions of the sampling according to the rate of change of the data, while ensuring the data quality, reducing data redundancy and processing burden, and ensuring the richness and integrity of the data.

[0039] The multi-source data fusion module uses an adaptive fusion algorithm that integrates biological neuron coordination mechanism, tensor decomposition and deep learning attention mechanism to simulate the information interaction and integration between neurons and perform feature encoding on infrared, visual and lidar data. Tensor decomposition decomposes multi-source data from high-dimensional space into low-dimensional subspace to extract the core features of the data. The deep learning attention mechanism dynamically adjusts the weight of each sensor data according to the dynamic changes of the environment through competitive learning and collaborative feedback to achieve deep fusion of multi-source data and provide a high-quality data foundation for subsequent processing. At the same time, cross-modal consistency constraints are introduced to ensure that the consistency and complementarity of information of various modal data are maintained during the fusion process by constructing a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information.

[0040] The feature extraction enhancement module adopts a convolutional neural network architecture that integrates quantum enhancement, generative adversarial network and residual network. It uses the superposition and entanglement characteristics of quantum states to accelerate the feature extraction process. The generator in the generative adversarial network generates simulated feature data, and the discriminator distinguishes between real and simulated features. More representative features are mined through adversarial training. The residual network solves the gradient vanishing problem in deep networks through jump connections, enhances the network's ability to extract complex features, and combines the attention mechanism with the improved dilated convolution technology to focus on key target features, expand the receptive field, and enhance the ability to extract target features in complex scenarios.

[0041] The dynamic model optimization module introduces a model optimization strategy based on generative adversarial networks (GANs), transfer learning, and genetic algorithms. The generator generates simulated target and scenario data, and the discriminator distinguishes between real and simulated data. Adversarial training is used to improve the generalization ability of the model. Transfer learning transfers model knowledge trained in similar scenarios to accelerate the convergence speed of the model in new scenarios. The genetic algorithm searches for the optimal model configuration in the model space by encoding, selecting, crossing, and mutating the structure and parameters of the model, thereby achieving rapid optimization and adaptive adjustment of the model.

[0042] The target state estimation module uses an algorithm that integrates reinforcement learning, adaptive Kalman filtering and particle filtering to model the target state estimation problem as a Markov decision process. Through the reward function for the drone target positioning scenario, it guides the intelligent agent to learn the optimal state estimation strategy. The adaptive Kalman filter dynamically adjusts the filter parameters according to the rapid changes in the target motion and the interference of complex environmental noise. The particle filter estimates the target state by sampling a large number of particles and updating the weights.

[0043] The positioning solution module is based on multi-view data fusion and improved triangulation principle, combined with the real-time attitude information of the UAV, and integrates the mechanisms of dynamic coordinate transformation, error compensation and least squares fitting. By real-time monitoring of the flight attitude of the UAV and changes in environmental parameters, the coordinate system in the positioning process is dynamically adjusted, and the measurement error is compensated online. The least squares fitting is used to find the optimal positioning solution in multiple sets of measurement data, adapt to the flight attitude changes of the UAV and the dynamic interference of the environment in real time, and achieve high-precision three-dimensional positioning of the target;

[0044] The flight decision planning module adopts a dual-depth Q network architecture based on deep reinforcement learning, ant colony algorithm and A algorithm, combined with experience replay and priority experience replay mechanism, which effectively reduces the overfitting problem of Q value estimation and accelerates the learning process. Deep reinforcement learning will learn the optimal flight strategy. The ant colony algorithm simulates the pheromone transmission in the process of ants foraging to find the potential optimal path in the path search space. The A algorithm uses heuristic functions to quickly search for the optimal path from the current position to the target position, while considering the energy consumption of the drone, flight safety constraints and environmental dynamic factors, and planning the optimal flight trajectory in real time. Ensure that the drone can achieve long endurance and safe flight while efficiently tracking the target.

[0045] In the multi-source data fusion module, the adaptive fusion algorithm based on biological neuron synergy mechanism, tensor decomposition and deep learning attention mechanism constructs a neuron connection weight matrix to simulate the excitation and inhibition relationship between neurons, and combines the low-dimensional feature representation after tensor decomposition and the weight distribution of the attention mechanism to achieve efficient fusion of sensor data, thereby improving the system's adaptability to complex and changing environments and the accuracy of target perception.

[0046] The parallel computing capability of quantum bits in the feature extraction enhancement module explores the optimal solution direction during the parameter update process of the convolution kernel. The adversarial training mechanism of the generative adversarial network prompts the network to mine representative features. The skip connection of the residual network ensures the effective extraction of deep-level features and accelerates the convergence of the model. It can also mine deep-level target features that are difficult to discover with traditional methods.

[0047] In the dynamic model optimization module, the generative adversarial network uses adversarial training to make the data distribution generated by the generator gradually approach the real data distribution. Transfer learning reduces the training time and sample requirements of the model in new scenarios through feature mapping and knowledge transfer between the source domain and the target domain. The genetic algorithm further improves the performance and adaptability of the model by optimizing the model structure and parameters.

[0048] In the target state estimation module, reinforcement learning guides the agent to select the optimal estimation strategy under the target motion mode and environmental noise conditions. The adaptive Kalman filter quickly adjusts the filter parameters. The particle filter improves the estimation accuracy through particle sampling and weight update, keeping the estimation error within a small range. This effectively improves the stability and reliability of target state estimation.

[0049] When considering the energy consumption of the UAV, the flight decision planning module establishes an energy model to associate factors such as flight speed, altitude, and attitude with energy consumption. When planning the flight trajectory, it uses deep reinforcement learning, ant colony algorithm, and A algorithm to give priority to paths with low energy consumption. When considering flight safety constraints, it combines environmental perception data to avoid obstacles and dangerous areas to ensure the safety of UAV flight.

[0050] When collecting data, the data acquisition module dynamically and intelligently adjusts the sampling frequency and resolution of the sensor according to the real-time changes in environmental complexity and target motion status, thereby minimizing data redundancy and processing burden while ensuring data quality.

[0051] During the data fusion process, the multi-source data fusion module introduces a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information to measure the similarity and complementarity between different modal data from various angles, ensuring that different modal data maintain a high degree of consistency and complementarity of information during the fusion process, further improving the quality and reliability of the fused data.

[0052] The different modal data involved here include infrared radiation data collected by infrared sensors, which can reflect the thermal characteristic distribution of the target and play a key role in identifying the heat-generating parts and temperature differences of the target; visible light image data acquired by high-definition visual cameras can present the shape, color, texture and other appearance characteristics of the target, which helps to identify the category and details of the target from a visual perspective; distance data collected by high-precision lidar can construct three-dimensional spatial structure information of the target and its surrounding environment, and accurately depict the relative distance between the target's position and surrounding objects.

[0053] This module encodes the features of the above-mentioned different modal data by simulating the information interaction and integration between neurons. Tensor decomposition is used to decompose multi-source data from high-dimensional space into low-dimensional subspaces to extract the core features of the data. The deep learning attention mechanism dynamically adjusts the weight of each sensor data through competitive learning and collaborative feedback according to the dynamic changes of the environment, such as weather changes, lighting changes, and targets entering different scenes, to achieve deep fusion of multi-source data and provide a high-quality data foundation for subsequent processing.

[0054] At the same time, cross-modal consistency constraints are introduced, and a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information is constructed to ensure that different modal data maintain information consistency and complementarity during the fusion process. Specifically, Wasserstein distance is used to measure the difference between the distribution of different modal data to ensure the consistency of the data in the overall distribution; cosine similarity measures the similarity between different modal feature vectors from the perspective of vector angle to capture the similarity of features; mutual information evaluates the degree of information sharing between different modal data to highlight the complementarity of data. Specific embodiment 2:

[0056] like Figure 1-4 As shown, the following is the specific usage process of the content mentioned in Example 1 in the actual usage scenario:

[0057] 1. Data collection stage:

[0058] After the drone takes off, the data acquisition module starts working. The high-resolution infrared sensor continuously scans the surrounding environment and accurately captures the infrared radiation information emitted by the target. For example, at night or in a smoky environment, it can clearly detect the location and general outline of the heating target. The high-definition visual camera also captures visible light images to obtain the appearance details of the target. For example, if the target is a vehicle, its color, shape and other features can be identified. The high-precision laser radar emits a laser beam to measure the distance between the target and surrounding objects and build a three-dimensional space model.

[0059] The adaptive sampling strategy based on fuzzy logic, particle swarm optimization and adaptive threshold runs in real time. The system uses fuzzy logic to evaluate the complexity of the environment and the ambiguity of the target's motion state based on factors such as the density of objects in the environment, light changes, and the target's motion speed and direction changes. For example, when the drone enters a high-rise area in the city, the environmental complexity increases, and the fuzzy logic outputs the corresponding results. The particle swarm optimization algorithm quickly searches for the optimal sampling parameters among many possible sampling frequency and resolution combinations based on the evaluation results of fuzzy logic. If the target motion suddenly accelerates, the particle swarm optimization will adjust the sampling frequency to capture the target dynamics at a higher frequency. The adaptive threshold dynamically determines whether to trigger sampling based on the data change rate such as infrared radiation intensity, changes in target features in visual images, and fluctuations in lidar distance data, ensuring that key data is collected while avoiding data redundancy caused by excessive collection.

[0060] 2. Multi-source data fusion stage:

[0061] The multi-source data fusion module receives infrared, visual and lidar data from the data acquisition module. The adaptive fusion algorithm based on biological neuron synergy mechanism, tensor decomposition and deep learning attention mechanism is started. First, the information interaction between neurons is simulated, and the features of different modal data are encoded to convert the data into a unified processable format.

[0062] The tensor decomposition algorithm decomposes high-dimensional multi-source data into low-dimensional subspaces, extracts core features, reduces the amount of data processing, and retains key information. For example, the features of three-dimensional lidar data and two-dimensional visual image data are integrated and refined.

[0063] The deep learning attention mechanism dynamically adjusts the weight of each sensor data according to dynamic environmental changes such as weather changes, lighting changes, and targets entering different scenes. For example, in a strong light environment, the weight of visual camera data is appropriately reduced, and the weight of infrared sensor and lidar data is increased. At the same time, by constructing a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity, and mutual information, it ensures that different modal data are highly consistent and complementary in terms of overall distribution (Wasserstein distance guarantee), feature vector similarity (cosine similarity measurement), and information sharing degree (mutual information evaluation) during the fusion process, generating high-quality fused data.

[0064] 3. Feature extraction enhancement stage:

[0065] The feature extraction enhancement module obtains the fused data and processes it using a convolutional neural network architecture that integrates quantum enhancement, generative adversarial networks, and residual networks. The quantum enhancement part uses the superposition and entanglement characteristics of quantum states. During convolution operations, the parallel computing power of quantum bits simultaneously explores the optimal solution direction for updating multiple convolution kernel parameters, greatly accelerating the feature extraction process.

[0066] In the generative adversarial network, the generator generates simulated feature data, and the discriminator strives to distinguish between real and simulated features. In the continuous adversarial training, the network is prompted to mine more representative target features. For example, for different types of targets, unique textures, shapes and other features can be accurately extracted. The residual network solves the gradient vanishing problem in deep network training through jump connections, ensuring that the network can effectively extract deep-level features. It combines the attention mechanism with the improved dilated convolution technology to focus on the key features of the target and expand the receptive field, thereby enhancing the ability to extract target features in complex scenes, such as accurately identifying the key parts of the target in a cluttered background.

[0067] 4. Dynamic model optimization stage:

[0068] The dynamic model optimization module adopts a model optimization strategy based on generative adversarial networks (GANs), transfer learning, and genetic algorithms. The generator of the generative adversarial network generates simulated target and scene data, and the discriminator distinguishes between real and simulated data. Through continuous adversarial training, the data distribution generated by the generator gradually approaches the real data distribution, improving the model's generalization ability for various targets and scenes.

[0069] Transfer learning transfers model knowledge trained in similar scenarios (such as similar geographical environments, lighting conditions, target motion patterns, etc.) to the current model, accelerating the convergence of the model in new scenarios. For example, a model previously trained in a mountainous environment can transfer part of its knowledge to a scenario with similar terrain. Genetic algorithms encode, select, crossover, and mutate the model's structure (such as the number of network layers and neurons) and parameters (such as weights and biases), search for the optimal model configuration in the model space, and achieve rapid optimization and adaptive adjustment of the model to meet different target positioning requirements.

[0070] 5. Target state estimation stage:

[0071] The target state estimation module uses an algorithm that integrates reinforcement learning, adaptive Kalman filtering and particle filtering. The target state estimation problem is modeled as a Markov decision process. Through a carefully designed reward function for the drone target positioning scenario, the intelligent agent is guided to learn the optimal state estimation strategy under different states (such as different motion stages of the target, different relative positions of the drone and the target).

[0072] When the target's motion state changes rapidly (such as sudden acceleration and turning) or is disturbed by complex environmental noise (such as electromagnetic interference and weather noise), the adaptive Kalman filter quickly and dynamically adjusts the filter parameters to maintain an accurate estimate of the target state. The particle filter further accurately estimates the target state by sampling and updating the weights of a large number of particles. For example, when the target is temporarily blocked, the distribution and weight changes of the particles can still be used to reasonably infer the position and state of the target.

[0073] 6. Positioning solution stage:

[0074] The positioning solution module works based on multi-view data fusion and improved triangulation principle, combined with the real-time attitude information of the drone (including pitch angle, yaw angle, and roll angle). By real-time monitoring of the drone's flight attitude and environmental parameter changes (such as wind speed, air pressure, etc.), the dynamic coordinate transformation mechanism dynamically adjusts the coordinate system during the positioning process to ensure the accuracy of positioning.

[0075] The error compensation mechanism performs online compensation for sensor measurement errors, calculation errors, etc. The least squares fitting method finds the optimal positioning solution in multiple sets of measurement data, comprehensively considers various factors, and achieves high-precision three-dimensional positioning of the target. For example, when the drone constantly adjusts its attitude and position during flight, it can always accurately calculate the three-dimensional coordinates of the target.

[0076] 7. Flight decision planning stage:

[0077] The flight decision-making and planning module adopts a dual-depth Q network architecture based on deep reinforcement learning, ant colony algorithm and A algorithm. Combined with the experience replay and priority experience replay mechanism, it reduces the overfitting problem of Q value estimation and accelerates the learning process. Deep reinforcement learning learns the optimal flight strategy according to different environments (such as urban environment, mountainous environment, and severe weather environment).

[0078] The ant colony algorithm simulates the pheromone transmission during the foraging process of ants to find the potential optimal path in the path search space. The A algorithm uses heuristic functions to quickly search for the optimal path from the current position to the target position. At the same time, considering the energy consumption of the drone (related to factors such as flight speed, altitude, attitude, air resistance, etc.), flight safety constraints (such as avoiding no-fly zones, obstacles, and bad weather areas), and environmental dynamic factors (such as wind direction changes, sudden changes in the direction of movement of the target), the optimal flight trajectory is planned in real time to ensure that the drone can achieve long endurance and safe flight while efficiently tracking the target. For example, when the drone's battery is low, it will prioritize planning a path with low energy consumption to return; when encountering obstacles ahead, the flight path will be adjusted in time to avoid them. Specific embodiment three:

[0080] like Figure 1-4 As shown, the following is a supplement to the content in Example 1:

[0081] Fuzzy logic membership function: When evaluating the complexity of the environment, taking the urban environment as an example, building density, road density, etc. can be taken into consideration. Assuming that the building density is expressed by the number of buildings per square kilometer, when the number is less than 50, it is defined as "low complexity" with a membership of 0-0.3; when the number is between 50-150, it is defined as "medium complexity" with a membership of 0.3-0.7; when the number is greater than 150, it is defined as "high complexity" with a membership of 0.7-1. For the target motion state, taking the target speed as an example, when the speed is less than 10m / s, it is defined as "low speed" with a membership of 0-0.3; when the speed is between 10-30m / s, it is defined as "medium speed" with a membership of 0.3-0.7; when the speed is greater than 30m / s, it is defined as "high speed" with a membership of 0.7-1. The division of these membership functions can be adjusted according to a large amount of experimental data and experience in actual scenarios.

[0082] Particle swarm optimization algorithm parameters: The number of particles can be between 20 and 100. For example, when optimizing data collection parameters for a small area, the number of particles is set to 30, and the inertia weight is adjusted between 0.4 and 0.9. When the algorithm needs to converge quickly, the inertia weight can be set to 0.4; if the algorithm is expected to have a wider search space, it can be set to 0.9. Learning factors are usually divided into cognitive learning factors and social learning factors, both of which are generally between 1 and 4, such as cognitive learning factors set to 1.5 and social learning factors set to 1.8. They determine the degree to which particles move toward their own historical optimal positions and global optimal positions.

[0083] Adaptive Kalman filter noise covariance matrix: During initialization, for the process noise covariance matrix, if the target motion is relatively stable, its diagonal elements can be set to smaller values, such as [0.01, 0.01, 0.01] (assuming that the state vector is three-dimensional, representing the position, velocity, and acceleration of the target, respectively). For the measurement noise covariance matrix, it is determined according to the sensor accuracy. If the infrared sensor has a high accuracy, its corresponding matrix element can be set to 0.001, while the corresponding element of the sensor with a slightly lower accuracy can be set to 0.01. During the update process, adjustments can be made according to the uncertainty changes in the target motion. When the target motion suddenly accelerates or is subject to greater interference, the value of the matrix can be appropriately increased to better track the target state changes.

[0084] Convolutional neural network architecture parameters: In the feature extraction enhancement module, assume that a network is built to identify small targets. The number of network layers can be set to 10, including 5 convolutional layers and 5 fully connected layers. The convolution kernel size of the convolution layer can be gradually increased from 3x3 to 5x5 to obtain features of different scales. For example, the convolution kernel size of the first three convolutional layers is 3x3, and the last two are 5x5. The number of neurons in each layer and the number of neurons in the convolution layer can be calculated based on the convolution kernel size and the number of input channels to ensure the effectiveness of feature extraction. The number of neurons in the fully connected layer can be set to 512, 256, 128, 64, and 10, respectively, to classify and output the extracted features.

[0085] Genetic algorithm encoding method for model structure: For convolutional neural network structure, binary encoding can be used. For example, use a number of binary digits to represent the number of network layers. Assuming 4 bits are used, a network with 1-16 layers can be represented. For the type of each layer (convolutional layer or fully connected layer), 1 bit is used to represent it, 0 represents the convolutional layer, and 1 represents the fully connected layer. For the size of the convolution kernel, if only 3x3 and 5x5 are considered, 1 bit is used to represent it, 0 represents 3x3, and 1 represents 5x5. For the number of neurons, it can be encoded according to its value range, such as dividing the range of the number of neurons into several intervals, and using the corresponding binary numbers to represent the interval.

[0086] Reward function design: In the drone target positioning scenario of the target state estimation module, the reward function can be designed as follows: when the error of the drone positioning the target is less than 0.5 meters, a reward of +10 is given; when the error is between 0.5-1 meters, a reward of +5 is given; when the error is greater than 1 meter, a reward of -5 is given. If the drone approaches a dangerous area (such as near a high-voltage line) during flight, a reward of -20 is given. When the drone successfully continues to track the target for more than a certain period of time (such as 30 seconds), a reward of +15 is given. At the same time, considering the energy consumption of the drone, a reward of -3 is given for every certain proportion (such as 5%) of power consumed. Such a reward function comprehensively considers factors such as positioning accuracy, safety, and energy consumption, and guides the agent to learn the optimal state estimation strategy. Specific embodiment four:

[0088] like Figure 1-4 As shown, the key algorithm mentioned in Example 1 is analyzed in detail below, including its core mathematical formula and explanation:

[0089] 1. Data acquisition module

[0090] Fuzzy logic: Fuzzy logic is used to evaluate the complexity of the environment and the degree of fuzziness of the self-standard motion state. Taking the evaluation of environmental complexity as an example, assuming that there are multiple influencing factors, such as object density D, light change rate L, scene change frequency S, etc., these factors are mapped to the [0,1] interval through the fuzzy membership function. For example, for object density D, the membership function μ D (x) can be expressed as:

[0091]

[0092] Among them, D min and D max are the minimum and maximum values ​​of the object density respectively. Then, the fuzzy value F of the environment complexity is calculated by fuzzy inference rules, such as weighted average method:

[0093] F=w1μ D (D)+w2μ L (L)+w3μ S (S)

[0094] Among them, w1, w2, and w3 are the weights of each factor. In the system, the fuzzy value is used as the basis for the subsequent particle swarm optimization algorithm to adjust the sampling parameters, helping the system to reasonably adjust the data collection frequency and resolution according to the complexity of the environment.

[0095] Particle Swarm Optimization Algorithm: Particle Swarm Optimization Algorithm is used to find the optimal solution among multiple possible sampling parameter combinations. The position x of each particle i in the n-dimensional search space is i =(x i1 ,x i2,…,x in ) represents a set of sampling parameters, speed v i =(v i1 ,v i2 ,…,v in ). The particle updates its position and velocity by tracking two extreme values: individual extreme value pbest i And the global extreme value gbest.

[0096] The speed update formula is:

[0097] v ij (t+1)=ωv ij (t)+c1r1(p ij -x ij (t))+c2r2(g j -x ij (t))

[0098] Among them, ω is the inertia weight, c1 and c2 are learning factors, and r1 and r2 are random numbers between [0,1]. The position update formula is:

[0099] x ij (t+1)=x ij (t)+v ij (t+1)

[0100] In the system, the particle swarm optimization algorithm searches for the optimal combination in the parameter space such as sampling frequency and resolution according to the environmental complexity and target motion state evaluated by fuzzy logic to balance data quality and processing burden.

[0101] 2. Multi-source data fusion module

[0102] Tensor decomposition: Tensor decomposition is used to decompose multi-source data from high-dimensional space into low-dimensional subspaces to extract the core features of the data. For example, the commonly used CP decomposition form is:

[0103]

[0104] Among them, a r ∈R I ,b r ∈R J ,c r ∈R K is a factor vector, Represents the vector outer product, and R is the decomposition rank. Through tensor decomposition, high-dimensional multi-source data (such as tensors composed of infrared, visual, and lidar data) are decomposed into low-dimensional vector combinations, and the core features of each modality data are extracted to provide a more concise and effective data representation for subsequent fusion.

[0105] Deep learning attention mechanism: The deep learning attention mechanism dynamically adjusts the weight of each sensor data according to the dynamic changes of the environment. Taking the self-attention mechanism as an example, for the input feature sequence Q, K, V, the attention output is:

[0106]

[0107] Among them, d k is the dimension of the key vector K. In the system, the features of different modal data are input into the attention mechanism as Q, K, and V respectively, and the weight of each modal data is calculated to achieve dynamic fusion of multi-source data. For example, when the quality of visual data decreases due to changes in ambient lighting, the attention mechanism will reduce the weight of visual data and relatively increase the weight of infrared and lidar data.

[0108] Cross-modal similarity measure function:

[0109] Wasserstein distance: used to measure the difference between different modal data distributions to ensure the consistency of the data in the overall distribution. For two probability distributions P and Q, the Wasserstein distance (also called Earth-Mover's distance) in one dimension is defined as:

[0110]

[0111] in, is the set of all joint distributions of P and Q. In the system, by calculating the Wasserstein distance of the feature distributions of different modal data, we can determine their similarity in overall distribution and ensure that the fused data remains consistent in distribution.

[0112] Cosine similarity: measures the similarity between feature vectors of different modes from the perspective of vector angle, capturing the similarity of features. For two feature vectors a and b, cosine similarity is defined as:

[0113]

[0114] In the system, the feature vectors extracted from different modal data are input into the cosine similarity calculation to evaluate the similarity between them, which helps to retain similar features during the fusion process and enhance the stability of the fused data.

[0115] Mutual information: evaluates the degree of information sharing between different modal data and highlights the complementarity of data. For two random variables X and Y, mutual information is defined as:

[0116]

[0117] Among them, p(x,y) is the joint probability distribution of X and Y, and p(x) and p(y) are their marginal probability distributions. In the system, by calculating the mutual information of different modal data, the complementary information between them is discovered, thus improving the quality of the fused data.

[0118] 3. Feature extraction enhancement module

[0119] Quantum enhancement: Use the superposition and entanglement characteristics of quantum states to accelerate the feature extraction process. In quantum computing, quantum bits can be in a superposition state of 0 and 1, such as |ψ>=α|0>+β|1>, where α and β are complex numbers, and |α| 2 +|β| 2 = 1. In the process of updating the convolution kernel parameters, the parallel computing capability of quantum bits allows exploring multiple parameter update directions at the same time, and the rapid optimization of parameters is achieved through quantum gate operations, thereby accelerating the feature extraction process. For example, through quantum gate operations such as Hadamard gates, quantum bits are transformed to achieve parallel updates of convolution kernel parameters, which can find the optimal parameters faster than traditional methods and improve feature extraction efficiency.

[0120] Generative adversarial network: The generator in the generative adversarial network generates simulated feature data, and the discriminator distinguishes between real and simulated features, and mines more representative features through adversarial training. The objective functions of the generator G and the discriminator D are:

[0121]

[0122] Among them, p data (x) is the real data distribution. The generator tries to generate simulated data similar to the real data to make the discriminator misjudge. The discriminator tries to correctly distinguish between real data and simulated data. In the system, the generative adversarial network continuously conducts adversarial training to prompt the generator to generate data closer to the real target features. In this process, the discriminator learns to better identify the real features, thereby mining more representative target features.

[0123] Residual network: The residual network solves the gradient vanishing problem in deep networks through skip connections and enhances the network's ability to extract complex features. In the residual network, assuming the input is x, and the output obtained after a series of convolutional layers is F(x), the output of the residual block is:

[0124] y=F(x)+x

[0125] Through this jump connection method, even if the number of network layers increases, the gradient can be back-propagated more smoothly, avoiding the gradient vanishing problem. In the system, the residual network is used to perform deep feature extraction on the fused data, which can effectively extract the key features of the target in complex scenes, such as accurately identifying the subtle texture and structural features of the target in a complex background.

[0126] 4. Target state estimation module

[0127] Adaptive Kalman filter: Adaptive Kalman filter dynamically adjusts the filter parameters according to the rapid changes in target motion and complex environmental noise interference. The prediction equation of the standard Kalman filter is:

[0128]

[0129] P k|k-1 =AP k-1|k-1 A T +Q

[0130] The update equation is:

[0131] K k =P k|k-1 H T (HP k|k-1 H T +R) -1

[0132]

[0133] P k|k =(IK k H)P k|k-1

[0134] in, is the predicted state at time k, is the updated state at time k, A is the state transfer matrix, B is the control input matrix, u k is the control input, P k|k-1 is the prediction co-error, P k |k is the update co-error, Q is the process noise covariance, K k is the Kalman gain, H is the observation matrix, z k is the observed value, and R is the observed noise covariance. In the adaptive Kalman filter, the values ​​of Q and R are dynamically adjusted according to the uncertainty of the target motion and the change of environmental noise. For example, when the target motion suddenly accelerates or is subject to greater interference, the value of Q is increased to better track the target state change; when the sensor measurement noise increases, the value of R is increased to reduce the weight of the observed value.

[0135] Particle filter: Particle filter estimates the target state by sampling and updating the weights of a large number of particles. Assume that the target state is x k , the observed value is z k First, according to the prior distribution p(x0), the particles Initialize, i = 1, ..., N, N is the number of particles. At time k, according to the state transition probability p(xk |x k-1 ) samples the particles and obtains Then, according to the observed value z k Calculate the weight of the particle:

[0136]

[0137] Finally, the particles are resampled, particles with larger weights are retained, particles with smaller weights are discarded, and a new set of particles is obtained. In the system, the particle filter can handle non-Gaussian and nonlinear target state estimation problems by sampling and updating the weights of a large number of particles, and accurately estimate the target state in complex environments. For example, when the target is temporarily blocked or the motion trajectory is complex, the distribution and weight changes of the particles can still be discussed to reasonably infer the position and state of the target.

[0138] 5. Positioning solution module

[0139] Dynamic coordinate transformation: By real-time monitoring of the UAV's flight attitude and environmental parameter changes, the coordinate system is dynamically adjusted during the positioning process. Assume that the attitude of the UAV is represented by the rotation matrix R, and the position vector is t. Under different flight attitudes, the coordinate x of the target in the UAV's local coordinate system is local Convert the coordinate x to the global coordinate system global , the formula is:

[0140] x global =Rx local +t

[0141] Among them, the rotation matrix R is based on the pitch angle θ and yaw angle of the drone The roll angle ω is calculated. In the system, dynamic coordinate transformation ensures that the target's position information can be accurately converted into a unified global coordinate system when the UAV's flight posture is constantly changing, providing an accurate data basis for subsequent positioning calculations.

[0142] Error compensation: Online compensation of measurement error. Assuming the measurement error is ∈, the measured value is z, and the true value is x, then the estimated value after error compensation is for:

[0143]

[0144] The error ∈ can be determined by the sensor's calibration data, statistical analysis of historical measurement data, or model-based error prediction. In the system, the error compensation mechanism can improve the accuracy of the measurement data and reduce the impact of sensor errors on target positioning.

[0145] Least squares fitting: used to find the optimal positioning solution in multiple sets of measurement data. Suppose we have n sets of measurement data (xi ,y i ,z i ), and a model function to be fitted f(x, y, z; θ), where θ is the parameter vector of the model. The goal of the least squares method is to find a set of parameters Minimize the sum of squared errors between the measured values ​​and the model predicted values, that is:

[0146]

[0147] where d i It is the actual distance or position information corresponding to the measured data. In the positioning based on the triangulation principle, f(x,y,z;θ) may be a distance function related to the target position (x,y,z) and the position and attitude of the drone. By minimizing the above error sum of squares, the optimal position estimate of the target is determined. In the system, the least squares fitting method comprehensively considers the measurement information after multi-view data fusion, combined with the real-time attitude of the drone, and adapts to the flight attitude changes of the drone and the dynamic interference of the environment in real time to achieve high-precision three-dimensional positioning of the target.

[0148] 6. Flight decision planning module

[0149] Deep reinforcement learning: In the dual-depth Q network architecture, the agent learns the optimal flight strategy by interacting with the environment. The core of Q learning is to learn a Q function Q(s,a), which represents the long-term cumulative reward expectation of taking action a in state s. The update formula of the Q function is:

[0150]

[0151] Where α is the learning rate, γ is the discount factor, and r t is in state s t Take action a t The reward obtained after t +1 is the next state. In the dual deep Q network, two neural networks are introduced, one for estimating the current Q value (online network) and the other for estimating the target Q value (target network) to reduce the overfitting problem of Q value estimation. In the system, deep reinforcement learning learns to select the optimal flight action based on the current position, target position, energy state, environmental information and other states of the drone, such as changing the flight speed, direction, altitude, etc., to efficiently track the target and meet energy and safety constraints.

[0152] Ant Colony Algorithm: The Ant Colony Algorithm simulates the pheromone transmission during the foraging process of ants to find the potential optimal path in the path search space. Ants release pheromones on the path, and the concentration of pheromones will affect the path selection of subsequent ants. Let the city (or location node) set be C, and the probability p of ant k choosing to move to location j when it is at location i isi j k for:

[0153]

[0154] where τ ij is the pheromone concentration on the path from position i to position j, η ij =1 / d ij is the heuristic information (d ij is the distance from position i to position j), α and β represent the relative importance of pheromone and heuristic information respectively, J k is the set of positions that ant k can choose next. As the ant moves, the pheromone is updated according to the following formula:

[0155] τ ij =(1-ρ)τ ij +Δτ ij

[0156]

[0157] where ρ is the pheromone volatilization rate, is the amount of pheromone left by ant k on the path (i, j). In the system, the ant colony algorithm gradually explores the potential optimal flight path in the path search space of the drone through the accumulation and volatilization of pheromones, and plans a reasonable flight trajectory for the drone considering the length and safety of the path and the collaboration with other algorithms.

[0158] A algorithm: A algorithm uses heuristic functions to quickly search for the optimal path from the current location to the target location.

[0159] The search process of algorithm A is based on an evaluation function f(n)=g(n)+h(n), where g(n) is the actual cost from the starting node to node n, and h(n) is the estimated cost from node n to the target node (heuristic function).

[0160] Algorithm A selects the node with the smallest f(n) value for expansion. For example, in a two-dimensional plane, if the Euclidean distance is used as the heuristic function, Where (x n ,y n ) is the coordinate of node n, (x goal ,y goal) are the coordinates of the target node. In the system, the A algorithm combines the current position of the drone and the target position to quickly search for a theoretically optimal path, while taking into account flight safety constraints (such as avoiding obstacles), providing an important reference for the drone's flight decision-making, and coordinating with deep reinforcement learning and ant colony algorithms to plan the optimal flight trajectory that meets multiple conditions in real time, ensuring that the drone can achieve long endurance and safe flight while efficiently tracking the target.

[0161] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprising a reference structure" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0162] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An infrared target positioning system for unmanned aerial vehicles integrating multi-model KF algorithm and deep learning, including a main system, characterized in that: The main system includes a data acquisition module, a multi-source data fusion module, a feature extraction and enhancement module, a dynamic model optimization module, a target state estimation module, a positioning solution module and a flight decision planning module. The data acquisition module deploys a high-resolution infrared sensor to accurately capture the target infrared radiation information, and integrates a high-definition visual camera and a high-precision lidar to collect environmental data in all directions. It adopts an adaptive sampling strategy based on fuzzy logic, particle swarm optimization and adaptive thresholds, and dynamically adjusts the sampling frequency and resolution of the sensor according to the complexity of the environment and the target motion state.

2. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 1 is characterized in that: The multi-source data fusion module uses an adaptive fusion algorithm that integrates biological neuron coordination mechanism, tensor decomposition and deep learning attention mechanism to simulate information interaction and integration between neurons and perform feature encoding on infrared, visual and lidar data. Tensor decomposition decomposes multi-source data from high-dimensional space into low-dimensional subspace to extract core features of the data. The deep learning attention mechanism dynamically adjusts the weight of each sensor data according to the dynamic changes of the environment through competitive learning and collaborative feedback to achieve deep fusion of multi-source data and provide a high-quality data foundation for subsequent processing. At the same time, cross-modal consistency constraints are introduced to ensure that the consistency and complementarity of information of various modal data are maintained during the fusion process by constructing a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information. The feature extraction enhancement module adopts a convolutional neural network architecture that integrates quantum enhancement, generative adversarial network and residual network, and uses the superposition and entanglement characteristics of quantum states to accelerate the feature extraction process. The generator in the generative adversarial network generates simulated feature data, and the discriminator distinguishes between real and simulated features. More representative features are mined through adversarial training. The residual network solves the gradient vanishing problem in the deep network through jump connections, enhances the network's ability to extract complex features, and combines the attention mechanism with the improved void convolution technology to focus on the key features of the target, expand the receptive field, and enhance the ability to extract target features in complex scenes. The dynamic model optimization module introduces a model optimization strategy based on generative adversarial networks (GANs), transfer learning, and genetic algorithms, wherein the generator generates simulated target and scenario data, the discriminator distinguishes between real and simulated data, and the generalization ability of the model is improved through adversarial training. The transfer learning transfers the model knowledge trained in similar scenarios to accelerate the convergence speed of the model in new scenarios. The genetic algorithm searches for the optimal model configuration in the model space by encoding, selecting, crossing, and mutating the structure and parameters of the model, thereby realizing rapid optimization and adaptive adjustment of the model. The target state estimation module uses an algorithm that integrates reinforcement learning, adaptive Kalman filtering and particle filtering to model the target state estimation problem as a Markov decision process. Through the reward function for the drone target positioning scenario, the intelligent agent is guided to learn the optimal state estimation strategy. The adaptive Kalman filter dynamically adjusts the filter parameters according to the rapid changes in the target motion and the interference of complex environmental noise. The particle filter estimates the target state by sampling and updating the weights of a large number of particles. The positioning solution module is based on multi-view data fusion and improved triangulation principle, combined with the real-time attitude information of the UAV, and integrates the mechanisms of dynamic coordinate transformation, error compensation and least squares fitting. By real-time monitoring of the flight attitude of the UAV and changes in environmental parameters, the coordinate system in the positioning process is dynamically adjusted, and the measurement error is compensated online. The least squares fitting is used to find the optimal positioning solution in multiple sets of measurement data, adapt to the flight attitude changes of the UAV and the dynamic interference of the environment in real time, and achieve high-precision three-dimensional positioning of the target; The flight decision planning module adopts a dual-depth Q network architecture based on deep reinforcement learning, ant colony algorithm and A algorithm, combined with experience replay and priority experience replay mechanisms, to effectively reduce the overfitting problem of Q value estimation and accelerate the learning process. The deep reinforcement learning will learn the optimal flight strategy, among which the ant colony algorithm simulates the pheromone transmission during the foraging process of ants to find the potential optimal path in the path search space, and the A algorithm uses heuristic functions to quickly search for the optimal path from the current position to the target position, while considering the energy consumption of the drone, flight safety constraints and environmental dynamic factors, and planning the optimal flight trajectory in real time.

3. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized in that: In the multi-source data fusion module, an adaptive fusion algorithm based on biological neuron synergy mechanism, tensor decomposition and deep learning attention mechanism constructs a neuron connection weight matrix to simulate the excitation and inhibition relationship between neurons, and combines the low-dimensional feature representation after tensor decomposition and the weight distribution of the attention mechanism to achieve efficient fusion of sensor data, thereby improving the system's adaptability to complex and changing environments and the accuracy of target perception.

4. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized in that: The parallel computing capability of quantum bits in the feature extraction enhancement module simultaneously explores the optimal solution direction during the parameter update process of the convolution kernel, and the adversarial training mechanism of the generative adversarial network prompts the network to mine representative features. The jump connection of the residual network ensures the effective extraction of deep-level features by the network, thereby accelerating the convergence of the model.

5. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized in that: In the dynamic model optimization module, the generative adversarial network uses adversarial training to gradually make the data distribution generated by the generator approach the real data distribution. The transfer learning reduces the training time and sample requirements of the model in new scenarios through feature mapping and knowledge transfer between the source domain and the target domain. The genetic algorithm further improves the performance and adaptability of the model by optimizing the model structure and parameters.

6. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized in that: The target state estimation module uses reinforcement learning to guide the intelligent agent to select the optimal estimation strategy under the conditions of target motion pattern and environmental noise, and the adaptive Kalman filter quickly adjusts the filtering parameters. The particle filter improves the estimation accuracy through particle sampling and weight updating, so that the estimation error is kept in a small range.

7. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized by: When considering the energy consumption of the UAV, the flight decision planning module establishes an energy model to associate factors such as flight speed, altitude, and attitude with energy consumption. When planning the flight trajectory, deep reinforcement learning, ant colony algorithm, and A algorithm are used to give priority to paths with low energy consumption. When considering flight safety constraints, the environmental perception data is combined to avoid obstacles and dangerous areas to ensure the flight safety of the UAV.

8. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized in that: When collecting data, the data acquisition module dynamically and intelligently adjusts the sampling frequency and resolution of the sensor according to the real-time changes in the complexity of the environment and the target's motion state, thereby minimizing data redundancy and processing burden while ensuring data quality.

9. The UAV infrared target positioning system integrating multi-model KF algorithm and deep learning according to claim 2 is characterized in that: During the data fusion process, the multi-source data fusion module introduces a cross-modal similarity measurement function based on Wasserstein distance, cosine similarity and mutual information to measure the similarity and complementarity between different modal data from various angles, ensuring that different modal data maintain a high degree of consistency and complementarity of information during the fusion process, further improving the quality and reliability of the fused data.

Images