Target detection locking method and system for robot dog and electromagnetic gun

Abstract

The invention discloses a target detection locking method and system for a robot dog and an electromagnetic gun. The system comprises an image acquisition module, a target detection module, an evaluation module and a target locking module. The target detection locking method comprises the following steps: S1, acquiring environment information through an image acquisition module, and performing multi-modal data fusion on the acquired information; s2, evaluating the target priority, and designing a feedback control mechanism; the method comprises the following steps: S2.1, constructing a multi-dimensional state vector, S2.2, adjusting a shooting parameter through a continuous action vector, S2.3, designing a multi-target reward function, and S2.4, minimizing a time sequence difference error; and S3, locking the selected target, and recording and feeding back information such as the position and the speed of the target in real time. The robot dog electromagnetic gun system has the cooperative advantages of high precision, high adaptability, low energy consumption and high robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual detection and situational awareness, and in particular, relates to a target detection and locking method and system for a robot dog and an electromagnetic gun. Background Art

[0002] With the development of artificial intelligence and computer vision technologies, AI vision algorithms will gradually become an important component of modern automated weapon systems and unmanned systems. These systems acquire multimodal complementary information from multiple sensors and use AI algorithms to detect, identify, select, and lock on targets, thereby achieving efficient and accurate target detection and lock in changing environments.

[0003] Traditional visual inspection methods rely on large amounts of labeled data to train models. However, battlefield targets (such as new camouflage equipment and asymmetric threats) may not be covered by historical data, resulting in poor generalization of the models in real-world scenarios and insufficient accuracy to meet combat requirements. Offline training also makes it difficult to dynamically update the knowledge base. For example, if the enemy suddenly changes their camouflage paint or employs new electromagnetic interference methods, traditional algorithms may become completely ineffective.

[0004] Patent publication number CN118759517A discloses a method and device for cooperative detection of drones using multi-source heterogeneous sensors, involving target tracking and identification technology. The method includes: guiding radar to search for targets based on ESM detection information, setting the radar's key search area based on the target's orientation provided by the ESM, and using the radar to track the target; photographing the target using an infrared camera and a CCD camera to obtain infrared and visible light images of the target, respectively; using a CNN as a feature extractor to extract feature vectors from the infrared and visible light images; fusing the extracted feature vectors, and identifying the target using the YOLO target detection algorithm based on the fused feature image. This patent lacks dynamic decision-making capabilities and cannot autonomously adjust its strategy based on real-time environmental changes (such as wind speed and target maneuvers), resulting in performance degradation during sudden interference or rapid target maneuvers. Static models also lack generalization capabilities when the environment changes dramatically, making them susceptible to new interference methods. Summary of the Invention

[0005] This invention addresses the problem that traditional visual detection methods rely on extensively labeled data to train models. However, targets in battlefield environments (such as new camouflaged equipment and asymmetric threats) may not be covered by historical data, resulting in poor generalization and insufficient accuracy in real-world scenarios. Furthermore, offline training makes it difficult to dynamically update the knowledge base. For example, if the enemy suddenly changes their camouflage pattern or employs new electromagnetic interference techniques, traditional algorithms may become completely ineffective. Therefore, a target detection and locking method and system for a robot dog and an electromagnetic gun are proposed.

[0006] A target detection and locking method for a robot dog and an electromagnetic gun includes the following steps:

[0007] S1. Acquire environmental information through the image acquisition module and perform multimodal data fusion on the acquired information; the target detection module performs target detection and classification on the multimodal data based on a convolutional neural network;

[0008] S2. Use deep reinforcement learning algorithms to evaluate target priorities and design feedback control mechanisms to adjust locking accuracy and target selection strategies in real time.

[0009] S2.1. The robot dog obtains information about the environment and its own state through sensors and constructs a multi-dimensional state vector. The state vector expression is:

[0010] S t =[x target ,y target ,v x ,v y ,θ dog ,φ dog ,ω,E charge ,T cooldown ,ω wind ,ρ air ]

[0011] Where x target ,y target ,v x ,v y is the target dynamics, θ dog ,φ dog ,ω is its own posture, E charge ,T cooldown is the electromagnetic gun state, ω wind ,ρ air For environmental factors;

[0012] S2.2. Define a continuous action space and adjust the shooting parameters using a continuous action vector. The continuous action vector is expressed as follows:

[0013] a t =[Δθ aim ,ΔE fire ,Δt delay ]

[0014] Where Δθ aim is the aiming angle, ΔE fire is the emission energy, Δt delay is the launch delay time;

[0015] S2.3. A multi-objective reward function is constructed by integrating hit reward, energy penalty, time penalty, and collision penalty. Its expression is:

[0016] R(s t ,a t )=α*∏ hit -β*||E fire ||-γ*t delay -δ*∏ collision

[0017] Where, α*∏ hit represents the hit reward, β*||E fire || represents energy penalty, γ*t delay represents the time penalty, δ*∏ collision represents a collision penalty;

[0018] S2.4. Based on the principle of minimizing the time series difference error, optimize the value function parameters, the expression is:

[0019]

[0020] Where L(φ) is the loss function, E is the expected value, V φ (s t ) is the parameterized value function, r t is the instant reward at time t, γ is the discount factor, V φ (s t+1 ) is the next state s t+1 estimated value of

[0021] S3. The target locking module locks the selected target and records the target's position, speed and other information in real time and provides feedback.

[0022] Furthermore, in step S1, the image acquisition module includes GPS, an inertial navigation system, a laser radar, and an infrared thermal imaging sensor.

[0023] Furthermore, in step S1, the multimodal data fusion includes performing spatiotemporal alignment on GPS information, INS information, lidar points, and infrared thermal imaging cloud data, and dynamically allocating data weights of different sensors based on an attention mechanism to generate a fusion feature map.

[0024] Furthermore, in step S1, the target detection module uses a convolutional neural network, such as ResNet, YOLO, and Faster R-CNN, to perform real-time target detection and recognition.

[0025] Furthermore, the evaluation module is provided with a manual mode and a managed mode.

[0026] Furthermore, the manual mode provides candidate targets based on the target detection algorithm and the evaluation module, and the locked target is selected manually.

[0027] Furthermore, the hosting mode performs target selection and priority assessment based on target appearance information, target importance, threat level, and relative position.

[0028] Furthermore, the aiming angle Δθ aim The adjustment range is Δθ aim ∈[-θ max ,θ max ]; emission energy ΔE fire Adjustment range is ΔE fire ∈[0,E max ]; Transmitting delay time Δt delay The range is Δt delay ∈[0,t max ].

[0029] Furthermore, the target locking module integrates the DeepSort algorithm, specifically including: using the Kalman filter to predict the target motion trajectory, and completing cross-frame data association based on the Hungarian algorithm of Mahalanobis distance and appearance feature similarity.

[0030] A target detection and locking system adopts the above-mentioned target detection and locking method. The target detection and locking system includes an image acquisition module, a target detection module, an evaluation module and a target locking module.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The system described in the present invention includes an image acquisition module, a target detection module, an evaluation module, and a target locking module. The multimodal sensors (GPS, INS, infrared thermal imager, and lidar) carried by the robot dog collect environmental data in real time. Then, a deep learning algorithm is used to achieve autonomous target identification and priority assessment. The speed of target selection should be able to meet the needs of real-time applications, ensuring that priority assessment and selection are completed within 100 milliseconds after the target appears. This invention solves the lag problem of traditional weapons relying on manual aiming. Strengthening learning dynamically adjusts the strategy to adapt to rapid changes on the battlefield, significantly improving strike efficiency and accuracy, and has low energy and high robustness.

[0033] 2. This invention fuses information from multimodal sensors (GPS / INS / LiDAR / infrared), eliminating the blind spots of single sensors in complex terrain (such as smoke and at night), improving environmental modeling accuracy, and providing a reliable basis for correcting firing parameters for electromagnetic guns. By integrating sensors with navigation systems and enabling coordinated operations, the overall combat unit's situational awareness and decision-making efficiency are improved, further enhancing combat effectiveness.

[0034] 3. The DeepSort algorithm is used, the Kalman filter is used for dynamic modeling of the target, and the Hungarian algorithm is used for data association, which can effectively handle the tracking problem of multiple targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of sensor multi-module data fusion of the present invention;

[0036] Figure 2 A diagram of the neural network structure trained by the deep reinforcement learning algorithm of the present invention;

[0037] Figure 3 This is a schematic diagram of the multi-sensor data fusion principle of the present invention. DETAILED DESCRIPTION

[0038] In order to clearly illustrate the technical features of the application scheme of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0040] Example 1

[0041] A target detection and locking method for a robot dog and an electromagnetic gun includes the following steps:

[0042] S1. Acquire environmental information through the image acquisition module and perform multimodal data fusion on the acquired information; the target detection module performs target detection and classification on the multimodal data based on a convolutional neural network;

[0043] S2. Use deep reinforcement learning algorithms to evaluate target priorities and design feedback control mechanisms to adjust locking accuracy and target selection strategies in real time.

[0044] S2.1. The robot dog acquires environmental and self-status information through sensors, constructing a multi-dimensional state vector that includes target dynamics (position, speed), robot dog posture (angle, angular velocity), electromagnetic gun status (battery level, cooling time), and environmental factors (wind speed, air density). This ensures that the system can dynamically perceive and quantify key variables in the current scene. The state vector expression is:

[0045] S t =[x target ,y target ,v x ,v y ,θdog ,φ dog ,ω,E charge ,T cooldown ,ω wind ,ρ air ]

[0046] Where x target ,y target ,v x ,v y is the target dynamics, θ dog ,φ dog ,ω is its own posture, E charge ,T cooldown is the electromagnetic gun state, ω wind ,ρ air For environmental factors;

[0047] S2.2. Define a continuous action space and adjust the shooting parameters using a continuous action vector. The continuous action vector is expressed as follows:

[0048] a t =[Δθ aim ,ΔE fire ,Δt delay ]

[0049] Where Δθ aim is the aiming angle, ΔE fire is the emission energy, Δt delay is the launch delay time;

[0050] S2.3. A multi-objective reward function is constructed by integrating hit reward, energy penalty, time penalty, and collision penalty. Its expression is:

[0051] R(s t ,a t )=α*∏ hit -β*||E fire ||-γ*t delay -δ*∏ collision

[0052] Where, α*∏ hit represents the hit reward, β*||E fire || represents energy penalty, γ*t delay represents the time penalty, δ*∏ collision represents a collision penalty;

[0053] S2.4. Based on the principle of minimizing the time series difference error, optimize the value function parameters, the expression is:

[0054]

[0055] Where L(φ) is the loss function, E is the expected value, V φ (s t ) is the parameterized value function, r t is the instant reward at time t, γ is the discount factor, V φ (s t+1 ) is the next state s t+1 estimated value of

[0056] S3. The target locking module locks the selected target and records the target's position, speed and other information in real time and provides feedback.

[0057] like Figure 1 and Figure 3 As shown, in this embodiment, the image acquisition module is equipped with GPS, lidar, infrared thermal imaging sensor and inertial navigation system (INS), and multi-sensor data and visual information are integrated through spatiotemporal alignment technology to improve the accuracy and robustness of target recognition. Specifically, the attention mechanism is used to dynamically allocate the weight of each sensor: in low-light environments, the weight of infrared thermal imaging data is increased to 70%, the weight of lidar data is 20%, and the weight of GPS / INS data is 10%; in complex obstacle environments, the weight of lidar data is increased to 50%. The fused feature map is input into the improved YOLOv5 network for real-time target recognition and detection, with a detection accuracy of 98.5%. The target detection module performs target detection and classification through a deep learning model. Target recognition distinguishes different types of targets, such as enemy personnel, drones, vehicles, etc. Specifically, the target detection module uses convolutional neural networks such as ResNet, YOLO, and Faster R-CNN for real-time target detection and recognition. The target locking module integrates the DeepSort algorithm, uses the Kalman filter to predict the target trajectory, and realizes cross-frame data association through Mahalanobis distance and appearance feature similarity, with a response time of less than 200ms.

[0058] Example 2

[0059] like Figure 2 As shown, this embodiment involves an autonomous shooting decision-making system based on deep reinforcement learning, integrating multi-source sensor data to achieve dynamic target strikes. The system uses AI algorithms for autonomous navigation and path planning, ensuring that the robot dog can efficiently and safely reach the combat area and achieve precise shooting operations. It also integrates an AI-assisted shooting system that dynamically adjusts shooting strategies based on deep reinforcement learning algorithms, including the following steps:

[0060] A deep reinforcement learning algorithm is used to evaluate target priority, and a feedback control mechanism is designed to adjust the locking accuracy and target selection strategy through real-time feedback.

[0061] (1) The robot dog obtains environmental and self-state information through sensors, and constructs a multi-dimensional state vector that includes target dynamics (position, speed), robot dog posture (angle, angular velocity), electromagnetic gun status (battery level, cooling time), and environmental factors (wind speed, air density). The system can dynamically compensate for environmental interference (such as wind force deviation) and self-motion disturbance (such as posture changes when the robot dog moves), significantly improving the hit rate in complex scenarios. The state vector expression is:

[0062] S t =[x target ,y target ,v x ,v y ,θ dog ,φ dog ,ω,E charge ,T cooldown ,ω wind ,ρ air ]

[0063] Where x target ,y target ,v x ,v y is the target dynamics, θ dog ,φ dog ,ω is its own posture, E charge ,T cooldown is the electromagnetic gun state, ω wind ,ρ air For environmental factors;

[0064] (2) Define the continuous action space. The flexible adjustment of the continuous action space (aiming angle, launch energy, delay time) enables the electromagnetic gun to optimize the ballistic trajectory according to real-time data. By dynamically correcting the aiming angle Δθ aim Offset the effect of lateral wind speed or adjust the emission energy ΔE fire To adapt to the change of target distance, the continuous action vector expression is as follows:

[0065] a t =[Δθ aim ,ΔE fire ,Δt delay ]

[0066] Where Δθ aim is the aiming angle, ΔE fire is the emission energy, Δt delay is the launch delay time;

[0067] (3) A multi-objective reward function is constructed by integrating hit reward, energy penalty, time penalty and collision penalty, and its expression is:

[0068] R(s t,a t )=α*∏ hit -β*||E fire ||-γ*t delay -δ*∏ collision

[0069] Where, α*∏ hit represents the hit reward, β*||E fire || represents energy penalty, γ*t delay represents the time penalty, δ*∏ collision represents the collision penalty; through the energy penalty β*||E fire || and time penalty γ*t delay , introducing collision penalties into the reward function, forcing the system to avoid potential risks (such as obstacles in the robot dog's path) when adjusting shooting parameters, ensuring that shooting efficiency and safety are met simultaneously during high-speed maneuvers. The system actively balances energy consumption and response speed while hitting the target. To avoid excessively increasing the launch energy and causing rapid depletion of the battery, a delay is added to wait for the cooling process to complete, ensuring the sustainable combat capability of the electromagnetic gun. Through the reward and punishment mechanism, the AI ​​is guided to strike a balance between energy efficiency, response speed, and safety while hitting the target, avoiding strategy imbalances caused by a single goal (such as excessive energy consumption or reckless collisions).

[0070] (4) Based on the principle of minimizing the time series difference error, optimize the value function parameters, and the expression is:

[0071]

[0072] Where L(φ) is the loss function, E is the expected value, V φ (s t ) is the parameterized value function, r t is the instant reward at time t, γ is the discount factor, V φ (s t+1 ) is the next state s t+1 estimated value.

[0073] In this embodiment, the reward function sets the hit reward α = 2.0, the energy penalty β = 0.5, the time penalty α = 0.3, and the collision penalty δ = 1.5, and optimizes the value function by minimizing the temporal difference error. In actual combat tests, the evaluation module can complete the target priority assessment within 1 second and automatically select the hosting mode according to the threat level: the aiming angle Δθ is preferentially assigned to high-speed moving targets (speed > 15m / s). aim Corrected the aiming angle to ∈[-30°, 30°] and shortened the launch delay time Δt delay At the same time, the electromagnetic gun supports energy graded emission, and uses E max=8kj high energy mode, using E for light targets fire =3kJ energy-saving mode, reducing energy consumption by 40%. A closed-loop reinforcement learning framework has been constructed: a closed-loop process from perception to action generation, effect evaluation, and parameter optimization enables the system to continuously evolve through a "trial-and-error-feedback" mechanism. For example, during multiple shooting missions, the system can autonomously learn how to adjust launch parameters under different wind speeds, reducing reliance on manual parameter adjustment and adapting to diverse mission requirements.

[0074] Multimodal data fusion is performed using GPS, an inertial navigation system, a lidar (lidar), and infrared thermal imaging sensors. Specifically, lidar and infrared thermal imaging (640×512 resolution) data are fused through a Kalman filter to construct a real-time 3D situational map. The system uses a neural network trained with a deep reinforcement learning algorithm. The input layer includes target range (0-800m), motion vector, environmental obstacle density, and remaining electromagnetic gun energy. The output layer is an attack priority score (0-1). The system also combines GPS / INS positioning data (positioning accuracy 0.3m) with the robot dog's attitude angle to correct the trajectory parameters in real time.

[0075] The specific implementation method is as follows: when 5 moving targets are detected, the system completes within 200ms: the lidar identifies the outline of the obstacle, and the infrared sensor marks the biological heat source; the threat assessment model calculates the weight coefficient of each target (such as +0.3 for carrying weapons, +0.2 for rapid approach); selects the optimal attack sequence based on the current battery power (energy-saving mode is triggered when it is below 30%), and simultaneously adjusts the direction of the robot dog to the optimal shooting angle.

[0076] Example 3

[0077] This embodiment designs a human-machine collaborative control solution suitable for rescue missions. The evaluation module supports seamless switching between manual mode and managed mode: in manual mode, the operator receives candidate targets (marked with threat level and position deviation) provided by the target detection module through the AR helmet, manually selects the target to lock on and adjusts Δθ aim and Δθ fire In managed mode, the system automatically makes decisions based on target appearance characteristics (such as heat signal strength) and relative position (priority +30% when the distance is <50m). In particular, when a friendly identifier (such as an RFID tag) is detected, a collision penalty mechanism is triggered, forcibly terminating the firing command. The electromagnetic gun is equipped with a dual cooling system. cooldomn The system's time is shortened to 3 seconds, supporting 10 consecutive shots. Experiments show that in mixed scenarios, the system's false-injury rate is less than 0.1%, and the mode switching delay is less than 100ms, significantly improving operational safety in complex environments.

[0078] The target detection and locking system includes an image acquisition module, a target detection module, an evaluation module and a target locking module.

[0079] Obviously, the above-described embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. A person skilled in the art would be able to make other variations or modifications based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A target detection and locking method for a robot dog and an electromagnetic gun, characterized in that: The following steps are involved: S1. Obtain environmental information through the image acquisition module and perform multimodal data fusion on the acquired information; The target detection module performs target detection and classification on multimodal data based on convolutional neural networks; S2. Use deep reinforcement learning algorithms to evaluate target priorities and design feedback control mechanisms to adjust locking accuracy and target selection strategies in real time. S2.

1. The robot dog obtains information about the environment and its own state through sensors and constructs a multi-dimensional state vector. The state vector expression is: S t =[x target ,y target ,v x ,v y ,i dog ,f dog Oh, E charge ,T cooldown ,oh wind ,r air ] Where x target ,y target ,v x ,v y is the target dynamics, θ dog ,φ dog ,ω is its own posture, E charge ,T cooldown is the electromagnetic gun state, ω wind ,ρ air Environmental factors; S2.

2. Define a continuous action space and adjust the shooting parameters using a continuous action vector. The continuous action vector is expressed as follows: a t =[Δθ aim ,D.E fire ,Δt delay ] Where Δθ aim is the aiming angle, ΔE fire is the emission energy, Δt delay is the launch delay time; S2.

3. A multi-objective reward function is constructed by integrating hit reward, energy penalty, time penalty, and collision penalty. Its expression is: R(s t ,a t )=α*∏ hit -b*||E fire ||-c*t delay -d*∏ collision Where, α*∏ hit represents the hit reward, β*||E fire || represents energy penalty, γ*t delay represents the time penalty, δ*∏ collision represents a collision penalty; S2.

4. Based on the principle of minimizing the time series difference error, optimize the value function parameters, the expression is: Where L(φ) is the loss function, E is the expected value, V φ (s t ) is the parameterized value function, r t is the instant reward at time t, γ is the discount factor, V φ (s t+1 ) is the next state s t+1 estimated value of S3. The target locking module locks the selected target and records the target's position, speed and other information in real time and provides feedback.

2. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 1, characterized in that: In step S1, the image acquisition module includes GPS, an inertial navigation system, a laser radar, and an infrared thermal imaging sensor.

3. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 1, characterized in that: In step S1, the multimodal data fusion includes temporal and spatial alignment of GPS information, INS information, lidar points and infrared thermal imaging cloud data, and dynamically allocating data weights of different sensors based on the attention mechanism to generate a fusion feature map.

4. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 1, characterized in that: In step S1, the target detection module uses a convolutional neural network, such as ResNet, YOLO, and Faster R-CNN, to perform real-time target detection and recognition.

5. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 1, characterized in that: The evaluation module is provided with a manual mode and a managed mode.

6. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 5, characterized in that: The manual mode provides candidate targets based on the target detection algorithm and the evaluation module, and the target is manually selected and locked.

7. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 5, characterized in that: The hosting mode performs target selection and priority assessment based on target appearance information, target importance, threat level, and relative position.

8. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 1, characterized in that: The aiming angle Δθ aim The adjustment range is Δθ aim ∈[-θ max ,θ max ]; emission energy ΔE fire Adjustment range is ΔE fire ∈[0,E max ]; Transmission delay time Δt delay The range is Δt delay ∈[0,t max ].

9. The target detection and locking method for a robot dog and an electromagnetic gun according to claim 1, characterized in that: The target locking module integrates the DeepSort algorithm, which specifically includes: using the Kalman filter to predict the target motion trajectory, and completing cross-frame data association based on the Hungarian algorithm based on the Mahalanobis distance and appearance feature similarity.

10. A target detection and locking system, characterized in that: The target detection and locking method according to any one of claims 1 to 9 is adopted, wherein the target detection and locking system includes an image acquisition module, a target detection module, an evaluation module and a target locking module.

Images