Target Detection Method for Tactical AR Helmets

Abstract

The present invention relates to a target detection method for a tactical AR helmet. The method comprises the following steps: head posture perception and spatial positioning module: after the helmet is started, the inertial measurement unit IMU sensor collects acceleration and angular velocity data of the head through a gyroscope, an accelerometer and a magnetometer; a sensor fusion algorithm including a Kalman filter is used to integrate the data of multiple sensors; a spatial mapping of a virtual scene and a real scene is created; a target detection module is triggered: the real-time video frame captured by the camera is input into a pre-trained convolutional neural network CNN model; the CNN model performs feature extraction according to different detection tasks including target classification and object boundary detection; multi-scale detection of the target is performed through multi-layer convolution and pooling operations; a plurality of candidate frames are generated using a regional proposal network RPN; real-time target tracking and calibration: the detected target position is passed to the target tracking algorithm to initialize the tracking; the size and position of the tracking frame are adjusted according to the change in the viewing angle of the helmet wearer.

Description

Technical Field

[0001] The present invention relates to a target detection method for a helmet, in particular to a target detection method for a tactical AR helmet. Background Art

[0002] At present, the target detection method of tactical AR helmets, such as the intelligent safety helmet based on target detection and depth estimation in China's invention patent publication number CN117617620A, has certain intelligent detection and safety protection functions, but compared with more complex tactical scenarios and needs, there are still many deficiencies and disadvantages. These deficiencies are particularly evident in tactical combat environments, especially in high-intensity, complex and changeable environments. This article will compare the technology in CN117617620A and analyze in detail the major limitations of existing target detection methods for tactical AR helmets.

[0003] First of all, the helmet in this patent uses a series of sensor integration modules, main control modules and alarm modules. It collects environmental data through sensors, and then combines target detection and depth estimation algorithms based on deep learning technology to accurately determine the target category and location information. Although this design can achieve a certain degree of target recognition and positioning, its function is too limited in tactical scenarios. In tactical scenarios, in addition to basic target detection, combatants also need to perform more complex classification and analysis of targets, such as making judgments based on multi-dimensional information such as target behavior patterns, weapons, threat levels, etc., while existing smart safety helmets are often limited to the detection of target categories and locations. Especially on the battlefield, the target's behavior pattern often determines its potential threat level. Abnormal behaviors such as sudden acceleration, irregular movement or frequent turning may indicate that the enemy is about to take action. However, the smart helmet described in CN117617620A does not fully take into account this dynamic behavior evaluation, and only stays on static target detection, which makes it difficult to provide comprehensive intelligence support in complex tactical scenarios.

[0004] Secondly, the target detection system in CN117617620A relies on a depth estimation algorithm, which may play a role in specific scenarios, especially when judging the distance and orientation of the target. However, the performance of the depth estimation algorithm in multi-target dense or fast-moving scenes is often unsatisfactory. In particular, when multiple targets appear in the wearer's field of view at the same time, it is difficult for the existing system to quickly distinguish the depth levels of different targets, which may cause overlap and confusion between multiple targets, thereby affecting the accuracy of the system. Enemy targets in tactical environments often move quickly and are not fixed in position, and relying solely on depth estimation algorithms cannot cope with such complex situations. Although the depth estimation of the prior art can provide the distance information of the target, its calculation process may be too dependent on static scenes or depth changes at specific angles. Once the scene changes rapidly, the response speed and accuracy of the system will be limited. In actual tactical scenarios, the system must be able to make judgments within milliseconds, which is not fully reflected in the solution of CN117617620A.

[0005] Another significant shortcoming is the alarm module mentioned in CN117617620A. Although it can broadcast the type and location information of the dangerous source to the wearer, this is far from enough. The alarm system can only provide limited warning functions and cannot dynamically adjust the threat assessment of the target. In tactical scenarios, threat level assessment is a crucial link. Combatants not only need to know the location and category of the target, but also need to conduct a comprehensive threat assessment based on the target's speed, behavior pattern, equipment, etc. For example, although the enemy vehicle moves slowly, if it is equipped with heavy weapons, the threat level must be increased. However, the existing smart helmets are seriously insufficient in this regard. They cannot dynamically adjust the threat level of the target and can only make simple broadcasts based on static data, failing to provide in-depth tactical assistance. This fixed-mode alarm mechanism cannot adapt to changing tactical needs, especially in complex multi-target environments, where combatants need to prioritize multiple targets in real time instead of just receiving simple alarm information.

[0006] In addition, although the intelligent safety helmet design in CN117617620A has certain display and communication functions, the functions of its display module and communication module are relatively basic. The display module is only used to display the captured danger source and its position information, and lacks further data visualization capabilities. In tactical scenarios, combatants not only need to see the location of the target, but also need to present multi-level information in their field of vision through augmented reality (AR) technology, such as the target's movement trajectory, predicted threat level, and tactical information of the surrounding environment. However, the current display module fails to effectively integrate these complex tactical information, resulting in the wearer being unable to obtain sufficient intelligence support from the helmet. The function of the communication module is also relatively limited. It is only used for simple communication between wearers and fails to fully support complex teamwork. In actual combat, information sharing and transmission are particularly important. The helmet system should be able to share tactical intelligence with the command center or other combatants in real time, such as the real-time location of the target, threat level, and friendly position. The communication module of the existing system lacks this efficient information sharing capability, which makes the coordination of tactical command low. Summary of the invention

[0007] The purpose of the present invention is to provide a target detection method for a tactical AR helmet, thereby solving some of the drawbacks and shortcomings pointed out in the background technology.

[0008] The present invention solves the above-mentioned technical problems by adopting the following technical solution, which includes the following steps:

[0009] S1, head posture perception and spatial positioning module:

[0010] S1.1. After the helmet is started, the inertial measurement unit (IMU) sensor collects the acceleration and angular velocity data of the head through the gyroscope, accelerometer and magnetometer;

[0011] S1.2, integrating data from multiple sensors using a sensor fusion algorithm including a Kalman filter;

[0012] S1.3, create spatial mapping between virtual scene and real scene;

[0013] S2. Triggering of target detection module:

[0014] S2.1, input the real-time video frames captured by the camera into the pre-trained convolutional neural network (CNN) model;

[0015] S2.2, CNN model performs feature extraction based on different detection tasks including target classification and object boundary detection;

[0016] S2.3, multi-scale detection of targets through multi-layer convolution and pooling operations;

[0017] S2.4, use the region proposal network RPN to generate multiple candidate boxes;

[0018] S3, Real-time target tracking and calibration:

[0019] S3.1, the detected target position is passed to the target tracking algorithm to initialize tracking;

[0020] S3.2. Adjust the size and position of the tracking frame according to the change of the viewing angle of the helmet wearer;

[0021] S4. Augmented reality assisted target information feedback:

[0022] S4.1. Generate target attribute data based on target detection and tracking results, and display the information in the form of AR tags in the user's field of view;

[0023] S4.2. Use laser ranging to calculate the distance between the target and the user in real time and display it on the AR interface;

[0024] S4.3. Dynamically assess the threat level based on the target's movement speed and behavior pattern information, and distinguish them with different colors or graphics;

[0025] S5. Target classification and prioritization:

[0026] S5.1, the detection module passes the identified target information to the classification model, and the classification model classifies the target according to the predefined categories;

[0027] S5.2. Display the targets in different levels according to preset priority rules, including the size, speed and distance of the targets.

[0028] Furthermore, the method for extracting features for different detection tasks includes:

[0029] S1. First, the primary convolutional network is responsible for extracting common low-level image features, including edges and textures. The primary convolutional network will be used for multiple tasks. Through customized convolutional layers, the classification task and the boundary detection task are processed differently. The sharing and separation of feature extraction, the nonlinear mapping function is defined as:

[0030]

[0031] Among them, x represents the dimension of the input image features, and t is the integral variable, which represents the feature map of each layer in the convolution process; It is an exponential function, which indicates the growth rate of high-order features. Sin(πt) represents the periodic convolution operation, which captures the edge information in the image. Represents the smoothing of features during convolution, cos(x 3) represents the nonlinear changes of the global features of the image through cubic terms;

[0032] S2, introduce the convolutional feature pyramid network FPN; through the top-down feature map fusion, process the global features of large targets and the local boundary details of small targets; detect targets of different sizes, and the multi-scale feature fusion process is through the integral formula:

[0033]

[0034] Among them, x represents the feature scale in the image, t is the integral variable used to represent the feature maps of different levels of the image; sin(t 3 ) represents the fluctuation of semantic information in high-level features; e -t It represents the process of decreasing features layer by layer, which is used to extract edge information from low-level features; 1+t in the denominator 6 For smoothing; log(x 4 +1) is used to enhance the detail features of small targets;

[0035] S3, combined with bounding box regression and non-maximum suppression NMS technology; through bounding box regression, the detected candidate box is further optimized to predict the boundary position of the object; while non-maximum suppression suppresses overlapping candidate boxes; the bounding box regression process uses the formula to optimize the candidate box:

[0036]

[0037] Among them, x b Represents the coordinate information of the candidate box, and t is the intermediate variable involved in the optimization process; Indicates the degree of enhancement of features between candidate boxes; sin(t) is the boundary fluctuation of the candidate box, which is used to fit the boundary changes of the target; Operation, calculate the gradient change of each layer of candidate boxes; The item is used to suppress overlapping candidate boxes.

[0038] Furthermore, the method used for dynamically assessing the threat level includes: integrating the speed of the target and analyzing the behavior pattern of the target; the behavior pattern is abnormal behavior, including sudden acceleration, frequent turning, and irregular movement; modeling the behavior sequence of the target through deep learning technology, analyzing and capturing abnormal behavior, and assessing the potential threat of the target; using nonlinear functions to model the behavior data:

[0039]

[0040] Among them, x t represents the time step, τ is the time variable in the behavior sequence; Represents the exponential growth of the target behavior change, used to describe the situation of sudden acceleration or sudden behavior; sin(τ 2 ) is used to describe periodic changes in behavior and to capture frequent turns and irregular movements; Represents changes in global behavior patterns and is used for modeling a wide range of behavior patterns.

[0041] Furthermore, the method for dynamically assessing the threat level includes: after obtaining the speed and behavior pattern information of the target, superimposing the target's threat level in real time through a weight algorithm; the threat assessment is adjusted according to the target's behavior changes, including that abnormal target behavior will increase the target's threat weight, even if its movement speed is low. When assessing the threat level, an integral formula is used to weight the sum of different factors:

[0042]

[0043] Among them, w v represents the weight variable in threat assessment, It is used to describe the impact of target speed on threat assessment, especially for high-speed targets; log(u+1) is used to describe the contribution of abnormal behavior to the threat level in combination with the complexity of the target's behavior pattern; 1+u in the denominator 4 To suppress noise and excessive effects, Then balance the speed and behavioral factors of the target.

[0044] Furthermore, the method for dynamically assessing the threat level includes: performing personalized regulation according to user needs or task scenarios; adjusting the user's preset behavior and speed type through personalized weights, and dynamically adjusting the weight parameters of the threat assessment using a function:

[0045]

[0046] Where p represents the parameter variable of personalized adjustment; sin(v 3 ) is used to regulate the emergent behavior in the behavior pattern, e -v It regulates the scope of influence of the behavioral pattern.

[0047] Furthermore, the method for the classification model to classify the target according to the predefined category includes: the detection module is responsible for capturing the shape, color, texture, and speed information of the target in real time, and extracting the candidate target area through the preliminary detection algorithm; through the nonlinear transfer function:

[0048]

[0049] The extracted candidate target region information is passed as input to the classification model, where d fRepresents the characteristic dimension of the target data, z is an intermediate variable, representing the characteristics of the candidate target area at different scales; represents the nonlinear amplification of features in the candidate region; sin(z 2 ) is used to capture the periodic characteristic changes of the local area; The function smoothes the data as it passes through.

[0050] Furthermore, the classification model adopts a convolutional neural network (CNN) and enhances local information in the input features, including edges, corners and texture patterns, through a feature enhancement mechanism; the feature enhancement process is modeled by the function:

[0051]

[0052] Among them, l represents the local attribute dimension of the target feature, Indicates the exponential growth of local features, amplifying complex local features, cos(s 3 ) captures the rapid changes of local edge features; the denominator 1+s 4 Used to smooth characteristic responses; It is used to control the influence of local features.

[0053] Furthermore, the classification model adopts a multi-task learning-based architecture to classify target types based on multiple target attributes; the classification includes distinguishing between personnel, vehicles, and drone target types, and performing detailed classification according to the target's weapon equipment and behavior patterns; the feature extraction process is described by the formula:

[0054]

[0055] Among them, x represents the characteristic dimension of the target, t is the intermediate variable involved in the extraction process; sin(t 4 ) represents the periodic feature changes in multi-task classification, which is used for feature sharing of multiple tasks, and e -t It is used to control the attenuation of features in time series; the denominator t 2 +1 balances the feature strength of each task; log(x 3 +1) is used to regulate the global feature distribution in multi-task learning.

[0056] The tactical AR helmet target detection method of the present invention has the following beneficial effects:

[0057] By combining advanced convolutional neural networks (CNNs) with a multi-task learning architecture, the helmet is able to quickly and accurately detect targets in complex tactical environments. Whether it is a dynamic target (such as a drone) or a static target (such as a vehicle), the system can capture the key features of the target in real time and quickly classify it, allowing combatants to make decisions quickly in a rapidly changing battlefield. The use of a multi-task learning model can not only simultaneously identify multiple targets (such as personnel, vehicles, and drones), but also classify them in detail based on the different attributes of the target (such as weapons equipment and behavior patterns), helping combatants to accurately identify and prioritize the most threatening targets. This feature is crucial in a combat environment, ensuring that troops can flexibly respond to different types of threats.

[0058] By combining speed, behavior patterns and multi-dimensional feature evaluation, the present invention can perform dynamic threat assessment based on the target's behavior changes (such as sudden acceleration, frequent turns, etc.). Even if the target's movement speed is low, its abnormal behavior pattern can increase the threat level, ensuring that combatants can detect potential dangers in a timely manner and respond.

[0059] The feature enhancement mechanism used greatly improves the system's ability to capture local information, especially when dealing with complex details such as edges, corners, textures, etc. of the target. Through nonlinear amplification and local feature smoothing, the system can accurately identify and enhance small and important features, thereby greatly improving classification accuracy, especially in noisy backgrounds or occlusions, and still maintain efficient target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of the target detection method of the tactical AR helmet of the present invention.

[0061] Figure 2 Flowchart of the method for feature extraction for different detection tasks of the present invention.

[0062] Figure 3 This is a flow chart of the method used by the present invention to dynamically assess threat levels. DETAILED DESCRIPTION

[0063] The specific implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0064] Combined with Figure 1The S1 step in the target detection method of the tactical AR helmet is the head posture perception and spatial positioning module, and its specific implementation steps are as follows: In step S1.1, when the helmet is started, the inertial measurement unit (IMU) sensor starts working, and the gyroscope, accelerometer and magnetometer in the IMU are used together to collect the acceleration and angular velocity data of the wearer's head. The accelerometer is responsible for measuring the linear acceleration of the head, while the gyroscope captures the angular velocity, and the magnetometer determines the wearer's orientation through the geomagnetic field. The collection of these data can provide original information about the head posture and provide a basis for subsequent posture estimation.

[0065] In step S1.2, the Kalman filter algorithm is used for sensor fusion for the data obtained from multiple sensors. Kalman filtering is a recursive estimation algorithm that can optimize the noise data collected by different sensors. Through sensor fusion, a more accurate head position and posture estimation can be obtained. The data collected by the IMU sensor alone will be affected by noise or drift. Therefore, sensor fusion can improve the accuracy and integrate the data information of the gyroscope, accelerometer and magnetometer to make the head posture perception more accurate.

[0066] In step S1.3, a spatial mapping between the virtual scene and the real scene is created, that is, the spatial position in reality is associated with the space in the virtual scene through the perceived head posture information. This step uses the spatial coordinate system to match the head position in reality with the position of the virtual object, so that in the display of the tactical AR helmet, virtual information can be accurately superimposed in the wearer's field of view, achieving an augmented reality effect that combines virtual and real.

[0067] Combined with Figure 1 In the S3 part, step S2 is the trigger of the target detection module, and its specific implementation steps are as follows: In step S2.1, the camera on the tactical AR helmet captures video frames in the environment in real time and passes these video frames as input to the pre-trained convolutional neural network (CNN) model. The CNN model is the core of the target detection task and can perform deep feature extraction based on the input image. Before being passed to the CNN, the real-time video frames captured by the camera will undergo basic preprocessing, such as resizing and normalization, to ensure that the input meets the model requirements and can improve the detection efficiency and accuracy of the model.

[0068] In step S2.2, after receiving the video frame, the CNN model starts to extract features according to different detection tasks. Specifically, CNN extracts features in the image through multi-layer convolution operations, including target classification tasks and object boundary detection tasks. For target classification, the model focuses on global features and identifies the category of the target; while for boundary detection, the model pays more attention to local features, especially the edges and shapes of objects. Through this task distinction, the model can simultaneously perform classification and boundary location to ensure the accuracy of the detection results. In step S2.3, the CNN model performs multi-scale detection of targets through multi-layer convolution and pooling operations. This means that the model not only pays attention to the global information in the image, but also captures detailed features at different scales to ensure that both small and large targets can be effectively detected. Multi-layer convolution is responsible for extracting features at different levels, while pooling operations are used to reduce the size of feature maps and extract more important features, while reducing the amount of calculation and increasing detection speed.

[0069] In step S2.4, the region proposal network (RPN) is used to generate multiple candidate boxes. RPN is a lightweight network that can generate a series of candidate regions containing objects based on feature maps. Each candidate box represents the location of the object, and these candidate boxes are further processed to determine whether they contain the object of interest and its specific category and boundaries.

[0070] Combined with Figure 1 The S3 part of the system is real-time target tracking and calibration. The specific implementation steps are as follows: In step S3.1, when the system detects the position of the target, the position information will be immediately passed to the target tracking algorithm to start tracking the target. The target tracking algorithm is responsible for tracking the position of the target in real time in subsequent video frames captured by the camera to ensure that the system can continue to focus on the target when the target moves or the perspective changes. Initializing tracking means that the system will lock the target based on the detected initial position and features, and provide basic data for subsequent tracking. At this time, the appearance features of the target (such as color, shape, and edge information) will be extracted and stored for use in the tracking process.

[0071] In step S3.2, as the perspective of the helmet wearer moves, the system will dynamically adjust the size and position of the tracking frame according to the wearer's head movement. Sensors such as the inertial measurement unit (IMU) in the helmet can sense changes in the wearer's perspective in real time and match it with the target position in the video frame captured by the camera. By adjusting the tracking frame, the system can ensure that the target always remains in the correct position in the wearer's field of view. Even when the wearer's head turns or moves quickly, the tracking frame can still be adjusted in real time as the target moves. In addition, the size of the tracking frame can also be corrected according to the change in the distance between the target and the wearer, ensuring that when the target approaches or moves away from the wearer, the size of the tracking frame can change dynamically, thereby maintaining accurate tracking effects.

[0072] Combined with Figure 1 The S4 part of the system is augmented reality-assisted target information feedback, and its specific implementation steps are as follows: In step S4.1, the system generates target attribute data based on the results of target detection and tracking. These data include key information such as target type, size, moving speed, and direction. These target attribute data are superimposed on the user's field of view in real time through augmented reality (AR) technology and presented in the form of AR tags. For example, if an enemy target is detected, the system will display an identifier above or next to the target, indicating the type of target (such as a vehicle, personnel, or drone), along with other important tactical information. This information feedback helps users quickly understand the attributes of the target and make timely decisions in complex tactical environments.

[0073] In step S4.2, using the laser ranging sensor on the helmet, the system can calculate the distance between the target and the user in real time. The ranging results are accurately displayed in the AR interface, together with the attribute information of the target. This distance information is crucial for tactical decision-making. For example, at long distances, users need to prepare long-range strike weapons, and at close ranges, they need to adjust tactical behavior. By superimposing the distance information directly in the user's field of view, the system ensures that the user can obtain key spatial perception information at any time, thereby optimizing combat response.

[0074] In step S4.3, the system dynamically evaluates the threat level of the target based on the target's movement speed and behavior pattern information. The target's behavior patterns include sudden acceleration, frequent changes of direction, or movement on abnormal paths. The system determines the potential threat of the target by analyzing these behaviors. For example, the system can identify a target that approaches in an irregular manner and evaluate it as a high threat. Based on the evaluation results, the system will use different colors or graphic logos to distinguish the threat level of the target. Usually, color coding is used to intuitively display the threat, such as red for high threat, yellow for medium threat, and green for low threat.

[0075] Combined with Figure 1The S5 step is target classification and priority sorting, and its specific implementation steps are as follows: In step S5.1, the target detection module is responsible for detecting potential targets from the environment and extracting relevant feature information, such as the shape, color, texture, etc. of the target. The identified target information will be immediately passed to the classification model, which classifies the targets according to predefined categories through pre-trained deep learning algorithms or other pattern recognition methods. Predefined categories can be set according to combat requirements, including different categories of targets such as vehicles, personnel, and drones. The classification model can accurately classify each target through this feature information, thereby helping users quickly understand potential threats or resources in the environment. This process not only depends on the accuracy of the classification model, but also requires the classification model to have sufficient speed to ensure that the classification results can be updated in real time and fed back to the user in a timely manner.

[0076] In step S5.2, the system will display the classified targets in levels according to the pre-set priority rules. Priority rules usually include multiple dimensions such as the size, speed, and distance of the target. For example, larger and faster-moving targets are given higher priorities, while smaller and farther-away targets are considered less threatening and are given lower priorities. These priority rules can be customized according to specific tactical needs to ensure that the most threatening targets are displayed to the user first. The system will sort the targets according to these priorities and display the target information in the user's AR interface in layers according to importance, helping users focus on the most important threats or resources during combat.

[0077] Embodiment 1:

[0078] Combined with Figure 2 In a tactical scenario, a special operations team is performing an urban street fighting mission. Tactical AR helmets are used to identify potential threats in the environment, such as hidden enemy personnel, drones, and vehicles. In order to cope with complex combat environments, the images captured by the camera on the helmet need to be processed by a real-time target detection module, and the system must classify and detect boundaries of multiple targets in the shortest possible time. At this time, the convolutional neural network (CNN) of the AR helmet starts running, and through a hierarchical feature extraction method, it ensures that the classification and boundary detection tasks are completed efficiently and accurately.

[0079] The convolutional network at the primary layer is first responsible for extracting common low-level features from the image, such as the edges and textures of the target. These low-level features are the basis of the image and can usually capture the contour information of the target object. In order to use these low-level features for multiple tasks, the first few layers of the convolutional network extract conventional features such as edges and contours, and then use customized convolutional layers to process classification tasks and boundary detection tasks respectively. For example, the team detects that an enemy vehicle is moving slowly, but needs to further determine its specific category and boundary location. At this time, the system needs to process the vehicle's contour (boundary detection) and vehicle type (classification task) at the same time.

[0080] In actual calculation, the sharing and separation of feature extraction is achieved through nonlinear mapping functions; the function is defined as follows:

[0081]

[0082] Where x represents the dimension of the input image feature, and its value range is set to [0,10], representing features from simple to complex; and t is the feature mapping variable in the convolutional layer. Exponential function Indicates the growth rate of features and can capture high-order complex features; the periodic function sin(πt) is used to capture the edge information of the target, especially for objects with regular shapes such as vehicle edges; fractional terms The features are smoothed to avoid over-amplification of unnecessary detail noise, and the cubic term cos(x 3 ) represents the nonlinear change of the global characteristics of the image.

[0083] Next, we will use a practical example to illustrate how to calculate and apply this solution. In this battle, the helmet detected a fast-moving drone and a stationary enemy tank. To simplify the demonstration, we first calculate the boundary feature extraction process of the drone. Let x = 3, indicating that the feature complexity of the drone is high. At this time, the integral in the formula can be used to extract the edge features of the drone. First, calculate the derivative term:

[0084]

[0085] Then, set the value range of t to [0,3] and substitute the derivative term into the original integral function. At this time, the edge features of the drone will be gradually amplified at different scales. For example, when t = 1, the intensity of the edge features is:

[0086]

[0087] This indicates the degree of extraction of the edge features of the drone. Although the periodic function is partially eliminated, the cosine (1 3 ) still played a role.

[0088] For a stationary tank, x is set to a larger value, such as x=7, to capture the global contour features of a larger target. Through the same calculation process, we can obtain:

[0089]

[0090] The calculation found that as x increases, the influence of global features becomes more and more obvious. For small targets such as drones, the edge features are extracted more accurately, while for large targets such as tanks, the convolution layer can effectively capture their overall contour information. Some coefficients in this formula can be adjusted according to the actual scenario. The specific value range is as follows: For x values, the value range is [0,10], and the value range of t is set between [0,5] to ensure calculation stability.

[0091] The special operations team is faced with a complex combat environment where they are tracking large targets (such as enemy tanks) and small targets (such as drones) at the same time. In order to efficiently handle targets of different sizes in this complex scenario, the target detection system of the tactical AR helmet further introduces the convolutional feature pyramid network FPN. The role of FPN is to process the global features of large targets and the local boundary details of small targets through top-down feature map fusion, ensuring that the system can take into account targets of different scales at the same time during the detection process. This multi-scale processing method is particularly important for tactical scenarios with rapidly changing and multi-dimensional targets.

[0092] In actual operation, when the drone approaches quickly from a distance and the enemy tank moves slowly, the system must be able to capture the features of both targets at the same time. To achieve this goal, FPN starts to work, fusing high-level semantic information and low-level detail features to ensure the accuracy of classification and boundary detection. In the multi-scale feature fusion process, the system uses the following integral formula:

[0093]

[0094] Where x represents the feature scale in the image, and its value range is set to [0,10] to process the feature maps of objects of different sizes. The integral variable t represents the feature maps of the image at different levels, sin(t 3 ) represents the fluctuation of semantic information in high-level features, such as global features used to detect large targets such as tanks, while e -t It is used for low-level features and is particularly suitable for edge feature extraction of small targets such as drones.

[0095] Next, we will explain how to process these targets through calculations. First, the system starts to capture the global features of the distant tank. Let x = 6, which represents the characteristic scale for detecting large targets. In order to calculate the feature extraction of the tank, we first start with the integral formula:

[0096]

[0097] For large objects such as tanks, global semantic features are crucial, so sin(t 3 ) mainly captures the outline and overall shape of the target. Specifically, when t = 1:

[0098]

[0099] When t=3:

[0100]

[0101] It can be seen that as t increases, the weight of semantic features decreases, while low-level edge features become more important. In addition, log(6 4 +1) enhances the smoothness of feature processing, and the result is:

[0102] log(1297)≈7.1697

[0103] Therefore, the global features of the tank are fused to obtain smoother and more accurate results through multi-scale features, ensuring that large targets can be accurately captured in complex backgrounds.

[0104] On the other hand, the system also needs to deal with small drone targets in the distance. In order to ensure that the detailed features of small targets such as drones are fully extracted, x = 2 is set, and the edge features play a dominant role. The formula is:

[0105]

[0106] When t = 1:

[0107]

[0108] When t = 2:

[0109]

[0110] The calculation results show that as the target features are gradually refined, the low-level edge information is effectively extracted. At the same time, log(2 4 +1) is calculated as:

[0111] log(17)≈2.8332

[0112] This item is used to enhance the details of small targets of the drone and ensure that their boundaries can be clearly displayed.

[0113] Through these calculations, FPN successfully combines high-level semantic information with low-level detail features, allowing the system to handle both large and small targets. In practical applications, the system's feature scale x usually ranges from [0,10], which can handle both the boundary features of small targets (such as drones) and the global features of large targets (such as tanks). The integral variable t ranges from [0,5] to control the depth and smoothness of the feature extraction process. Through the flexible adjustment of these parameters, the system can maintain stable and efficient performance in different combat scenarios.

[0114] Special operations teams are using tactical AR helmets to track enemy tanks and drones. The target detection system on the helmet has successfully captured the global and local features of these targets through the convolutional feature pyramid network (FPN). However, as the combat situation changes, the system needs to further optimize the detected candidate target boxes to ensure that the captured targets have accurate boundary positions. At this time, bounding box regression and non-maximum suppression (NMS) technology are introduced to further optimize the boundary positions of candidate targets and reduce the interference of overlapping candidate boxes.

[0115] In this process, bounding box regression is first used to adjust the detected candidate box so that it can accurately fit the actual boundary of the target. For example, when the drone is moving quickly, the candidate box detected by the system is slightly deviated due to the rapid movement of the drone and cannot accurately fit its outline. The goal of bounding box regression is to adjust the position and size of the candidate box through the regression model to reduce this deviation. The formula is as follows:

[0116]

[0117] Among them, x b represents the coordinate information of the candidate box, and its value range is set to [0,10]. t is the intermediate variable involved in the candidate box optimization process. First, the exponential function It indicates the degree of enhancement of the candidate box features and plays a significant role in the detection of fast-moving targets (such as drones). It enhances the accuracy of the bounding box by amplifying high-order features. At the same time, sin(t) is used to capture the fluctuations of the target boundary, especially when the target moves quickly or has a complex shape (such as the rotor of a drone). Sin(t) can fit these subtle boundary changes well.

[0118] To illustrate its practical application, the initial position of the bounding box is set to x b =4. This means that the candidate box roughly locks the position of the drone at the beginning. Next, the gradient change of each layer of the candidate box is calculated by derivation:

[0119]

[0120] Substituting the value of t=1:

[0121]

[0122] When t = 2:

[0123]

[0124] From these calculations, it can be seen that as t increases, the feature changes of the candidate boxes are further enhanced, especially for complex moving targets (such as drones). The system further amplifies the boundary features by increasing the gradient, ensuring that the candidate boxes can better fit the target.

[0125] Next, use The term is used to suppress overlapping candidate frames. When the target objects are close to each other (such as a drone and a tank appearing in the same field of view), multiple candidate frames will overlap. At this time, the NMS technology eliminates unnecessary overlapping candidate frames through this part of the formula. Set the candidate frame position x of the drone b =4, perform calculation to suppress overlap:

[0126]

[0127] By using this small value, the system can effectively suppress overlapping candidate boxes and ensure that the final bounding box only retains the best candidate box. Similarly, for the enemy tank, since its boundary is relatively clear, let x b =7 for bounding box regression and overlap suppression calculation. At t = 2:

[0128]

[0129] For overlap suppression:

[0130]

[0131] Since the tank is a large target, the overlap of its candidate boxes is small, so the suppression effect is weaker, but the accuracy of the bounding box is still ensured. b The value range of is usually between [0,10], which is used to represent the coordinate information of the target and adapt to candidate boxes of different sizes. The value range of t is between [0,5], which is used to optimize the feature changes of the candidate boxes at each layer.

[0132] Embodiment 2:

[0133] Combined with Figure 3, a special operations force is using tactical AR helmets to perform complex urban combat missions. The enemy's behavior changes abnormally and is difficult to predict, especially in narrow urban streets, where the movement trajectories of enemy drones and ground vehicles become more complicated. At this time, the target detection system of the AR helmet not only needs to identify the enemy target, but also needs to assess the threat level of the target in real time.

[0134] The method of dynamically assessing the threat level takes into account the speed and behavior pattern of the target. This method is particularly suitable for identifying abnormal behavior patterns of enemy targets, such as sudden acceleration, frequent turns, irregular movements, etc. At this time, the deep learning module of the helmet will model the behavior of the enemy target, analyze and capture abnormal changes in behavior.

[0135] To model the behavior of the target, the following nonlinear function is used:

[0136]

[0137] Among them, x t Indicates the time step, with a value range of [0,10], which is used to simulate the time process of the target behavior; τ is the time variable in the behavior sequence, which is responsible for capturing the behavior characteristics at different times. It is used to describe the exponential change of target behavior. It is particularly suitable for analyzing sudden acceleration or sudden behavior of enemy targets. For example, when a drone suddenly accelerates from a low speed to a high speed, the system can accurately capture the acceleration behavior through this function. 2 ) is used to capture frequent turns or irregular movements. This function can help the system identify persistent abnormal trajectories of enemy targets, such as vehicles suddenly turning multiple times in narrow streets. Changes in global behavior patterns are achieved through To model the overall behavior pattern of the target, such as long-term irregular movement or abnormal path planning.

[0138] Next, we will demonstrate the feasibility of this solution through example calculations. In a complex tactical scenario, the system detects that the enemy drone is undergoing frequent behavioral changes. First, the drone moves slowly at a low speed, then suddenly accelerates and changes direction frequently in an attempt to evade detection. Set the time step x t =3 to capture the behavior changes of the drone in the early stage and calculate the results of the nonlinear function.

[0139] First, calculate the exponential term in the behavioral model and the periodic term sin(τ 2 ):

[0140] 1. When τ = 1:

[0141]

[0142] 2. When τ = 2:

[0143]

[0144] It can be seen here that as time increases, the change in target behavior is rapidly amplified, especially when the drone suddenly accelerates. The exponential function captures this drastic change in behavior. The periodic function gradually captures the frequent turns of the drone at a high time step. This periodic behavior is consistent with the trajectory of the drone, indicating that the target is trying to evade tracking through a complex path.

[0145] Finally, calculate the change in the global behavior pattern

[0146] cos(3 5 )=cos(243)≈-0.956

[0147] This result reflects that the global behavior of the drone has a large deviation, indicating that its behavior is significantly abnormal in the entire time series. By adding up the individual calculation results, the system can obtain an overall evaluation of the target behavior.

[0148] Therefore, the final calculation result is:

[0149]

[0150] Through this negative result, the system is able to determine that the drone's behavior is abnormal, and based on the exponentially growing behavior pattern, the target has a higher threat level.

[0151] In the same tactical scenario, the enemy's ground vehicle moves regularly, and the system detects that the vehicle's behavior is relatively stable and has no obvious abnormalities. t =6 to capture the complete time series of vehicle behavior, and perform similar calculations:

[0152] 1. When τ = 1:

[0153]

[0154] 2. When τ = 3:

[0155]

[0156] Through this exponential calculation, the system is able to capture the vehicle's stable state with small periodic changes, indicating that the target has a low threat level.

[0157] In practical applications, x tThe value range of is between [0,10], which is usually used to represent the time series of the target behavior, while τ is between [0,5], which represents the target behavior characteristics captured at different times.

[0158] During this exercise, the team encountered multiple enemy targets with different movement speeds and behavior patterns, such as drones flying at high speeds and slow-moving but abnormally behaving enemy vehicles.

[0159] At this stage, the system superimposes speed and behavior patterns through a weighting algorithm to generate a real-time threat level score. In particular, when the target's behavior is abnormal, even if its speed is low, the system will increase the threat score of the target by increasing the weight. For example, if a slow-moving vehicle exhibits abnormal behaviors such as sudden turns and irregular routes, the system will evaluate it as a high-threat target. On the contrary, even if another target (such as a drone) is fast, if the behavior pattern is relatively regular, the system will reduce its threat level.

[0160] To accomplish this real-time threat assessment, the system uses the following integration formula:

[0161]

[0162] Among them, w v represents the weight variable in threat assessment, usually in the range of [0,10], and is used to describe the overall threat weight of the target. The integral variable t represents different factors that affect the threat level, such as the target's speed and behavior pattern. First, It reflects the impact of target speed on threat assessment, especially for high-speed targets (such as drones), which has a larger weight increase, while log(u+1) combines the complexity of the target behavior pattern to describe the contribution of abnormal target behavior to the threat level.

[0163] To illustrate the feasibility of this solution, assume that the system detects two targets on the battlefield: a high-speed moving drone and a slow-moving but abnormally behaving enemy vehicle. First, the threat of the drone is evaluated. The drone is fast, so w is set. v =5 to indicate that the target has a higher speed weight. Next, calculate the integral:

[0164]

[0165] First calculate when u=1:

[0166]

[0167] Next, calculate when u=3:

[0168]

[0169] Finally, the terms that suppress the noise and influence are calculated:

[0170]

[0171] Therefore, the final threat score for high-speed drones is:

[0172] h vw (5)≈1.705+98.837+0.0059≈100.548

[0173] The drone received a high threat score due to its high speed, despite its relatively routine behavior.

[0174] Next, we evaluate the slow-moving but abnormally behaving vehicle. Since the vehicle is moving slowly, let w v =3, but the vehicle's behavior pattern is unusual, so its behavior pattern will significantly affect its threat level. Calculate the score:

[0175]

[0176] First calculate when u=1:

[0177]

[0178] Next, calculate when u=2:

[0179]

[0180] Finally, the term that suppresses the noise is calculated:

[0181]

[0182] Therefore, the final threat score for the slow-moving but unusually behaving vehicle is:

[0183] h vw (3)≈1.705+3.27+0.0332≈5.008

[0184] Despite the vehicle's slower speed, its unusual behavior pattern caused the threat score to remain high.

[0185] Through the above calculations, the system can dynamically evaluate the threat level of different targets. For drones, although their behavior patterns are relatively normal, their high-speed flight makes their threat score high. However, slow-moving enemy vehicles, due to their abnormal behavior patterns, still receive a high threat score despite their slow speed. In practical applications, w in the threat assessment formula is v The value range is [0,10], which is used to represent the weights of different threat factors, and the value range of u is [0,5], which is used to describe the influence of different speed and behavior factors.

[0186] To further improve combat efficiency, commanders hope to be able to adjust the threat assessment criteria and weights for different targets based on specific combat mission requirements and actual scenarios. For different combat environments, the behavior of certain targets is more important. For example, in an encirclement and suppression mission, the sudden acceleration and abnormal behavior of drones are considered to be a higher threat. In some reconnaissance missions, vehicles that move slowly but exhibit abnormal behavior require a higher priority. To achieve this, the system allows users to preset behavior and speed types, and dynamically change the parameters of threat assessment through personalized weight adjustments.

[0187] In actual applications, the system uses the following formula to adjust the threat assessment weight of personalized regulation:

[0188]

[0189] Among them, p represents the parameter variable of personalized adjustment, and its value range is usually [0,10], which is used to adjust the weight according to different task requirements. v is the time variable in the behavior sequence, which controls the sudden changes and impact range of the behavior pattern.

[0190] Example 1: Personalized control of drones in encirclement and suppression missions

[0191] Suppose that the special operations forces are carrying out a siege mission, and the mission scenario requires special attention to the sudden acceleration behavior of the enemy UAV. In order to deal with this high-threat situation, the commander hopes to give a higher weight to the suddenness and acceleration behavior of the UAV through the system. The system will adjust the personalized parameter values ​​according to this requirement.

[0192] Set p = 5 to indicate a high priority assessment for the drone. First, calculate the behavior burst weight in the formula:

[0193]

[0194] First, calculate when v=1:

[0195]

[0196] Then, calculate when v=3:

[0197]

[0198] Next, the term that adjusts the global weight is calculated:

[0199] log(5 2 +1)=log(25+1)=log(26)≈3.258

[0200] Therefore, through the comprehensive calculation after personal adjustment, the system gives a higher weight to the suddenness of the drone's behavior, and the result is:

[0201] k p (5)≈0.154+0.00476+3.258=3.41676

[0202] This shows that the threat assessment results of the drone during sudden acceleration are significantly improved, and the system is able to respond quickly and adjust priorities based on the commander's needs.

[0203] Example 2: Personalized control of slow-moving vehicles in reconnaissance missions

[0204] In another reconnaissance mission, the commander wants to focus on enemy vehicles that are moving slowly but showing abnormal behavior. In this case, the slow-moving target itself is not a high threat, but its abnormal behavior, such as frequent turns or sudden stops, indicates a higher potential risk. The system needs to adjust the weight of the behavior pattern based on this requirement.

[0205] For this vehicle, p=3 is set to indicate a high priority evaluation of the behavior pattern. First, the weight of the sudden behavior is calculated:

[0206]

[0207] Calculate when v = 1:

[0208]

[0209] Then, when v=2:

[0210]

[0211] Finally, the terms for global adjustment weights are calculated:

[0212] log(3 2 +1)=log(10)≈2.303

[0213] Therefore, the threat assessment results after personalized regulation are:

[0214] k p (3)≈0.154+0.0267+2.303=2.4837

[0215] Through this personalized adjustment, the system improves the threat assessment level of the vehicle in abnormal behavior mode, ensuring that combatants can pay attention to these targets that pose potential risks in a timely manner. In actual tactical applications, the value range p of personalized regulation is usually between [0,10], so that the weight can be flexibly adjusted according to the mission scenario. The value range of the time variable v is usually [0,5], which is used to balance the suddenness of behavior and the global weight.

[0216] Embodiment 3:

[0217] In an urban street fighting exercise, special operations teams are using tactical AR helmets to monitor and capture enemy targets in real time. At this time, the helmet's camera starts working, capturing and analyzing multiple targets in the scene in real time, including enemy soldiers, drones, and ground vehicles. The shape, color, texture, and speed of each target are detected and recorded by the system in real time. In order to efficiently distinguish these different categories of targets in a complex tactical environment, the system uses a classification model to quickly classify each target according to predefined categories.

[0218] First, the system's detection module receives real-time data from the camera and captures the shape, color, and texture information of the enemy target. For example, when an enemy soldier suddenly appears in the shadow of a building, the system uses the detection module to identify the soldier's shape and outline, while extracting the color and texture features of his clothing. Next, through the preliminary detection algorithm, the system separates the candidate target area in the image from the background. At this point, the system has roughly locked the area where the soldier is located.

[0219] To further classify the objects, a nonlinear transfer function is used:

[0220]

[0221] Among them, d f Represents the characteristic dimension of the target data, usually in the range of [0,10], used to describe feature data from simple to complex. z is an intermediate variable, representing the scale of feature change of the candidate target area. Exponential function It is used to nonlinearly amplify the features in the target area and is suitable for processing the complex shapes and detail changes of the target. For example, the outline of the enemy soldier changes more obviously in the background. Through the enhancement of the exponential function, the system can more accurately identify its edges and shapes. 2 ) is used to capture the periodic changes of local features, which is very useful in identifying the rotor motion of a drone or the texture of a vehicle. The function is used for smoothing to ensure that noise interference is reduced during data transmission, thereby obtaining stable classification results.

[0222] Next, we will demonstrate the feasibility of this process through example calculations. Assume that the system captures an image of an enemy drone. The rotor and fuselage are complex and require fine feature extraction through the system. Set the feature dimension d of the target data f =4 to represent the complex shape characteristics of the drone. First, the nonlinear amplification part of the calculation formula is:

[0223]

[0224] First calculate the integral part. When z = 1:

[0225]

[0226] When z = 3:

[0227]

[0228] As z increases, the system is able to significantly zoom in on the intricate details of the drone, ensuring that the drone's edges and rotors are captured accurately.

[0229] Next, the smoothing part is calculated:

[0230] log(4 5 +1)=log(1024+1)=log(1025)≈6.932

[0231] Therefore, the final transfer function calculation result is:

[0232] f c (4)≈2.287+2.191×10 11 +6.932≈2.191×10 11

[0233] This result shows that through the nonlinear amplification of the transfer function, the system is able to effectively extract the complex features of the drone in a complex background and pass it as input to the classification model.

[0234] The classification model then further processes the received candidate target area information. The shape features of the drone, the periodic changes of the rotors, and the speed information are integrated by the system and classified as a "drone" through a deep learning algorithm. At the same time, the enemy vehicle on the other side is also monitored and classified by the system in real time. The vehicle's slow movement, large shape, and regular texture are identified. Through the same feature extraction process, the classification model classifies it as a "ground vehicle." In actual applications, the system's d f The value range is [0,10], which is dynamically adjusted according to the complexity and feature dimension of the target; while the value range of z is [0,5], which is used to represent feature changes at different scales.

[0235] Special operations teams are using tactical AR helmets for target detection. The detection module not only needs to identify enemy targets, but also accurately classify different types of targets. To further improve the accuracy of classification, the system uses a convolutional neural network (CNN) and uses a feature enhancement mechanism to enhance local information in the input image. This feature enhancement focuses on the edges, corners, and texture patterns of the target, and is particularly suitable for identifying fast-moving small targets and large targets with clear edges in complex backgrounds.

[0236] In the actual scene, an enemy drone suddenly and quickly appears above a building, while another ground vehicle moves slowly, showing a more complex outline and texture features. At this time, the system's convolutional neural network starts to work, and through the feature enhancement mechanism, it further strengthens the drone's rotor edge information and the vehicle's texture features to ensure that the classification model can accurately distinguish them.

[0237] The process of feature enhancement is modeled by the following function:

[0238]

[0239] Where l represents the local attribute dimension of the target feature, with a value range of [0,10], which is used to process local features from simple to complex. s is the time variable in the local feature enhancement process, which is responsible for capturing feature changes at different time steps. Used to magnify complex local features, such as the complex edges of drone rotors. The exponential term can quickly magnify these details. The periodic function cos(s 3 ) is used to capture fast changes in edges and is particularly suitable for identifying vehicle textures and corners. The denominator 1+s 4 It is used for smoothing to avoid excessive local feature response and affect the stability of classification results.

[0240] Next, we will illustrate the feasibility of this process through specific calculations. First, for the rotor of the drone, we set the local feature dimension l = 3 to process the complex details of the drone edge. We use the above formula for calculation:

[0241]

[0242] First, calculate the integral part. For s = 1:

[0243]

[0244] Next, calculate s=2:

[0245]

[0246] Finally, calculate the smoothing control part:

[0247]

[0248] Therefore, the feature enhancement result for the edge of the drone rotor is:

[0249] h e (3)≈1.629+3.211+0.0332=4.8732

[0250] This result shows that the system is able to effectively capture the edge details of the drone by magnifying local features, ensuring that its complex rotor features are accurately classified.

[0251] Next, the system needs to process the texture features of ground vehicles. Vehicles move slowly, have complex contours, and have significant texture changes. Set l = 6 to process the local texture features of the vehicle. The calculation formula is as follows:

[0252]

[0253] Calculate the integral part, when s = 1:

[0254]

[0255] When s=3:

[0256]

[0257] Calculation control items:

[0258]

[0259] Therefore, the enhancement result for vehicle texture features is:

[0260] h e (6)≈1.629+98.83+0.00239=100.46139

[0261] This calculation shows that the system effectively amplifies the complex texture features of the vehicle through the exponential term, so that the edge and texture details of the vehicle can be accurately captured and classified. In practical applications, the value of l in the system is usually in the range of [0,10], which is adjusted according to the complexity of different targets. The value of s is in the range of [0,5], which is used to control the amplification and capture of local features.

[0262] Special operations forces not only need to quickly identify enemy drones, vehicles, and personnel, but also need to further classify targets based on the enemy's weapons and behavior patterns. At this time, the classification architecture based on multi-task learning adopted by the tactical AR helmet begins to play a key role. The system can not only distinguish different target types, but also perform more detailed classification based on their additional attributes, such as weapon type, target behavior pattern, etc. This multi-task learning architecture allows multiple tasks to be classified at the same time, enabling combatants to quickly and accurately obtain detailed information about the enemy, thereby providing support for combat decisions.

[0263] In this exercise, the team's AR helmet captured three targets: a slowly moving armored vehicle, a fast-flying drone, and an enemy soldier with a gun. To ensure that these targets are carefully classified, the system uses a multi-task learning architecture to not only identify the target type, but also classify its weapons and behavior patterns in detail. The features of each target are input into the multi-task learning classification model, and the model extracts features using the following formula:

[0264]

[0265] Here, x represents the feature dimension of the target, and its value range is between [0,10]; and t is an intermediate variable involved in the feature extraction process, which controls the sharing and adjustment of the features of each task in multi-task learning. 4 ) is responsible for capturing periodic feature changes in multiple tasks and is particularly suitable for distinguishing multiple target types. For example, the system uses this function to identify the rapid periodic changes of drone rotors and the slower rotation cycle of vehicles. -t It is used to balance the feature changes in the time series and ensure that the features in the time series are not lost over time. 2 +1 is used to balance the feature strength of each task, and log(x 3 +1) Control the overall feature distribution through global regulation to ensure that the system effectively shares resources among multiple tasks.

[0266] First, the system starts to process the enemy's armored vehicle. The vehicle moves slowly, and the system captures its armor thickness, contour features, and equipped weapons information. The feature dimension of the vehicle is set to x = 6, and the feature extraction formula is:

[0267]

[0268] First, calculate the integral part. When t = 1:

[0269]

[0270] When t=3:

[0271]

[0272] Next, calculate the global adjustment part:

[0273] log(6 3 +1)=log(216+1)=log(217)≈5.379

[0274] Therefore, the feature extraction result of the armored vehicle is:

[0275] k(6)≈0.154+0.00313+5.379=5.53613

[0276] This result shows that the system successfully captures the global characteristics and detailed changes of the vehicle through feature enhancement, and is particularly effective in identifying the weapon equipment and slow-moving characteristics of armored vehicles.

[0277] Next, the system processes a fast-moving drone. The drone's rotors change rapidly and the system needs to identify its flight mode, weapon type, and onboard sensors. Set x = 4 to represent the characteristic dimension of the drone. The calculation formula is as follows:

[0278]

[0279] When t = 1:

[0280]

[0281] When t = 2:

[0282]

[0283] Finally, the global adjustment part is calculated:

[0284] log(4 3 +1)=log(64+1)=log(65)≈4.174

[0285] Therefore, the feature extraction results of the drone are:

[0286] k(4)≈0.154-0.00775+4.174=4.32025

[0287] Through this result, the system is able to effectively identify the drone's periodic motion characteristics and flight patterns, while accurately classifying its weapon configuration and sensor functions.

[0288] Finally, the system identifies an enemy soldier with a gun. Since the target's behavior pattern is relatively complex, the system needs to not only classify its weapon type, but also analyze its behavior. Set x = 3 to represent the characteristic dimension of the soldier. The calculation is as follows:

[0289]

[0290] When t = 1: 0.154

[0292] When t = 2:

[0293] -0.00775

[0294] Global adjustment section:

[0295] log(3 3 +1)=log(28)≈3.332

[0296] The final soldier feature extraction result is:

[0297] k(3)≈0.154-0.00775+3.332=3.47825

[0298] The calculation shows that the system can identify the characteristics of soldiers holding guns and moving behavior patterns, ensuring that high-threat targets are dealt with first during combat. Through this multi-task learning architecture, the system can identify the target type and classify the additional attributes of each target, such as weapon equipment and behavior patterns, according to different mission requirements.

[0299] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. The target detection method of the tactical AR helmet is characterized by The following steps are involved: S1, head posture perception and spatial positioning module: S1.

1. After the helmet is started, the inertial measurement unit (IMU) sensor collects the acceleration and angular velocity data of the head through the gyroscope, accelerometer and magnetometer; S1.2, using sensor fusion algorithms including Kalman filtering to integrate data from multiple sensors; S1.3, create spatial mapping between virtual scene and real scene; S2. Triggering of target detection module: S2.1, input the real-time video frames captured by the camera into the pre-trained convolutional neural network (CNN) model; S2.2, CNN model performs feature extraction based on different detection tasks including target classification and object boundary detection; S2.3, multi-scale detection of targets through multi-layer convolution and pooling operations; S2.4, use the region proposal network RPN to generate multiple candidate boxes; S3, Real-time target tracking and calibration: S3.1, the detected target position is passed to the target tracking algorithm to initialize tracking; S3.

2. Adjust the size and position of the tracking frame according to the change of the viewing angle of the helmet wearer; S4. Augmented reality assisted target information feedback: S4.

1. Generate target attribute data based on target detection and tracking results, and display the information in the form of AR tags in the user's field of view; S4.

2. Use laser ranging to calculate the distance between the target and the user in real time and display it on the AR interface; S4.

3. Dynamically assess the threat level based on the target's movement speed and behavior pattern information, and distinguish them with different colors or graphics; S5. Target classification and prioritization: S5.1, the detection module passes the identified target information to the classification model, and the classification model classifies the target according to the predefined categories; S5.2, displaying the targets in different levels according to preset priority rules, including the size, speed and distance of the targets; The method used for dynamically assessing the threat level includes: integrating the speed of the target and analyzing the behavior pattern of the target; the behavior pattern is abnormal behavior, including sudden acceleration, frequent turns, and irregular movement; modeling the behavior sequence of the target through deep learning technology, analyzing and capturing abnormal behavior, and assessing the potential threat of the target; using nonlinear functions to model the behavior data: Among them, x t represents the time step, τ is the time variable in the behavior sequence; Represents the exponential growth of the target behavior change, used to describe the situation of sudden acceleration or sudden behavior; sin(τ 2 ) is used to describe periodic changes in behavior and to capture frequent turns and irregular movements; Represents changes in global behavior patterns and is used for modeling a wide range of behavior patterns; The method for dynamically assessing the threat level includes: after obtaining the speed and behavior pattern information of the target, superimposing the target's threat level in real time through a weight algorithm; the threat assessment is adjusted according to the target's behavior changes, including that abnormal target behavior will increase the target's threat weight, even if its movement speed is low. When assessing the threat level, an integral formula is used to weight the sum of different factors: Among them, w v represents the weight variable in threat assessment, It is used to describe the impact of target speed on threat assessment, especially for high-speed targets; log(u+1) is used to describe the contribution of abnormal behavior to the threat level in combination with the complexity of the target's behavior pattern; 1+u in the denominator 4 To suppress noise and excessive effects, Then balance the speed and behavior factors of the target; The method for dynamically assessing the threat level includes: performing personalized regulation according to user needs or task scenarios; adjusting the user's preset behavior and speed type through personalized weights, and dynamically adjusting the weight parameters of the threat assessment using a function: Where p represents the parameter variable of personalized adjustment; sin(v 3 ) is used to regulate the emergent behavior in the behavior pattern, e -v It regulates the scope of influence of the behavioral pattern.

2. The target detection method of the tactical AR helmet according to claim 1 is characterized in that The method for performing feature extraction for different detection tasks includes: S1. First, the primary convolutional network is responsible for extracting common low-level image features, including edges and textures. The primary convolutional network will be used for multiple tasks, and the classification task and the boundary detection task will be processed differently through the customized convolutional layer. S2. Introduce the convolutional feature pyramid network FPN; through top-down feature map fusion, process the global features of large objects and the local boundary details of small objects; S3, combined with bounding box regression and non-maximum suppression NMS technology; through bounding box regression, the detected candidate box is further optimized to predict the boundary position of the object; while non-maximum suppression suppresses overlapping candidate boxes; the bounding box regression process uses the formula to optimize the candidate box: Among them, x b Represents the coordinate information of the candidate box, and t is the intermediate variable involved in the optimization process; Indicates the degree of enhancement of features between candidate boxes; sin(t) is the boundary fluctuation of the candidate box, which is used to fit the boundary changes of the target; Operation, calculate the gradient change of each layer of candidate boxes; The item is used to suppress overlapping candidate boxes.

3. The target detection method of the tactical AR helmet according to claim 1 is characterized in that The method for classifying the target according to the predefined category by the classification model includes: the detection module is responsible for capturing the shape, color, texture, and speed information of the target in real time, and extracting the candidate target area through the preliminary detection algorithm; through the nonlinear transfer function: The extracted candidate target region information is passed as input to the classification model, where d f Represents the characteristic dimension of the target data, z is an intermediate variable, representing the characteristics of the candidate target area at different scales; represents the nonlinear amplification of features in the candidate region; sin(z 2 ) is used to capture the periodic characteristic changes of the local area; The function smoothes the data as it passes through.

4. The target detection method of the tactical AR helmet according to claim 3 is characterized in that The classification model adopts convolutional neural network (CNN) and enhances local information in input features, including edges, corners and texture patterns, through feature enhancement mechanism; the feature enhancement process is modeled by function: Among them, l represents the local attribute dimension of the target feature, Indicates the exponential growth of local features, amplifying complex local features, cos(s 3 ) captures the rapid changes of local edge features; the denominator 1+s 4 Used to smooth characteristic responses; It is used to control the influence of local features.

5. The target detection method of the tactical AR helmet according to claim 3 is characterized in that The classification model adopts a multi-task learning-based architecture to classify target types while performing classification based on multiple target attributes; the classification includes distinguishing between personnel, vehicles, and drone target types, and performing detailed classification based on the target's weapon equipment and behavior patterns.

Images