A method for continuous directional tracking of a key target under a perspective of an unmanned system

Abstract

The application discloses a kind of key target's continuous directional tracking method under unmanned system visual angle, belong to unmanned system and computer vision field, and the target under unmanned system visual angle is tracked directionally, solve the problem of existing technology unmanned system video target tracking difficulty, the system includes: key target detection module, feature extraction module based on multi-dimensional feature fusion, multi-task information coding module and directional tracking module.Key target detection module detects the key target in video frame;Feature extraction module based on multi-dimensional feature fusion carries out extraction and fusion of important information in video frame;Multi-task information coding module carries out compression coding and directional pose prediction to feature information;Directional tracking module tracks and pose calibration to key target, the method of the application can accurately, efficiently track directionally key target under unmanned system visual angle, and provide effective help for the application popularization of unmanned system.

Description

Technical Field

[0001] This invention relates to the fields of unmanned systems and computer vision technology, specifically to a method for continuous directional tracking of key targets from the perspective of an unmanned system. Background Technology

[0002] In recent years, unmanned systems technology has developed rapidly and has been widely applied in fields such as agricultural planting, bridge inspection, aerial photography for entertainment, and traffic monitoring, becoming an important platform for information acquisition. Compared with traditional fixed imaging systems, unmanned systems are more mobile and can maintain shooting for a longer period. Although unmanned systems have many advantages, their inherent characteristics also bring many new challenges to computer vision research.

[0003] For videos generated by unmanned systems (UAVs) depicting high-altitude and long-distance scenes, the video footage covers a wide range of scenes, and the background information is more complex than that of natural images. Furthermore, due to the inherent motion characteristics of UAVs, the footage is subject to mechanical shaking compared to surveillance videos, and the shooting angle also changes randomly, causing the target's direction and shape to constantly shift. This necessitates that the tracking network possess both stability and adaptability, capable of learning the target's key features to address changes in target direction and shape. Intelligent analysis technology for UAV system videos has long been a research focus, and accurate and continuous target orientation tracking remains one of the most challenging problems. Moreover, with the increasingly widespread application of unmanned technology and remote sensing target detection, the importance of directional target tracking in UAV system videos is becoming increasingly apparent. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, artificial intelligence technology is used to perform targeted tracking of key targets in unmanned systems, thus solving the difficulty of video target tracking in existing unmanned systems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for continuous directional tracking of key targets from the perspective of an unmanned system includes a key target detection module, a feature extraction module based on multi-dimensional feature fusion, a multi-task information encoding module, and a directional tracking module, specifically including the following steps: S1. The key target detection module uses the detector to detect key targets in the video captured from the perspective of the unmanned system frame by frame. When a specified key target is detected, the detector stops running, outputs the horizontal position information and category information of the key target in the current frame, and specifies the next frame as the input of the feature extraction module of the multi-dimensional feature fusion. S2. The feature extraction module based on multi-dimensional feature fusion extracts multi-scale and multi-dimensional feature information through multi-dimensional feature fusion to obtain a feature map with strong representational ability for the current input frame. S3. The multi-task information encoding module takes the feature map of the current video frame output in step S2 and the horizontal position information of the previous video frame and its corresponding key target as network inputs, respectively, and outputs the target position encoding information of the current frame and the target position offset of the previous frame through the target position information encoding network and the orientation pose information extraction network in parallel. S4. The orientation tracking module takes the encoded information of the current frame feature map output in step S3, the orientation pose offset of the target in the previous frame, and the horizontal position information of the key target in the previous frame as inputs. It uses the key target tracking network to track and locate the key target in the current frame, outputs the horizontal position information of the key target in the current frame, and uses the target orientation pose calibration network to determine the orientation pose information of the key target in the previous video frame. S5. Iteratively execute steps S2, S3, and S4 until step S4 outputs the horizontal position information of the last frame of the current video. Using the last frame of the video as input, the orientation pose information of the key target in the last frame is obtained through the orientation pose information extraction network in the multi-task information encoding module and the target orientation pose calibration network in the orientation tracking module.

[0006] As a preferred embodiment of the present invention: the model optimization process of the feature extraction module, multi-task information encoding module, and directional tracking module based on multi-dimensional feature fusion is as follows: For video sequences acquired from the perspective of an unmanned system, given the horizontal position information and classification information of the key target in the first frame, the system iterates and optimizes sequentially according to steps S2, S3, S4, and S5 until the evaluation result of the directional tracking of the key target reaches the preset condition or the iteration ends. The continuously directional tracking model trained in the last iteration is then output. The loss function for the current video sequence is a weighted sum of the target tracking loss and the target pose prediction loss, as shown in the following formula: ,in, express Horizontal position information of key targets predicted in frames. express Frame-specific horizontal position information of key targets. express Pose information for key target prediction in frames. express The true pose information of key targets in the frame. This represents the weights of the target tracking loss and the target orientation pose prediction loss.

[0007] As a preferred embodiment of the present invention: the feature extraction module based on multi-dimensional feature fusion includes two networks: a low-order information extraction network and a multi-dimensional information fusion network. The feature extraction module based on multi-dimensional feature fusion operates by the following steps: S7, using the first video frames As input to the low-order information extraction network, the low-order information extraction network fits two different convolutions, generating feature maps containing multi-scale information through different sampling rates. ; S8. Load the feature map generated in S7. The multidimensional information fusion network extracts the channel and spatial information of the feature map through squeezing and activation operations and double pooling operations, respectively, to obtain rich feature information and generate a feature map. .

[0008] As a preferred embodiment of the present invention, the process for extracting the channel information is as follows: S811, Input Features By grouping channels along the channel dimension and performing convolutions at different scales on channels in different groups, feature maps at different scales are obtained. S812. Use squeezing and excitation operations to obtain channel attention vectors of feature maps at different scales; S813. Use the Softmax function to recalibrate the channel information vectors of feature maps at different scales to obtain the feature weights at different scales after calibration. S814. Perform element-wise dot product on the calibrated weights and the corresponding feature maps to obtain the output features; The formula for performing element-wise multiplication of the calibrated weights and the corresponding feature maps in S814 is as follows: ,in, Represents grouped convolution. This indicates squeezing and excitation operations. This represents the Softmax function.

[0009] As a preferred embodiment of the present invention, the spatial information extraction process is as follows: S821, Input Features The average and maximum values ​​of each channel are calculated based on spatial location to obtain two features, and these two features are then concatenated. S822. Perform fixed-size convolution and normalization operations on the concatenated features to obtain the output features; S823. Finally, the features obtained after processing the two dimensions of information are summed to obtain the output features. ; The formula for performing fixed-size convolution and normalization operations on the stitched features in S822 is as follows: ,in This indicates the operation of taking the maximum value of the feature map. This indicates the operation of averaging the feature map. This indicates a splicing operation. The convolution operation represents the spatial information extraction process. This represents the Sigmoid function, and the output feature in S823 is... The output formula is: .

[0010] As a preferred embodiment of the present invention: the multi-task information encoding module includes two parallel networks: a target position information encoding network and a orientation pose information extraction network. The multi-task information encoding module specifically includes the following steps when it is working: S9. The output feature map of the feature extraction module based on multi-dimensional feature fusion. and The frame and its corresponding horizontal position information are used as input. The target position information encoding network encodes and concatenates key information from the input feature map through a prior knowledge-guided approach, generating the final feature encoding information. The formula is as follows:

[0011] in, , These represent two encoding functions for the target location information encoding network; S10, Use The frame and its corresponding horizontal position information are used as input. The orientation pose information extraction network predicts the orientation pose of the target within the target's horizontal position information and outputs... Frame target orientation pose offset .

[0012] As a preferred embodiment of the present invention: the positioning and tracking module includes a key target tracking network and a target orientation and pose calibration network, and the positioning and tracking module specifically includes the following steps when it is working: S11. Use the encoded information output by the target location information encoding network in step S9. As network input, multiple convolutions are connected to locate and track targets in the current frame, and the horizontal position information of key targets in the current frame is output. ; S12. Use the orientation pose information extraction network from step S10 to output the predicted offset of the orientation pose of the key target. As input to the target orientation pose calibration network, the target orientation pose is fine-tuned and calibrated, and the target orientation pose calibration result is output. ; Among them, the horizontal position information of key targets , This indicates the position of the center point of the target horizontal bounding box. Indicates the width of the target box. Indicates the height of the target bounding box. Target orientation and pose calibration results. , This indicates the angle between the target orientation frame and the horizontal line.

[0013] As a preferred embodiment of the present invention: the specific prediction process of the orientation pose information in S10 is as follows: the orientation pose information extraction network extracts the input... Using the target's horizontal position information in the video frame as a reference, features within the horizontal position are extracted to generate the key target orientation pose prediction offset. The objective function formula for predicting the offset of the key target orientation pose is as follows:

[0014]

[0015]

[0016]

[0017]

[0018] in This represents the orientation and pose information predicted for key targets. This represents the true orientation and pose information of the key target. This indicates the modulo operation.

[0019] As a preferred embodiment of the present invention: the loss function of the key target tracking network is the generalized intersection-union loss function, as shown in the following formula:

[0020] in, This indicates the horizontal position information of key targets. This indicates the true horizontal position information of key targets.

[0021] As a preferred embodiment of the present invention: the specific fine-tuning and calibration process of the target orientation pose information in S12 is as follows: for the input Key target orientation pose prediction offset of video frame The target orientation pose calibration network fine-tunes and calibrates the orientation pose to the original video frame. The calculation formula for the key target orientation pose calibration is as follows:

[0022] in, This represents the target orientation and pose calibration network. Represents network After obtaining the offset by the weight parameters, the network fine-tunes and calibrates the offset to output the final key target orientation and pose prediction information. The loss function of the target orientation pose calibration network is: The loss is calculated using the following formula: .

[0023] Compared with the prior art, the beneficial effects of the present invention are: 1. The feature extraction module based on multi-dimensional feature fusion in this invention uses a low-order information extraction network to extract important information from the video frames of the unmanned system, which is used for iterative training of the key target tracking network to achieve accurate positioning of key targets from the perspective of the unmanned system. 2. The multi-task information encoding module in this invention compresses and encodes the feature map, and predicts the target pose by using the main information within the horizontal position of the key target, thereby achieving efficient localization of key targets in unmanned systems. 3. The directional tracking module in this invention uses information encoding to locate and track key targets, and fine-tunes and calibrates the target pose to achieve precise continuous directional tracking of key targets in unmanned systems. Attached Figure Description

[0024] Figure 1 This is an overall flowchart of the continuous directional tracking method for key targets from the perspective of an unmanned system, as described in this invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Please see Figure 1 The core idea of ​​this invention is to use a feature extraction module based on multi-dimensional feature fusion to extract and fuse important information in video frames, use a multi-task information encoding module to compress and encode feature information and predict target orientation pose, and finally use a directional tracking module to track and calibrate key targets, so as to achieve the purpose of directional tracking of key targets from the perspective of unmanned systems.

[0027] The overall process of the continuous directional tracking method for key targets from the perspective of an unmanned system, as described in this invention, is as follows: Figure 1 As shown, it includes the following modules and steps: Step A10: The key target detection module uses a detector to detect key targets in the video captured from the perspective of the unmanned system frame by frame. When a specified key target is detected, the detector stops running, outputs the horizontal position information and category information of the key target in the current frame, and designates the next frame as the input of the feature extraction module for multi-dimensional feature fusion. The feature extraction module for multi-dimensional feature fusion extracts features from the video frames from the perspective of the unmanned system. This module contains two networks: a low-order information extraction network and a multi-dimensional information fusion network. The method proposed in this module mainly uses the low-order information extraction network to extract important information from the image, and then fuses spatial features and channel features using the extracted information. The steps are as follows: Step A20: Use video frames from the perspective of the unmanned system as input to the low-order information extraction network. The low-order information extraction network fits two different convolutional modules and generates feature maps containing multi-scale information through different sampling rates. Step A30: Load the multi-scale feature map generated in A20. The multi-dimensional information fusion network extracts the channel information and spatial information of the feature map through squeezing and excitation operations and double pooling operations, respectively, to obtain rich feature information and generate feature maps. Multi-scale feature maps use features learned at multiple scales to enrich the representation of low-level semantic features, which helps to enhance the ability to locate key targets at different scales in video frames. For the extraction of multi-scale feature maps in step A20, optionally, the low-order information extraction network uses dilated spatial convolutional pooling pyramids with several different lengths. Filter Dilated convolutions are performed on the mappings of features extracted by the network to sample the output feature maps. for: in, Indicates the network input number. One video frame, This indicates the weight of the module. This represents the mapping of features extracted by the network. The sampling rate is optional. Low-order information extraction networks use sampling rates of 6, 12, 18, and 24. Different sampling rates can extract features at different scales and generate multi-scale feature maps.

[0028] In step A30, multi-dimensional information fusion is used to extract channel feature information and spatial feature information from the video frame. Channel information extraction is the extraction of input features. The algorithm groups channels along the channel dimension and performs convolutions at different scales on channels within different groups to obtain feature maps at different scales. Then, a squeezing excitation method is used to obtain channel attention vectors for feature maps at different scales. Next, the Softmax function is used to recalibrate the channel information vectors of the feature maps at different scales, obtaining calibrated feature weights at different scales. Finally, the calibrated weights and the corresponding feature maps are multiplied element-wise to obtain the output features. The formula is as follows: in, Represents grouped convolution. This represents the attention vector extraction operation for feature map channels at different scales. This represents the Softmax function.

[0029] Spatial information extraction is the extraction of input features The average and maximum values ​​of each channel are calculated based on spatial location to obtain two features. These two features are then concatenated. Finally, a fixed-size convolution and normalization operation is performed on the concatenated features to obtain the output feature. The formula is as follows: in, This indicates the operation of taking the maximum value of the feature map. This indicates the operation of averaging the feature map. This represents the convolution operation of the spatial information extraction module. This represents the Sigmoid function. Finally, the features obtained from merging the two modules are summed to obtain the output features. The formula is as follows: The multi-task information encoding module compresses and encodes the feature map and predicts the pose of the key target. This module contains two parallel networks: a target position information encoding network and a orientation pose information extraction network. The method proposed in this module mainly encodes and concatenates the feature map information, the previous video frame, and the horizontal position information of its key target to generate feature encoding. It also extracts key information from the horizontal position information of the previous video frame and its key target to generate the pose offset of the key target in the previous frame. The steps are as follows: Step A40: Using the output feature map of the feature extraction module based on multi-dimensional feature fusion. and The frame and its corresponding horizontal position information are used as input. The target position information encoding network encodes and concatenates key information from the input feature map through a prior knowledge-guided approach, generating the final feature encoding information. ; Step A50, using The frame and its corresponding horizontal position information are used as input. The orientation pose information extraction network predicts the pose of the target within the horizontal position and outputs... Offset of frame target pose information ; For the generated encoded information in step A40, optionally, the multi-task information encoding module performs attention convolution operations on the feature map. and Key information about the target is extracted from the frame and its corresponding horizontal position, and then weighted to limit the process. The influence of prior knowledge about frames on the feature information of the current frame requires calculating the encoding information on the horizontal axis and the vertical axis separately during the encoding process, mapping them onto a vector of dimension 10000, and finally concatenating the two encoding information to obtain the feature encoding information of the current video frame. The formula is as follows: in, express Video frames, express Horizontal position information of key targets in the frame. This indicates a splicing operation. These represent the positions of the eigenvectors on the horizontal and vertical axes in the feature map, respectively. Indicates the index of the channel. This indicates the dimension in which the feature encoding information output by the network is located. Indicates restriction Weights of frame information.

[0030] For the offset of the key target pose information generated in step A50, the orientation pose information extraction network uses the target's horizontal position information as a reference to extract features within the horizontal position, generating key target orientation pose prediction information. Key target orientation pose offset. The objective function formula is as follows: in This represents the pose information predicted for the key target. This indicates the position of the center point of the target horizontal bounding box. Indicates the width of the target box. Indicates the height of the target bounding box. Indicates the angle between the target orientation box and the horizontal line. This represents the true pose information of the key target. This indicates the modulo operation.

[0031] The directional tracking module reads feature-encoded information, locates key targets, and calibrates their orientation and pose information. This module includes two networks: a key target tracking network and a target orientation and pose calibration network. Target orientation and pose refer to a non-horizontal target bounding box with a rotation angle, which can accurately cover key target information and reduce information redundancy. The main concept of the method proposed in this module is how to extract the horizontal position information of key information through the key target tracking network and calibrate the predicted offset of the key target orientation and pose to the original video frame through the target orientation and pose calibration network. The steps are as follows: Step A60: Use the encoded information output by the target position information encoding network in step A30 as the input of the key target tracking network, connect multiple convolutional modules to locate and track the target in the current frame, and output the horizontal position information of the key target in the current frame. Step A70: Use the key target pose prediction offset output by the orientation pose information extraction network in step A50 as the input to the target orientation pose calibration network to fine-tune and calibrate the target orientation pose to the original video frame, and output the key target pose prediction result. The optimization process of the feature extraction module, multi-task information encoding module, and directional tracking module based on multi-dimensional feature fusion is as follows: For the video sequence acquired from the perspective of the unmanned system, given the horizontal position information and classification information of the key target in the first frame, iterate and optimize sequentially according to steps A20, A30, A40, A50, A60, and A70. Update the corresponding parameters in the direction of decreasing loss function value through backpropagation until the evaluation result of the directional tracking of the key target reaches the preset condition or the iteration end condition, and output the continuous directional tracking model trained in the last iteration round.

[0032] Target tracking in step A60 is achieved through feature-encoded information. The key target is located in the current frame, and the key target tracking network encodes feature information. Decoding generates a weight value, which is then used to predict the horizontal position of the key target. The target localization information prediction loss function of the key target tracking network is the generalized intersection-union loss, as shown in the following formula: in, This indicates the horizontal position information of key targets. This indicates the true horizontal position information of key targets.

[0033] Step A70, target orientation pose calibration, involves fine-tuning the orientation pose offset of the key target and calibrating it to the original video frame. For the input... Key target orientation pose offset in video frame The formula for calibrating the orientation and pose of key targets is as follows: Where G represents the target orientation and pose calibration network, The weight parameters of network G are represented by the offsets. After obtaining the offsets, the network fine-tunes and calibrates the offsets and outputs the final key target orientation and pose prediction information. The loss function of the target orientation pose calibration network is: The loss is calculated using the following formula: The loss function of the continuous orientation tracking method for key targets from the perspective of the unmanned system in the current iteration process is a weighted sum of the target sequence vector loss and the target orientation pose prediction loss, as shown in the following formula: in, express Horizontal position information of key targets predicted in frames. express Frame-specific horizontal position information of key targets. express Pose information for key target prediction in frames. express The true pose information of key targets in the frame. This represents the weights of the target tracking loss and the target orientation pose prediction loss.

[0034] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for continuous directional tracking of key targets from the perspective of an unmanned system, comprising a key target detection module, a feature extraction module based on multi-dimensional feature fusion, a multi-task information encoding module, and a directional tracking module, characterized in that, Specifically, the following steps are included: S1. The key target detection module uses the detector to detect key targets in the video captured from the perspective of the unmanned system frame by frame. When a specified key target is detected, the detector stops running, outputs the horizontal position information and category information of the key target in the current frame, and specifies the next frame as the input of the feature extraction module of the multi-dimensional feature fusion. S2. The feature extraction module based on multi-dimensional feature fusion extracts multi-scale and multi-dimensional feature information through multi-dimensional feature fusion to obtain a feature map with strong representational ability for the current input frame. S3. The multi-task information encoding module takes the feature map of the current video frame output in step S2 and the horizontal position information of the previous video frame and its corresponding key target as network inputs, respectively, and outputs the target position encoding information of the current frame and the target position offset of the previous frame through the target position information encoding network and the orientation pose information extraction network in parallel. S4. The orientation tracking module takes the encoded information of the current frame feature map output in step S3, the orientation pose offset of the target in the previous frame, and the horizontal position information of the key target in the previous frame as inputs. It uses the key target tracking network to track and locate the key target in the current frame, outputs the horizontal position information of the key target in the current frame, and uses the target orientation pose calibration network to determine the orientation pose information of the key target in the previous video frame. The horizontal position information of the key target , This indicates the position of the center point of the target horizontal bounding box. Indicates the width of the target box. Indicates the height of the target bounding box; Target orientation pose calibration results , Indicates the angle between the target orientation frame and the horizontal line; S5. Iteratively execute steps S2, S3, and S4 until step S4 outputs the horizontal position information of the last frame of the current video. Using the last frame of the video as input, the orientation pose information of the key target in the last frame is obtained through the orientation pose information extraction network in the multi-task information encoding module and the target orientation pose calibration network in the orientation tracking module.

2. The method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 1, characterized in that: The model optimization process of the feature extraction module, multi-task information encoding module, and directional tracking module based on multi-dimensional feature fusion is as follows: For video sequences acquired from the perspective of an unmanned system, given the horizontal position information and classification information of the key target in the first frame, the system iterates and optimizes sequentially according to steps S2, S3, S4, and S5 until the evaluation result of the directional tracking of the key target meets the preset conditions or the iteration ends. The continuously directional tracking model trained in the last iteration is then output. The loss function for the current video sequence is a weighted sum of the target tracking loss and the target pose prediction loss, as shown in the following formula: ,in, express Horizontal position information of key targets predicted in frames. express Frame-specific horizontal position information of key targets. express Pose information for key target prediction in frames. express The true pose information of key targets in the frame. This represents the weights of the target tracking loss and the target orientation pose prediction loss.

3. The method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 1, characterized in that: The feature extraction module based on multi-dimensional feature fusion includes two networks: a low-order information extraction network and a multi-dimensional information fusion network. The feature extraction module based on multi-dimensional feature fusion operates by the following steps: S7, using the first video frames As input to the low-order information extraction network, the low-order information extraction network fits two different convolutions, generating feature maps containing multi-scale information through different sampling rates. ; S8. Load the feature map generated in S7. The multidimensional information fusion network extracts the channel and spatial information of the feature map through squeezing and activation operations and double pooling operations, respectively, to obtain rich feature information and generate a feature map. .

4. The method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 3, characterized in that: The process for extracting the channel information is as follows: S811, Input Features By grouping channels along the channel dimension and performing convolutions at different scales on channels in different groups, feature maps at different scales are obtained. S812. Use squeezing and excitation operations to obtain channel attention vectors of feature maps at different scales; S813. Use the Softmax function to recalibrate the channel information vectors of feature maps at different scales to obtain the feature weights at different scales after calibration. S814. Perform element-wise dot product on the calibrated weights and the corresponding feature maps to obtain the output features; The formula for performing element-wise multiplication of the calibrated weights and the corresponding feature maps in S814 is as follows: ,in, Represents grouped convolution. This indicates squeezing and excitation operations. This represents the Softmax function.

5. The method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 3, characterized in that: The process for extracting the spatial information is as follows: S821, Input Features The average and maximum values ​​of each channel are calculated based on spatial location to obtain two features, and these two features are then concatenated. S822. Perform fixed-size convolution and normalization operations on the concatenated features to obtain the output features; S823. Finally, the features obtained after processing the two dimensions of information are summed to obtain the output features. ; The formula for performing fixed-size convolution and normalization operations on the stitched features in S822 is as follows: ,in This indicates the operation of taking the maximum value of the feature map. This indicates the operation of averaging the feature map. This indicates a splicing operation. The convolution operation represents the spatial information extraction process. This represents the Sigmoid function, and the output feature in S823 is... The output formula is: .

6. The method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 1, characterized in that: The multi-task information encoding module includes two parallel networks: a target position information encoding network and a orientation pose information extraction network. The multi-task information encoding module operates by specifically including the following steps: S9. The output feature map of the feature extraction module based on multi-dimensional feature fusion. and The frame and its corresponding horizontal position information are used as input. The target position information encoding network encodes and concatenates key information from the input feature map through a prior knowledge-guided approach, generating the final feature encoding information. The formula is as follows: in, , These represent two encoding functions for the target location information encoding network; S10, Use The frame and its corresponding horizontal position information are used as input. The orientation pose information extraction network predicts the orientation pose of the target within the target's horizontal position information and outputs... Frame target orientation pose offset .

7. The method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 1, characterized in that: The orientation tracking module includes a key target tracking network and a target orientation pose calibration network. The positioning and tracking module operates by specifically including the following steps: S11. Use the encoded information output by the target location information encoding network in step S9. As network input, multiple convolutions are connected to locate and track targets in the current frame, and the horizontal position information of key targets in the current frame is output. ; S12. Use the orientation pose information extraction network from step S10 to output the predicted offset of the orientation pose of the key target. As input to the target orientation pose calibration network, the target orientation pose is fine-tuned and calibrated, and the target orientation pose calibration result is output. .

8. A method for continuous directional tracking of key targets from the perspective of an unmanned system, as described in claim 6, characterized in that: The specific prediction process for the orientation pose information in S10 is as follows: the orientation pose information extraction network extracts the input... Using the target's horizontal position information in the video frame as a reference, features within the horizontal position are extracted to generate the key target orientation pose prediction offset. The objective function formula for predicting the offset of the key target orientation pose is as follows: in This represents the orientation and pose information predicted for key targets. This represents the true orientation and pose information of the key target. This indicates the modulo operation.

9. A method for continuous directional tracking of key targets from the perspective of an unmanned system, as described in claim 7, characterized in that: The loss function of the key target tracking network is the generalized intersection-union loss function, as shown in the following formula: in, This indicates the horizontal position information of key targets. This indicates the true horizontal position information of key targets.

10. A method for continuous directional tracking of key targets from the perspective of an unmanned system according to claim 7, characterized in that: The specific fine-tuning and calibration process for the target orientation pose information in S12 is as follows: For the input... Key target orientation pose prediction offset of video frame The target orientation pose calibration network fine-tunes and calibrates the orientation pose to the original video frame. The calculation formula for the key target orientation pose calibration is as follows: in, This represents the target orientation and pose calibration network. Represents network After obtaining the offset by the weight parameters, the network fine-tunes and calibrates the offset to output the final key target orientation and pose prediction information. The loss function of the target orientation pose calibration network is: The loss is calculated using the following formula: 。

Images