Sonar target tracking method
By introducing a LoRA layer and a heterogeneous computing architecture into the sonar target tracking algorithm, the problems of insufficient feature discrimination capability and computational resource limitation of the sonar target tracking algorithm in the underwater environment are solved, realizing efficient and real-time sonar target tracking that is adaptable to complex underwater acoustic environments.
Patent Information
- Application Number
- CN202510981255.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-16
- Publication Date
- 2025-10-31
AI Technical Summary
Existing sonar target tracking algorithms suffer from insufficient ability to discriminate target appearance features in underwater environments and limitations in the computing resources of embedded devices, resulting in unstable tracking and computational delays, making it difficult to achieve robust, low-power real-time tracking.
We employ a dual-modal coupled progressive low-rank decomposition learning paradigm, learn the diffraction and deformation features of sonar images through LoRA layers, and combine a heterogeneous computing architecture and a progressive fine-tuning strategy to construct an adaptive sonar target tracking system, achieving dynamic adaptation to the appearance of sonar targets and efficient computation.
Efficient and real-time sonar target tracking was achieved on low-computing embedded devices, which can dynamically adapt to complex underwater acoustic environments, reduce computing resource consumption and latency, and improve tracking accuracy and stability.
Smart Images
Figure CN120876537A_ABST
Abstract
Description
Technical Field
[0001] This application relates to a sonar target tracking method, belonging to the field of underwater unmanned vehicle tracking technology. Background Technology
[0002] Sonar, as a core detection device in the underwater environment, is widely used in marine resource exploration, underwater target surveillance, military defense, and autonomous underwater robot operations. Sonar target tracking technology based on forward-looking sonar images enables continuous and stable tracking of moving underwater targets, which is crucial for improving the autonomous operation capabilities of underwater vehicles. However, current sonar target tracking algorithms used in underwater vehicles still face several technical bottlenecks, limiting their performance and reliability in practical applications.
[0003] Current sonar systems suffer from insufficient ability to discriminate target appearance features. Most mainstream sonar target tracking algorithms are based on general-purpose visual target trackers (such as Siamese networks, correlation filters, or Transformer-based trackers). However, these algorithms are typically trained on natural light image datasets (such as COCO and LaSOT), lacking specific optimization for sonar images. Due to the unique physical characteristics of sonar imaging (such as low resolution, strong noise interference, and blurred target contours), general-purpose visual trackers struggle to effectively extract discriminative features of targets in sonar scenarios. This makes them susceptible to background interference or interference from similar targets during tracking, leading to tracking drift. Furthermore, the appearance of sonar targets can change significantly due to changes in the underwater environment (such as turbidity and sound wave refraction), further exacerbating tracking instability.
[0004] Model optimization challenges under the computational resource constraints of embedded devices. Underwater vehicles typically carry low-power embedded computing devices (such as Jetson series, FPGAs, etc.), with limited GPU memory and computing power. Although the vehicle can acquire new samples of sonar targets in real time during mission execution (e.g., through detection algorithms or manual annotation), existing tracking algorithms that employ full-scale fine-tuning strategies to adapt to new targets need to update all parameters of the entire deep neural network. This not only consumes a large amount of GPU memory but also leads to computational latency, making it difficult to meet real-time requirements. Therefore, existing methods usually only support lightweight updates (such as online classifier adjustment or local parameter optimization), but this strategy is difficult to fully adapt to the dynamic appearance changes of sonar targets, limiting the improvement of tracking accuracy.
[0005] In existing technologies, improvements are made through the following methods:
[0006] Data augmentation and transfer learning: This involves fine-tuning a general tracker on a sonar dataset to improve its adaptability to sonar features. However, sonar data annotation is costly, and the limited amount of data makes it difficult to cover complex and varied real-world scenarios.
[0007] Lightweight network design: Techniques such as pruning, quantization, or knowledge distillation are used to reduce the computational cost of the model. However, these methods usually come at the cost of tracking accuracy, making it difficult to balance performance and efficiency.
[0008] Online learning strategies: Updating only some layers of the network (such as the classification head) can reduce computational overhead, but cannot adapt to global changes in the appearance of the target (such as abrupt changes in scale or pose).
[0009] In summary, existing sonar target tracking technologies are limited by insufficient feature discrimination capabilities and embedded device resource constraints, making it difficult to achieve real-time tracking with high robustness and low power consumption. Summary of the Invention
[0010] To address the problem that existing sonar target tracking technologies are limited by insufficient feature discrimination capabilities and embedded device resource constraints, making it difficult to achieve high robustness and low power consumption in real-time tracking, this application provides a sonar target tracking method.
[0011] This application provides a sonar target tracking method, comprising:
[0012] The target detector and target tracker are trained using existing sonar image data, and a LoRA layer is inserted into the target tracker.
[0013] During target tracking, the sonar image is input to the target detector, and the physical features of the target bounding box obtained by the target detector are simultaneously input to the parameter self-updating system and the target tracker. The parameter self-updating system constructs a sample pool based on the input, and the LoRA layer learns the sample pool to obtain the fine-tuned LoRA parameters, which are then updated to the corresponding LoRA layer of the target tracker.
[0014] The updated target tracker outputs the tracking results.
[0015] Preferably, the LoRA layer includes a diffraction nucleus and a deformable substrate.
[0016] Preferably, the target tracker includes a feature extraction network, an OSTrack backbone network, and an attention fusion network; the OSTrack backbone network is inserted with a LoRA layer;
[0017] The target detector is used to detect the physical features of the bounding box of a target in a sonar image;
[0018] The feature extraction network is used to process the physical features of the target bounding box to obtain diffraction mode features and deformation mode features, and after processing, diffraction feature vectors and deformation feature vectors are obtained.
[0019] The diffraction feature vector and deformation feature vector are input into the OSTrack backbone network. The OSTrack backbone network outputs diffraction mode attention and deformation mode attention. The diffraction mode attention and deformation mode attention are cross-modally processed by the attention fusion network to obtain the tracking result.
[0020] As a preferred embodiment, the diffraction mode features are initialized with a Green's function to obtain the diffraction feature vector, and the deformation mode features are parameterized with a thin plate spline to obtain the deformation feature vector.
[0021] As a preferred embodiment, the parameter self-updating system is used to simultaneously calculate the geometric deformation measurement and acoustic diffraction measurement values of the physical features of the input target bounding box, filter the samples based on the geometric deformation measurement to construct a deformation sample pool, and filter the samples based on the acoustic diffraction measurement values to construct a diffraction sample pool.
[0022] The LoRA layer learns the diffraction sample pool and the deformation sample pool to obtain the fine-tuned LoRA parameters for the diffraction and deformation modes:
[0023] In the first stage, all parameters of the OSTrack backbone network except for the LoRA layer corresponding to the diffraction mode are frozen, and the LoRA parameters corresponding to the diffraction mode are trained only using the diffraction sample pool.
[0024] In the second stage, all parameters of the OSTrack backbone network except for the LoRA layers corresponding to the diffraction mode and the deformation mode are frozen, and the LoRA parameters corresponding to the diffraction mode and the deformation mode are trained simultaneously using the deformation sample pool and the diffraction sample pool.
[0025] In the third stage, the trained LoRA parameters are quantized and compressed, and then updated online to the LoRA layers corresponding to the diffraction and deformation modes in the OSTrack backbone network.
[0026] Preferably, speckle contrast gating is used in the first and second stages to achieve noise-adaptive sparse parameter updates.
[0027] Preferably, the target detector, parameter self-updating system, and target tracker are all deployed within an embedded processor, wherein:
[0028] The main computing unit simultaneously calculates the geometric deformation and acoustic diffraction measurements of the physical features of the input target bounding box. A deformation sample pool is constructed based on the geometric deformation measurement, and a diffraction sample pool is constructed based on the acoustic diffraction measurement.
[0029] The physical features of the input target bounding box are processed by a DSP coprocessor to obtain diffraction mode features, and the diffraction feature vector is obtained after processing.
[0030] The remaining functions are performed by the embedded processor.
[0031] The beneficial effects of this application are: (1) This application replaces full-size fine-tuning by adopting a progressive low-rank decomposition learning paradigm architecture that introduces dual-modal coupling, which requires training only a small number of low-rank matrix parameters, greatly reducing memory usage and computation, enabling it to run efficiently on low-computing embedded devices, and enabling the model to have both the ability to model the a priori physical laws of sound wave propagation and the ability to characterize the non-rigid changes of the target.
[0032] (2) This application matches the physical evolution law of the sonar target through a phased optimization strategy (diffraction first, deformation later), realizes noise adaptive sparse parameter update through pattern contrast gating, and effectively solves the feature confusion problem between diffraction artifacts and real deformation through the modal cross attention (MCA) module, so that the model can dynamically adapt to the complex underwater acoustic environment.
[0033] (3) This application constructs a complete "perception-learning-deployment" adaptive closed-loop system. The asynchronous fine-tuning process is isolated from the main tracking task, effectively reducing the peak resource consumption and realizing online tracking and learning to a certain extent. Attached Figure Description
[0034] Figure 1 A schematic diagram illustrating the principle of a sonar target tracking system;
[0035] Figure 2 This is a schematic diagram illustrating the hardware deployment principle of this application. Detailed Implementation
[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0037] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.
[0038] The present application will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the application.
[0039] The sonar target tracking method of this embodiment includes:
[0040] Construct a sonar target tracking system, including a parameter self-updating system and a real-time tracking system. The real-time tracking system includes a target detector and a target tracker.
[0041] The target detector and target tracker are trained using existing sonar image data, and a LoRA layer is inserted into the target tracker.
[0042] During target tracking, the sonar image is input to the target detector, and the physical features of the target bounding box obtained by the target detector are simultaneously input to the parameter self-updating system and the target tracker. The parameter self-updating system constructs a sample pool based on the input, and the LoRA layer learns the sample pool to obtain the fine-tuned LoRA parameters, which are then updated to the corresponding LoRA layer of the target tracker.
[0043] The updated target tracker outputs the tracking results.
[0044] This implementation inserts a LoRA layer into the target tracker mounted on the underwater vehicle. During learning, the original model parameters are frozen, and only sonar target data collected by a single-stage real-time detector is used to perform progressive fine-tuning guided by physical laws. The fine-tuned LoRA parameters are then updated to the corresponding LoRA layer of the target tracker. This solves the tracking drift problem caused by insufficient multipath interference suppression, poor target deformation adaptability, and insufficient computing power of embedded devices in traditional sonar tracking algorithms.
[0045] Furthermore, the target tracker includes a feature extraction network, an OSTrack backbone network, and an attention fusion network; a LoRA layer is inserted into the OSTrack backbone network; specifically, the LoRA layer includes a diffraction kernel and a deformation substrate.
[0046] The target detector is used to detect the physical features of the bounding box of a target in a sonar image;
[0047] A feature extraction network processes the physical features of the target bounding box to obtain diffraction mode features and deformation mode features, resulting in diffraction feature vectors and deformation feature vectors. Further, the diffraction mode features are initialized using a Green's function to obtain the diffraction feature vector, and the deformation mode features are parameterized using a thin-plate spline to obtain the deformation feature vector. The diffraction mode features are initialized using a Green's function and infused with acoustic propagation priors, allowing the system to calculate the target's diffraction mode features. These features primarily describe changes in the target's appearance caused by acoustic phenomena such as underwater multipath effects. The deformation mode features are parameterized using a thin-plate spline and infused with geometrically non-rigid deformation priors, allowing the system to calculate the target's deformation mode features. These features primarily describe changes in the target's morphology (such as swimming and deformation).
[0048] The diffraction feature vector and deformation feature vector are input into the OSTrack backbone network. The OSTrack backbone network outputs diffraction mode attention and deformation mode attention. The attention fusion network is used to perform modal cross-interaction of diffraction mode attention and deformation mode attention, and to perform deep fusion and information interaction. This helps the model effectively distinguish between real deformation and acoustic artifacts, thereby extracting more robust target features and finally outputting the tracking result of the current frame.
[0049] During the deployment phase, the device is deployed on an underwater vehicle. The hardware architecture primarily includes an embedded processor, a main computing unit, and a DSP coprocessor. A parameter self-updating system with a heterogeneous computing architecture is constructed to initially adapt to the sonar image domain, and then deployed in conjunction with it within the embedded processor. Both target detection and target tracker are deployed within the embedded processor. In this implementation, the embedded processor is implemented using an embedded GPU, responsible for single-stage target detection feature extraction, OSTrack backbone network inference, and the calculation of diffraction mode Green's function and modal cross-attention fusion. The main computing unit simultaneously calculates geometric deformation and acoustic diffraction measurements on the input target bounding box physical features. A deformation sample pool is constructed based on the geometric deformation measurement, and a diffraction sample pool is constructed based on the acoustic diffraction measurement. The DSP coprocessor processes the input target bounding box physical features to obtain diffraction mode features, resulting in a diffraction feature vector. The remaining functions in the tracking process are completed by the embedded processor.
[0050] The front end of the perception system is equipped with a high-resolution forward-looking sonar device, whose imaging data, after preprocessing, is input to the target detector and target tracker. The computational resource allocation and task scheduling in this embodiment are optimized to ensure efficient parallel computing with limited power consumption. During the training phase, the target detector and target tracker are trained using existing sonar image data.
[0051] This deep learning-based single-stage target detection framework utilizes an labeled sonar dataset for end-to-end training of the target detector and target tracker, enabling them to locate and classify underwater targets. The training process employs data augmentation strategies such as synthetic multipath interference and elastic mesh deformation. A diffraction consistency constraint is added to the loss function to improve the model's robustness to degradation factors such as noise and blurring in sonar images. After training, the model is converted to a lightweight format for embedded deployment environments.
[0052] A bimodal adaptation layer is inserted into the ViT backbone network of OSTrack. Depending on the actual situation, the diffraction branch inserts a projection layer after multi-head attention after constrained weight initialization, the deformation branch inserts an FFN layer after determining the parameterization method, and the modal cross-attention is inserted before layer normalization.
[0053] During target tracking, sonar images are input to a target detector. The physical features of the target bounding box obtained by the target detector are simultaneously input to a parameter self-updating system and a target tracker. The parameter self-updating system constructs a sample pool based on the input, and the LoRA layer learns from the sample pool to obtain fine-tuned LoRA parameters, which are then updated in the LoRA layer of the OSTrack backbone network. The updated target tracker outputs the tracking results. Specifically, this includes:
[0054] Sample pool construction includes a diffraction sample pool and a deformation sample pool. After normalization and resizing of the original sonar image, the diffraction feature matrix and deformation feature vector need to be calculated and input into the target detector to analyze the physical features of the target bounding box. The input target bounding box physical features are simultaneously subjected to geometric deformation and acoustic diffraction measurements (speckle contrast, contour curvature change, etc.). A deformation sample pool is constructed based on the geometric deformation measurement, and a diffraction sample pool is constructed based on the acoustic diffraction measurement.
[0055] Accumulate samples and prepare to initiate the progressive parameter fine-tuning process, which specifically involves: the LoRA layer learning the diffraction sample pool and the deformation sample pool to obtain the fine-tuned LoRA parameters for the diffraction and deformation modes.
[0056] In the first stage, all parameters of the OSTrack backbone network except for the LoRA layer corresponding to the diffraction mode are frozen, and the LoRA parameters corresponding to the diffraction mode are trained only using the diffraction sample pool; only the LoRA parameters that process diffraction features are preferentially trained to adapt to the current acoustic environment.
[0057] In the second stage, all parameters of the OSTrack backbone network except for the LoRA layers corresponding to the diffraction and deformation modes are frozen. The LoRA parameters corresponding to the diffraction and deformation modes are trained simultaneously using the deformation sample pool and the diffraction sample pool. After the diffraction mode is stabilized, joint optimization is performed, and the LoRA parameters of the two modes are fine-tuned to learn the intrinsic relationship between the two features.
[0058] In the third stage, the trained LoRA parameters are quantized and compressed, and then updated online to the LoRA layers corresponding to the diffraction and deformation modes in the OSTrack backbone network.
[0059] Specifically, in the first and second stages, speckle contrast gating is used to achieve noise-adaptive sparse parameter updates.
[0060] This implementation sets up a double-ring buffer mechanism for diffraction, deformation and other sequences, as well as a sample elimination mechanism based on physical similarity.
[0061] Implement a progressive fine-tuning strategy: After environmental detection of a sonar image, if the sound speed changes abruptly by 0.5 m / s, emergency diffraction fine-tuning is initiated, and two rounds of training are started; if the curvature change reaches 15%, the LoRA parameters of the deformation mode are updated, and three rounds of training are started; joint fine-tuning is only performed in a stable state, and a complete set of five rounds of training is started before subsequent quantization deployment.
[0062] Model hot update mechanism: After fine-tuning, the low-rank parameters are converted to a new format and then seamlessly replace the running tracking model components using memory mapping technology, achieving zero-downtime updates. The updated model is immediately applied to the current tracking scenario, and its performance improvement is monitored in real time through online evaluation metrics.
[0063] Closed-loop adaptive system: It continuously collects new samples and triggers the next round of fine-tuning, forming a closed-loop process of "detection-tracking-learning-optimization". By setting an adaptive trigger threshold, computational overhead and performance gain can be balanced to achieve long-term stable target tracking.
[0064] After fine-tuning, a physical sensing quantization strategy is deployed into the tracking process to adapt to the target domain perceived by the current underwater vehicle.
[0065] The solution provided in this application enables the sonar target tracking algorithm mounted on an underwater vehicle to adapt to the target appearance domain in the current environment. By using a LoRA-based parameter insertion and fine-tuning paradigm, it is possible to learn the appearance of sonar targets with a small number of parameters, thereby adaptively achieving accurate discrimination of the appearance features of sonar targets.
[0066] While this application has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other embodiments.
Claims
1. A sonar target tracking method, characterized in that, include: A target detector and a target tracker are trained using existing sonar image data, and a LoRA layer is inserted into the target tracker; During target tracking, the sonar image is input to the target detector, and the target bounding box physical features obtained by the target detector are simultaneously input to the parameter self-updating system and the target tracker. The parameter self-updating system constructs a sample pool based on the input, and the LoRA layer learns the sample pool to obtain the fine-tuned LoRA parameters, which are then updated to the corresponding LoRA layer of the target tracker. The updated target tracker outputs the tracking results.
2. The sonar target tracking method according to claim 1, characterized in that, The LoRA layer comprises a diffraction nucleus and a deformable substrate.
3. The sonar target tracking method according to claim 2, characterized in that, The target tracker consists of a feature extraction network, an OSTrack backbone network, and an attention fusion network; the OSTrack backbone network is inserted with a LoRA layer. The target detector is used to detect the physical features of the bounding box of a target in a sonar image; The feature extraction network is used to process the physical features of the target bounding box to obtain diffraction mode features and deformation mode features, and after processing, diffraction feature vectors and deformation feature vectors are obtained. The diffraction feature vector and deformation feature vector are input into the OSTrack backbone network. The OSTrack backbone network outputs diffraction mode attention and deformation mode attention. The diffraction mode attention and deformation mode attention are cross-modally processed by the attention fusion network to obtain the tracking result.
4. The sonar target tracking method according to claim 1, characterized in that, The diffraction mode features are initialized with Green's function to obtain the diffraction feature vector, and the deformation mode features are parameterized with thin plate splines to obtain the deformation feature vector.
5. The sonar target tracking method according to claim 1, characterized in that, The parameter self-updating system is used to simultaneously calculate the geometric deformation and acoustic diffraction measurements of the physical features of the input target bounding box. It filters the samples based on the geometric deformation measurements to build a deformation sample pool, and filters the samples based on the acoustic diffraction measurements to build a diffraction sample pool. The LoRA layer learns the diffraction sample pool and the deformation sample pool to obtain the fine-tuned LoRA parameters of the diffraction mode and the deformation mode: In the first stage, all parameters of the OSTrack backbone network except for the LoRA layer corresponding to the diffraction mode are frozen, and the LoRA parameters corresponding to the diffraction mode are trained only using the diffraction sample pool. In the second stage, all parameters of the OSTrack backbone network except for the LoRA layers corresponding to the diffraction mode and the deformation mode are frozen, and the LoRA parameters corresponding to the diffraction mode and the deformation mode are trained simultaneously using the deformation sample pool and the diffraction sample pool. In the third stage, the trained LoRA parameters are quantized and compressed, and then updated online to the LoRA layers corresponding to the diffraction and deformation modes in the OSTrack backbone network.
6. The sonar target tracking method according to claim 1, characterized in that, In the first and second stages, speckle contrast gating is used to achieve noise-adaptive sparse parameter updates.
7. The sonar target tracking method according to claim 1, characterized in that, The target detector, parameter self-updating system, and target tracker are all deployed within the embedded processor, wherein: The main computing unit simultaneously calculates the geometric deformation and acoustic diffraction measurements of the physical features of the input target bounding box. A deformation sample pool is constructed based on the geometric deformation measurement, and a diffraction sample pool is constructed based on the acoustic diffraction measurement. The physical features of the input target bounding box are processed by a DSP coprocessor to obtain diffraction mode features, and the diffraction feature vector is obtained after processing. The remaining functions are performed by the embedded processor.
8. A computer-readable storage device storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the sonar target tracking method as described in any one of claims 1 to 7.
9. A sonar target tracking device, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the sonar target tracking method as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the sonar target tracking method as described in any one of claims 1 to 7.