High-precision signal time difference direction-finding positioning method and system combined with double-path visual pod space positioning
By combining the space positioning of the dual-channel visual pod and differential GPS, the YOLOv8 visual recognition model and three-station time difference method are used to solve the problem of size limitation and jitter error in low-frequency signal detection in small and medium-sized drones, achieving high accuracy The time difference directional positioning of low-frequency band signals.
Patent Information
- Application Number
- CN202411497100.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-10-25
AI Technical Summary
Small and medium-sized drones face size limitations and dragged antenna position jitter errors in low-frequency electronic signals detection, resulting in low signal positioning accuracy and making it difficult to achieve high-precision low-frequency signal time difference directional positioning.
Using a high-precision signal time difference direction measurement positioning method combining spatial positioning of dual-channel visual pods, the real-time image of drag pods is detected by combining differential GPS and dual-channel vision, using the YOLOv8 visual recognition model and CMOS sensor, combined with the three-station time difference method Direction finding positioning algorithm reduces jitter error and improves signal positioning accuracy.
It realizes high-precision positioning of low-frequency electronic signals on small and medium-sized drone platforms, reduces jitter errors during the operation of the towed pod, and improves the position determination accuracy of ground radiation source signals.
Smart Images

Figure CN119024268A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of UAV visual positioning technology, and in particular to a high-precision signal time difference direction finding and positioning method and system that combines dual-path visual pod spatial positioning. Background Technology
[0002] Due to the numerous and complex signal frequency bands, manned / unmanned reconnaissance aircraft need to pay additional weight and size costs to achieve high-precision wideband signal reconnaissance, which severely restricts the high maneuverability advantage of airborne platforms. Therefore, how to achieve wideband signal reconnaissance as much as possible on a single small and medium-sized unmanned aerial vehicle platform is of great significance for improving signal reception efficiency.
[0003] Due to the characteristics of electromagnetic signals, lower-frequency electromagnetic waves experience less attenuation during propagation in the atmosphere than higher-frequency electromagnetic waves. Low-frequency electromagnetic waves are widely used in communication and emergency rescue applications. However, the longer wavelength of low-frequency electromagnetic waves compared to high-frequency waves makes direction finding and positioning more difficult. This longer wavelength constraint limits the size of airborne drones and unmanned aerial vehicles (UAVs). For example, the typical US Global Hawk UAV has a wingspan of 35 meters, meeting the requirements for carrying low-frequency electromagnetic waves, but its unit price exceeds several hundred million US dollars, hindering large-scale application. Meanwhile, small and medium-sized platforms face the problem of being too small to operate at low frequencies for single-unit reconnaissance. Currently, towed antennas, unrestricted by the internal space of aircraft, can achieve larger electrical dimensions, enabling them to receive and transmit low-frequency electromagnetic waves, thus effectively solving this problem.
[0004] Target localization technology based on direction finding information is an important research direction in the field of passive positioning and one of the most common positioning methods. When the target and the direction finding station are on the same plane (i.e., considering two-dimensional spatial positioning), the azimuth information obtained by the direction finding station is used to estimate the two-dimensional position coordinates of the target. However, when the target and the direction finding station are not on the same plane (i.e., considering three-dimensional spatial positioning), the azimuth and elevation information obtained by the direction finding station are used simultaneously to estimate the three-dimensional position coordinates of the target.
[0005] Although there have been many research results on towed antennas for UAVs and multi-band direction finding and positioning technology, the research on combining the two and applying them to small and medium-sized UAV platforms to detect low-frequency electronic signals is still in the preliminary research stage. Moreover, the towed antenna is dragged by the traction steel cable and sways in the airflow. As a signal receiver, its position jitter error will have a significant impact on the detection results, making it difficult to obtain the accurate position of the towed pod, which will ultimately cause a large deviation in the determination of the location of the ground radiation source signal. Summary of the Invention
[0006] To address the shortcomings of the existing technologies, this invention aims to propose a high-precision signal time-difference direction finding and positioning method and system that combines dual-path visual pod spatial positioning. The method is designed for small and medium-sized aircraft to detect low-frequency electronic signals, fully considering the impact of aircraft wake turbulence on towed pods. It constructs a high-precision positioning method that integrates differential GPS and dual-path visual positioning for towed pods, further reducing jitter errors during towed pod operation. An improved three-station time-difference direction finding and positioning technique is used to obtain the precise location of ground radiation sources, thereby solving the problems existing in the background technology.
[0007] The above-mentioned technical objective of the present invention is achieved through the following technical solution: The first aspect of the technical solution disclosed in this invention is: a high-precision signal time difference direction finding and positioning method combining dual-path visual pod spatial positioning, comprising the following steps: Step 1: Construct a visual recognition model; Step 2: Deploy the visual recognition model on the edge AI computing platform, build a dual-path visual spatial positioning system, and use the dual-path visual spatial positioning system to detect real-time images of the towed pod; Step 3: Transmit the detected real-time images to the visual recognition model to obtain the spatial position information of the towed pod; Step 4: Fuse the differential GPS on the aircraft with the spatial position information of the towed pod, and determine the location of the radiation source using the three-station time difference method orientation and positioning algorithm.
[0008] As a further preferred embodiment of the above technical solution, step 1 involves constructing a visual recognition model using the YOLOv8 visual recognition network, and the steps include: Step 101: Data Acquisition; Two CMOS sensors on the simulated aircraft acquire images of the towed pod from the perspective of the aircraft, resulting in a dual-channel image dataset. Step 102: Data labeling; The acquired dual-channel image dataset was labeled using Labelimg, and the labeled data was divided into training set, validation set and test set; Step 103: Optimize the YOLOv8 visual recognition network using the training set, validation set, and test set to obtain the visual recognition model.
[0009] A further preferred method of the above scheme is as follows: Step 103, optimizing the YOLOv8 visual recognition network, includes: The YOLOv8 visual recognition network is trained using the training set within its embedded training mode. The trained YOLOv8 visual recognition network is then validated using the validation set, and finally tested using the test set to obtain the visual recognition model.
[0010] As a further preferred embodiment of the above technical solution, step 2, which involves establishing a dual-path visual spatial positioning system and using this system to detect real-time images of the towed pod, specifically includes: Step 201: After converting the visual recognition model into RKNN format, deploy it on an edge AI computing platform; Step 202: Install CMOS sensors on the two wingtips and two wing roots of the aircraft, and connect the CMOS sensors to the edge AI computing platform to form a dual-path visual spatial positioning system for dynamic detection of towed pods; Step 203: Acquire real-time images of the towed pod using a CMOS sensor and transmit them to the edge AI computing platform.
[0011] Based on the above technical solution, in step 203, the MEMS sensor is embedded in the CMOS sensor, and the MEMS sensor is used to perform dynamic jitter elimination on the real-time image acquired by the CMOS sensor, and to stabilize the real-time image.
[0012] As a further preferred embodiment of the above technical solution, step 3, which involves obtaining the spatial location information of the towed pod, includes: Step 301: Use a visual recognition model to identify the position information of the towed pod in the real-time image; Step 302: Transmit the position information of the towed pod to the back-end processing module of the edge AI computing platform to calculate the spatial position of the towed pod.
[0013] As a further preferred embodiment of the above technical solution, step 4, the step of determining the location of the radiation source, includes: Step 401: The three-station time difference method direction finding and positioning algorithm measures the time difference between the arrival of the radiation source at the main receiving station and the two auxiliary receiving stations. The main receiving station is a differential GPS antenna mounted on the aircraft fuselage, and the auxiliary receiving stations are antennas mounted on a towed pod. Step 402: The location of the radiation source is determined using the time difference between the main receiving station and the two auxiliary receiving stations. The solution process is as follows: (1) Calculate the distance difference between the radiation source and the main receiving station and the two auxiliary receiving stations. The expression is as follows: ①; In the formula, The speed of electromagnetic wave propagation. The time difference measurement between stations. , These represent the distance differences between the radiation source and the auxiliary receiving station, and the distance difference between the radiation source and the main receiving station, respectively, with the coordinates of the radiation source being... The two-dimensional coordinates of the main receiving station and the auxiliary receiving station are: ;in The main station is represented by one of the main stations, and the others by auxiliary receiving stations. 、 Main receiving station 、 Direction coordinates; (2) By linearizing formula ①, we obtain a system of linear equations for the location of the radiation source, the expression of which is as follows: ②; ③; ④; In the formula, let , , These are the absolute distances of the auxiliary receiving station and the main receiving station relative to the origin, respectively. , They are the main receiving station and the auxiliary receiving station. , The relative distance difference in direction; (3) Solve the linear equations of the radiation source to obtain the position coordinates of the radiation source.
[0014] The second aspect of the technical solution disclosed in this invention is: a high-precision signal time difference direction finding and positioning system combining dual-channel visual pod spatial positioning, wherein the system is implemented based on a high-precision signal time difference direction finding and positioning method combining dual-channel visual pod spatial positioning, and the system includes: The system construction module deploys the constructed visual recognition model on the edge AI computing platform and builds a dual-path visual spatial positioning system detection module to detect real-time images of the towed pod using the dual-path visual spatial positioning system. The recognition module identifies real-time images of the towed pod and obtains the relative position information of the towed pod in the image; The backend processing module uses relative position information to calculate the spatial position of the towed pod. The calculation module is used to fuse the differential GPS on the aircraft with the spatial position information of the towed pod to calculate the location of the radiation source.
[0015] As a further preferred embodiment of the above solution, the detection module includes: The real-time acquisition submodule is used to acquire real-time images of the towed pod. The real-time de-jitter submodule is used to stabilize the acquired real-time images.
[0016] As a further preferred embodiment of the above solution, the computing module includes: The time difference measurement submodule is used to measure the time difference between the arrival of the radiation source at the main receiving station and the two auxiliary receiving stations; The location calculation submodule uses the time difference to obtain the distance difference between the radiation source and the main receiving station and the two auxiliary receiving stations, and then calculates the location coordinates of the radiation source.
[0017] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: 1. By installing two CMOS sensors on each of the two wings of the aircraft, dynamic real-time image detection of the towed pod at the tail of the wing is performed, and the real-time image is transmitted to a visual recognition model based on the YOLOv8 network framework for recognition. The relative position information of the pod in the image is identified, and the spatial position information of the towed pod is calculated.
[0018] 2. In order to minimize the non-periodic oscillation of the wing caused by the towed pod, a MEMS sensor is embedded in the CMOS sensor to realize active anti-shake processing of the directly acquired images, and the real-time acquired images are stabilized to improve the stability of the acquired images and effectively improve the accuracy of obtaining the position information of the towed pod.
[0019] 3. This invention integrates differential GPS with a dual-path visual spatial positioning system to simultaneously obtain the spatial position information of the aircraft itself and the towed pod, and calculates the accurate position of the radiation source signal using a three-station time difference method orientation positioning algorithm. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the towed pod configuration of the present invention; Figure 2 This is a performance comparison chart of YOLOv8 and multiple versions of YOLO based on the present invention; Figure 3 This invention relates to an image acquisition system constructed in a laboratory. Figure 4 To construct the directory structure for datasets of different categories; Figure 5 The confusion matrix generated from the training results of YOLOv8n and YOLOv8s in this invention, where a is the confusion matrix of YOLOv8n and b is the confusion matrix of YOLOv8s. Figure 6The normalized confusion matrix generated for the training results of YOLOv8n and YOLOv8s in this invention, where a is the normalized confusion matrix of YOLOv8n and b is the normalized confusion matrix of YOLOv8s. Figure 7 This is a confidence curve of the evaluation index of the present invention, where a is the confidence curve of the evaluation index of YOLOv8n and b is the confidence curve of the evaluation index of YOLOv8s. Figure 8 This is a precision curve diagram of the present invention, where a is the precision curve of YOLOv8n and b is the precision curve of YOLOv8s. Figure 9 This is a recall curve diagram of the present invention, where a is the recall curve of YOLOv8n and b is the recall curve of YOLOv8s. Figure 10 This is a precision-recall curve of the present invention, where a is the precision-recall curve of YOLOv8n and b is the precision-recall curve of YOLOv8s. Figure 11 Label images, where a is the label image for YOLOv8n and b is the label image for YOLOv8s; Figure 12 Here are the label correlation graphs, where a is the label correlation graph for YOLOv8n and b is the label correlation graph for YOLOv8s. Figure 13 The training results are shown, where a is the training result of YOLOv8n and b is the training result of YOLOv8s. Figure 14 The results of image recognition on the test set of this invention; Figure 15 The development board OrangePi5 is used to deploy the model of this invention; Figure 16 This is the model conversion process of the present invention; Figure 17 This invention compares the target detection results based on PyTorch, ONNX, and RKNN format models. Figure 18 This is a diagram of the dual-path visual spatial positioning structure of the present invention; Figure 19 Output image for visual recognition; Figure 20 The results of the dual-channel visual recognition of the present invention are shown in Figure 1, where a is the recognition result of the left CMOS sensor and b is the recognition result of the right CMOS sensor. Figure 21 To reconstruct the spatial trajectory of the towed pod using the model of this invention; Figure 22The fluid domain mesh diagram (a) and the solid domain mesh diagram (b) of this invention are shown. Figure 23 This is a simulation diagram of the transient position change of the towed antenna pod obtained by the present invention. Figure 24 The displacement characteristics of the towed pod in the direction of incoming flow (a), vertical direction (b), and spanwise direction (c) are shown. Figure 25 This invention relates to a CMOS+MEMS data acquisition system for eliminating jitter. Figure 26 The data provided are MEMS sensor data of the present invention, wherein a is pitch axis gyroscope data and b is roll axis gyroscope data; Figure 27 The original image optical flow method two-axis jitter estimation value of the present invention, where a is the optical flow method pitch axis jitter estimation value and b is the optical flow method roll axis jitter estimation value; Figure 28 The image stabilization results are the two-axis jitter estimates using the optical flow method after image stabilization according to the present invention, where a is the pitch axis jitter estimate using the optical flow method and b is the roll axis jitter estimate using the optical flow method. Figure 29 This is a comparison diagram of the displacement trajectory before and after image stabilization according to the present invention, where a is before image stabilization and b is after image stabilization; Figure 30 These are comparison images of spatial displacement trajectories, where a is a comparison image of spatial displacement trajectories before image stabilization, and b is a comparison image of spatial displacement trajectories after image stabilization. Figure 31 This is a schematic diagram of the differential GPS principle. Figure 32 This is a schematic diagram illustrating the principle of the three-station time difference method for orientation and positioning of the present invention. Figure 33 The impact of introducing jitter error (a) and eliminating jitter error (b) on the lateral accuracy of the target in the spanwise positioning of the towed pod; Figure 34 The impact of introducing jitter error (a) and eliminating jitter error (b) on the lateral accuracy of the target in the axial positioning of the towed pod; Figure 35 The impact of introducing jitter error (a) and eliminating jitter error (b) on the lateral accuracy of the target in the oblique positioning of the towed pod; Figure 36 The impact of errors (a) and (b) on the lateral accuracy of the target before correction of the spanwise positioning position of the towed pod; Figure 37 The impact of the errors (a) and (b) before and after the correction of the axial positioning position of the towed pod on the lateral accuracy of the target; Figure 38The impact of the error before (a) and after (b) correction of the oblique positioning position of the towed pod on the lateral accuracy of the target. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.
[0022] Reference Figures 1-38 This invention provides a high-precision signal time difference direction finding and positioning method combining dual-path visual pod spatial positioning, comprising the following steps: Step 1: Construct a visual recognition model; Step 2: Deploy the visual recognition model on the edge AI computing platform, build a dual-path visual spatial positioning system, and use the dual-path visual spatial positioning system to detect real-time images of the towed pod; Step 3: Transmit the detected real-time images to the visual recognition model to obtain the spatial position information of the towed pod; Step 4: Fuse the differential GPS on the aircraft with the spatial position information of the towed pod, and determine the location of the radiation source using the three-station time difference method orientation and positioning algorithm.
[0023] Specifically, this invention is based on the principles of deep learning-based object detection technology and uses the newly released YOLOv8 visual recognition network framework to construct a visual recognition model. YOLOv8 is an improved version of YOLO, further enhancing its flexibility and accuracy. Its speed, accuracy, and ease of use make it an excellent choice for completing a wide range of object detection and tracking, instance segmentation, image classification, and pose estimation tasks, such as... Figure 2 By comparing the performance of YOLOv8 with multiple versions of YOLO, it can be seen that YOLOv8 has significantly reduced the number of parameters while maintaining the same performance compared to previous YOLO frameworks, and the inference speed of the model has also been greatly improved with the same number of parameters.
[0024] YOLOv8's backbone network is similar to YOLOv5, with modifications to the CSP layer called the C2f module. The C2f module (with a cross-stage bottleneck of two convolutions) combines high-level features and contextual information to improve detection accuracy. YOLOv8 uses an anchorless model with decoupled heads to independently handle objectivity, classification, and regression tasks, allowing each branch to focus on its task design and improving overall model accuracy. YOLOv8's output layer uses the sigmoid function as the activation function for the object score, representing the probability that the bounding box contains an object. It uses the softmax function for class probabilities, representing the probability of an object belonging to each possible class. YOLOv8 uses CIoU and DFL loss functions for bounding box loss and binary cross-entropy for classification loss. Furthermore, YOLOv8 provides a semantic segmentation model, YOLOv8-Seg, with a CSPDarknet53 feature extractor as its backbone. This model achieves state-of-the-art results on various object detection and semantic segmentation benchmarks while maintaining high efficiency.
[0025] Building a visual recognition model mainly involves three steps: data acquisition, data annotation, and model optimization. (1) Data collection: like Figure 3 As shown, a dual-channel image acquisition simulator was built in the laboratory to simulate the image acquisition of the towed pod from the perspective of the CMOS sensor on the aircraft. Since two CMOS sensors are required on each side of the aircraft to identify and track the towed pod from two directions in actual flight, a dual-channel image acquisition simulator with two angles was built on the ground. The simulator consists of two CMOS image sensors, one on the left and one on the right. Each image sensor acquires 300 images from different angles to provide input data for the next step of data annotation.
[0026] (2) Data labeling: Since the YOLOv8 visual recognition network framework used is a supervised learning model, the collected data needs to be labeled. Data labeling helps deep learning algorithms understand the meaning of the data, such as the position, color, shape, and size of objects. This invention uses Labelimg to label 600 collected images and divides the labeled data into training, validation, and test sets in a 7:2:1 ratio. Figure 4 The directory structure shown is used to construct the dataset.
[0027] (3) Model optimization: Optimization of the visual recognition model includes model training, model validation, and model testing. The YOLOv8 visual recognition network framework used in this invention has an embedded model training mode. After the dataset is constructed, training can be started with one click in CLI mode. Table 1 shows a comparison of the five basic training models provided by YOLOv8. Based on the performance of the training platform and deployment platform used in this application, this invention trains two models: YOLOv8n and YOLOv8s.
[0028] ; like Figure 5 a, Figure 5 As shown in Figure b, the confusion matrix is a tabular tool for evaluating the performance of a classification model. It is a statistical summary of sample results based on the model's predicted classes and the actual classes (true values). When dealing with binary classification problems, the confusion matrix is typically 2x2, encompassing four basic elements: TruePositive (TP), TrueNegative (TN), FalsePositive (FP), and FalseNegative (FN). The normalized confusion matrix obtained from the training results of YOLOv8n and YOLOv8s in this invention (e.g.) Figure 6 As shown in a and b), by dividing the value of each element by the actual number of samples in its corresponding category, the values in the matrix are converted into a percentage reflecting the classification accuracy. This allows for a direct comparison of the classification accuracy between different categories, clarifying the merits of different models.
[0029] like Figure 7 a, Figure 7 b, Figure 8 a, Figure 8 b, Figure 9 a, Figure 9 As shown in Figure b, during the training of both YOLOv8n and YOLOv8s models, the F1 curve depicts the changes in F1 scores under different classification thresholds. The horizontal axis represents the confidence level, and the vertical axis represents the F1 score. We can see that the confidence level of the YOLOv8n model is 0.822, while the confidence level of the YOLOv8s model is 0.791. The accuracy curve shows the trend of accuracy increasing with confidence level. Figure 8 a, Figure 8Figure b shows that the detection precision of both models increases with increasing confidence; the recall curve represents the relationship between recall and confidence, showing that recall decreases as confidence increases; the PR curve shows the relationship between precision and recall at different classification thresholds. The closer the PR curve is to the upper right corner of the coordinate system, the better the model's performance, the more accurately it can identify positive samples, and the higher its precision. The closer it is to the left, the weaker the model's ability to identify positive samples and the lower its recall. An important feature of the PR curve is that precision and recall change accordingly as the classification threshold is adjusted.
[0030] Generally, a model that can maintain both high precision and high recall will have a high PR curve. Conversely, if a model prioritizes either high precision or high recall, the curve will shift towards lower precision or lower recall. The PR curve helps evaluate a model's performance at different thresholds and aids in selecting appropriate thresholds to balance precision and recall. The area under the PR curve (i.e., average precision, AP) provides a quantitative evaluation of the model; a higher AP value indicates better model performance. Figure 10 a, Figure 10 As shown in Figure b, the YOLOv8n and YOLOv8s models can exhibit good performance while maintaining high precision and recall.
[0031] Figure 11 a, Figure 11 The label image in b represents the category and bounding box information of each detected target, which mainly includes four types of information: target category, target location, target size, and target rotation angle. Figure 12 a, Figure 12 The 'b' plot is used to display the correlation between the predicted labels of YOLOv8n and YOLOv8s during training. The label correlation plot is a color matrix that shows the relationships between the labels in the training set. It can reveal the behavior and performance of the object detection algorithm during the training phase, as well as the interaction between different predicted labels. By analyzing the label correlation plot, we can gain a deeper understanding of the strength of the algorithm's ability to distinguish between different categories, as well as its accuracy in predicting different categories. In addition, comparing the label correlation plots of different algorithms or datasets helps to evaluate the algorithm performance and dataset quality.
[0032] like Figure 13 a, Figure 13As shown in Figure b, after the YOLOv8n and YOLOv8s models were trained, the three columns on the left showed multiple loss values during the training and validation process, all of which remained below 1, meeting the performance requirements. The two columns on the right showed several key performance indicators, including accuracy (measures the proportion of samples correctly predicted by the model; in object detection, if the predicted bounding box matches the ground truth bounding box, the prediction is considered accurate), recall (measures the model's ability to identify all real positive samples; in object detection, if the ground truth bounding box matches the predicted bounding box, the sample is considered successfully recalled), mAP50 (mean precision, the average precision calculated across multiple classes; mAP50 refers to the average precision at a 50% Intersection over Union (IoU) threshold), and mAP50-95 (the average precision calculated within the 50% to 95% IoU threshold range; by averaging the mAP values within this range, the model's performance at different IoU thresholds can be evaluated more accurately). Intersection over Union (IoU) is a metric that evaluates the degree of overlap between predicted and ground truth bounding boxes. A higher IoU value indicates a higher similarity between the two bounding boxes. Generally, an IoU greater than 0.5 is considered a detected target. By comparing performance metrics, it can be seen that the recognition performance difference between the YOLOv8n and YOLOv8s models is not significant on the dataset used.
[0033] After training the YOLOv8n and YOLOv8s models, the trained visual recognition models were validated using a validation set, and then tested using a test set. The image recognition results on the test set are as follows. Figure 14 As shown, the visual recognition model of the present invention can effectively recognize all images in the test set.
[0034] After the visual recognition model is optimized, deployment needs to be considered. This invention deploys the visual recognition model on the domestic development board OrangePi5 in the form of edge AI, such as... Figure 15 As shown, the Rockchip RK3588 computing chip integrated in the OrangePi5 is used to achieve real-time inference of video data and real-time output of inference results. The inference function of the RK3588 chip mainly relies on its built-in high-performance NPU module, whose peak computing performance reaches 6 TOPS. The model obtained by optimizing the YOLOv8 network framework is based on the PyTorch framework. The RK3588 chip's NPU module only supports the RKNN format, so the PyTorch format needs to be converted to the RKNN format. The specific conversion path is as follows: Figure 16 As shown, the PyTorch format is first converted to ONNX format, and then the ONNX format model is converted to an RKNN model using rknn_toolkit2. The object detection results of the PyTorch, ONNX, and RKNN format models are compared below. Figure 17 a, Figure 17 b, Figure 17 As shown in Figure c, it can be seen that target identification and localization can be achieved under all three model formats.
[0035] Table 2 shows a comparison of the inference speeds of the two models trained in this invention on the RTXA4500 and RK3588 platforms, respectively. It can be seen that after format conversion, both YOLOv8n and YOLOv8s models can achieve inference performance with a spatial target update rate of over 30 FPS when deployed on the RK3588 platform, which meets the application requirements of this invention.
[0036] ; Typically, single-channel vision technology can only measure the motion of a target in a two-dimensional plane. To measure the motion of a target in three-dimensional space, at least dual-channel vision information is required. In this invention, to better detect the real-time position changes of the towed pod at the rear of the aircraft, two CMOS sensors are deployed on each of the aircraft's two wings to establish a dual-channel vision spatial positioning system for the towed pod. Specifically, a CMOS sensor is installed at both the wingtip and wing root on each side, and these sensors are connected to an onboard edge AI computing platform. The CMOS sensors on both wings acquire real-time images of the towed pod and transmit these images to the edge AI computing platform. By running a visual recognition model on the edge AI computing platform, the relative position information of the target (towed pod) in the image is identified, and the identified detection information is transmitted to the back-end processing module on the edge AI computing platform to complete the spatial position calculation of the towed pod. Figure 18 This is a structural diagram of the dual-path visual spatial positioning system of the present invention. The coordinates of the two CMOS sensors on the aircraft are known, and the CMOS sensors are used to collect real-time data of the towed pod.
[0037] by Figure 19 For example, the visual recognition output data is the coordinates of the four vertices of a rectangle centered on the target object. The accurate coordinates of the target object in the image are obtained through mathematical calculation. Table 3 shows the parameters of the CMOS sensor used in this invention.
[0038] ; Based on the CMOS size and focal length parameters in Table 3, the field of view of the CMOS sensor using a non-wide-angle lens is calculated as follows: ; In normal use of CMOS sensors, considering platform computing performance limitations, it is necessary to use CMOS sensors at reduced resolution. In this invention, the output image of the CMOS sensor is defined as... The feature length of each pixel can be calculated from the image resolution and sensor size. x .
[0039] ; ; Furthermore, the azimuth and elevation angles of a straight line in a spherical coordinate system, with the CMOS sensor as the origin and the target object as the endpoint, can be calculated using the following formula: This transforms finding the location of a spatial target into finding the intersection of two spatial lines.
[0040] ; Will Figure 20 a, Figure 20 The dual-path visual detection results of b are imported into a YOLOv8-based spatial localization algorithm to reconstruct the trajectory of the towed pod in space (e.g., Figure 21 ).
[0041] Table 4 shows the sensor measurement resolution calculated based on the CMOS sensor's viewing angle, output image resolution, and the distance between the towed pod and the CMOS sensor. It can be seen that the resolution of the CMOS sensor's output image determines the basic measurement error.
[0042] ; However, in real-world operating environments, aircraft are affected by unsteady wake fields. The towed pod will generate irregular oscillations in the wake field of the aircraft. The oscillations of the towed pod are transmitted to the connected wing through the tow cable, causing irregular oscillations on the upper part of the wing. At the same time, due to the instability of the incoming airflow, the wing will also generate irregular oscillations. These unexpected oscillations will further affect the accuracy of the images captured by the airborne CMOS sensor (mainly affecting the wingtip) from the towed pod. In order to better eliminate the impact of irregular oscillations on the load on the wing, this invention uses a two-way fluid-structure interaction method to analyze the impact of the towed pod on the wing and extract the oscillation data of key points as subsequent input.
[0043] When considering cable constraints, the entire large deformation component is actually affected by the cable. The cable material is usually steel cable, which plays a traction and fixation role after the towed pod is released. Because the terminal position is the relatively heavy antenna pod, the steel cable will also deform with the disturbance of airflow.
[0044] Considering the data interaction between the flow field mesh and the solid mesh, a tetrahedral mesh is used for the flow field, and the fluid domain computational mesh is as follows: Figure 22 As shown in Figure a, the mesh size is around 800W. During simulation, the mesh of the structural region is simplified (e.g., ...). Figure 22(b) Ignoring the small internal features of the wing structure, it is equivalent to a solid structure, and the total solid computational domain mesh is 4.2W.
[0045] During simulation, the fluid domain is solved first, and the aerodynamic forces applied to the fluid-structure interaction interface are calculated. These aerodynamic forces are then transferred to the solid domain via a coupling module to calculate the displacements in the solid domain. Finally, the displacement information from the solid domain is coupled back into the fluid domain for iterative calculation. The total computation time for the two-way fluid-structure interaction is calculated. Time step The simulation involved 20 iterations in the fluid domain, 25 iterations in the solid domain, and 3 iterations outside the coupling domain, for a total of 392,040 iterations in the fluid domain and 490,050 iterations in the solid domain. The final simulation result shows the positional change of the towed antenna pod as follows: Figure 23 a, Figure 23 b, Figure 23 c, Figure 23 As shown in Figure d, the towed pod is affected by gravity, the aerodynamic force of the incoming airflow, and the traction force of the cable, resulting in irregular oscillations in the wake region of the wing.
[0046] Figure 24 a, Figure 24 b, Figure 25 The value in C shows the displacement change of the towed pod, which is... Figure 24 As can be seen from b, the towed pod maintains irregular oscillations on the order of 10 cm in the vertical direction; from Figure 24 As shown in section a, the oscillation converges fastest in the direction of the incoming flow due to the greatest aerodynamic drag experienced by the pod, with the oscillation range narrowing to 2cm~4cm. Figure 24 As can be seen from c, the pod experiences extremely irregular oscillations in the spanwise direction due to the uneven velocity of the incoming flow on both sides, with an oscillation magnitude of 15cm.
[0047] In this invention, to reduce the impact of irregular oscillations on the acquired images, a MEMS sensor (such as...) is mounted on the CMOS sensor at the wingtip. Figure 25 This provides input data for image stabilization processing.
[0048] Figures 26-28 The image displays pitch and roll axis data from the embedded MEMS sensor within the CMOS sensor. To indicate the degree of jitter in the directly captured image, image optical flow is used to estimate the pitch and roll axis jitter (e.g., ...). Figure 27 a, Figure 27 (b) Image stabilization can be achieved by subtracting the oscillation data recorded by the MEMS sensor from the directly acquired image. The stabilized image is then subjected to optical flow method for pitch and roll axis jitter estimation, as shown in the figure. Figure 28 a, Figure 28As shown in Figure b, compared with the jitter estimate before processing, it can be found that the stability of the directly captured image is greatly improved after jitter subtraction.
[0049] Figure 29 The image shows a comparison of the displacement trajectory before and after image stabilization. Measurements show that the target center displacement caused by jitter before image stabilization was approximately 10 pixels. At this resolution, a measurement error of 20-30cm is introduced, and after image stabilization, the target center displacement still has a jitter of about 2 pixels. At this resolution, a measurement error of 5cm is introduced. The recognition data before and after image stabilization are then imported into a YOLOv8-based spatial localization algorithm to obtain the spatial localization of the towed pod before and after image stabilization (e.g., ...). Figure 30 a, Figure 30 As shown in b), it can be seen that the target jitter range after stabilization is smaller than the target jitter range before stabilization.
[0050] For electronic reconnaissance, the more accurate the reference receiver position, the better the positioning accuracy of the radiation source signal. Therefore, high-precision acquisition of the mobile pod's position is crucial. Conventional positioning uses differential GPS, but for signal reconnaissance, the accuracy of pod position acquisition is still relatively low due to error. Therefore, this invention uses differential GPS fused with dual-path vision for joint positioning. Specifically, differential GPS positioning is used to initially determine the location of the aircraft fuselage, the dual-path vision spatial positioning system is used to obtain the positions of the two towed pods, and then a three-station time difference method direction finding and positioning algorithm is used to determine the specific location of the ground radiation source.
[0051] The core principle of differential GPS relies primarily on the spatial and temporal correlations of satellite clock errors, satellite orbit errors, and ionospheric and tropospheric delays, such as... Figure 31 As shown, if a base station sends the error it measures to the receiver of a mobile station (user) via a data link, the mobile station can use this error data received from the base station to correct its distance measurements to the same satellite, thereby significantly improving its measurement and positioning accuracy.
[0052] Compared to visual-based target spatial positioning methods, differential GPS has a spatial positioning accuracy of 10-20 cm, lower than the 5 cm accuracy of dual-path visual spatial positioning. Since the edge AI computing platform used in this invention can only achieve image inference and spatial target position update rates of 30-40 FPS, and the position update rate of differential GPS is much higher than that of visual-based target spatial positioning methods, differential GPS positioning data can be used to compensate for the low dynamic range of visual positioning. In stable flight conditions, high-precision target positioning can be achieved using only the towed pod's spatial position calculated by visual methods. However, in unstable flight conditions, inaccurate positioning, and high-speed direction finding and positioning of targets, the pod's spatial position can be determined by combining visual and differential GPS methods.
[0053] The method of differential GPS fusion dual-channel visual spatial positioning is as follows: First, receive the dual-channel visual spatial positioning data, and determine whether visual spatial positioning data exists. If the visual spatial positioning is working well, it is used first. If there is no visual spatial positioning data, switch to high dynamic and low precision differential GPS spatial positioning. At this time, the accuracy of radiation source direction finding and positioning will be reduced.
[0054] This invention utilizes a three-station time difference of arrival (TDOA) method for direction finding and positioning to determine the location of a radiation source. TDOA positioning involves measuring the time difference between the arrival of the radiation source at different receiving points and then using mathematical methods to analytically solve for the target's location. The TDOA positioning algorithm calculates the distance difference between the radiation source and each receiving point by multiplying the time difference between the arrival of the radiation source at different receiving points by the electromagnetic wave propagation speed in space. The location of the radiation source is then determined using the TDOA positioning algorithm.
[0055] Suppose that the time difference positioning system consists of one main receiving station and N auxiliary receiving stations, and the two-dimensional coordinates of the receiving stations are: The coordinates of the radiation source are The distance between the radiation source and the receiving station can be expressed by the following formula: ; Right now: ; make: ; get, ; The distance between the radiation source and the auxiliary and main receiving stations can be expressed by the following formula: ①; In the formula, The speed of electromagnetic wave propagation. The time difference measurement between stations. , These represent the distance differences between the radiation source and the auxiliary receiving station, and the distance difference between the radiation source and the main receiving station, respectively, with the coordinates of the radiation source being... The two-dimensional coordinates of the main receiving station and the auxiliary receiving station are: ;in The main station is represented by one of the main stations, and the others by auxiliary receiving stations. 、 Main receiving station 、 Direction coordinates; To solve this system of linear equations, the above equation is linearized: ; Right now: ; when , Subtracting the two equations, we get: ; In the formula, , ,make , , , These are the absolute distances of the auxiliary receiving station and the main receiving station relative to the origin, respectively. , They are the main receiving station and the auxiliary receiving station. , The relative distance difference in direction; the above equation is a system of linear equations. The system can be solved using numerical iteration with initial values to obtain the coordinates of the radiation source.
[0056] Taking the two-dimensional three-station time difference positioning algorithm as an example, the time difference positioning system consists of one main receiving station and at least two auxiliary receiving stations, and its working principle is as follows: Figure 32 As shown, when the locations of the main receiving station and each auxiliary receiving station are known, and they receive pulse signals from the same radiation source, the time difference of arrival between the stations can be measured. Figure 32The time difference between the main receiving station C and the auxiliary receiving station A is proportional to the distance difference between the radiation source and these two stations. A hyperbola L1 with these two locations as its foci can be plotted. Similarly, the time difference between the main receiving station C and another auxiliary receiving station B can determine another hyperbola L2. The location of the radiation source is the intersection of these two hyperbolas. To locate the radiation source on a plane, at least three receiving stations are required. The main receiving station not only directly receives the signal emitted by the radiation source but also receives the signal relayed by the auxiliary receiving station. The main receiving station processes these signals, calculates the time difference between the radiation source signal arriving at the main receiving station and the auxiliary receiving station, and obtains the location of the radiation source through a direction-finding positioning algorithm. Then, the radiation source is tracked and identified. The auxiliary receiving station is a receiving station located at a certain distance. It receives the signal emitted by the radiation source and forwards the signal to the main receiving station.
[0057] Example: 1) The position of the towed pod was not updated using visual calculations: The results of two-way fluid-structure interaction simulation show that the towed pod, under stable towing conditions, exhibits aperiodic displacements of approximately 10 cm in the x, y, and z directions. Introducing this displacement data into the TDOA positioning algorithm model of this invention, the measured values are as follows: Figure 33 a, Figure 33 b, Figure 34 a, Figure 34 b, Figure 35 a, Figure 35 The impact of pod shaking on positioning accuracy, as shown in Figure b, is obtained from simulation results: the positioning results of axial targets are most affected by the shaking of the towed pod. The positioning error of the axial moving target at a distance of 100km from the receiving point under the state of towed pod shaking is 800~1000m, the positioning error of the spanwise moving target is about 100m, and the positioning error of the oblique moving target is about 200m.
[0058] Will Figures 33-35 The data in the table were converted to obtain the effect of pod shaking on target accuracy as shown in Table 5. It can be seen that the shaking of the towed pod has the greatest impact on the direction finding accuracy of the axial target.
[0059] ; 2) Visual calculation update of towed pod position: The dual-path visual spatial positioning system constructed by this invention dynamically updates the non-periodic 10cm displacements of the towed pod in the x, y, and z directions to a 5cm level, and then inputs this data into the TDOA positioning algorithm model to obtain the following results: Figure 36 a, Figure 36 b, Figure 37 a, Figure 37 b, Figure 38 a, Figure 38 The effect of the towed pod vibration on positioning accuracy, as shown in Figure b.
[0060] Will Figures 36-38 The data in the table were converted to obtain the impact of the towed pod position update on the target accuracy before and after the update, as shown in Table 6. It can be seen that the towed pod position update significantly improves the direction finding accuracy in all three directions of the target, with the maximum improvement reaching 150%.
[0061] ; The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. A high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning, characterized in that: The following steps are involved: Step 1: Build a visual recognition model; Step 2: Deploy the visual recognition model on the edge AI computing platform, build a dual-path visual spatial positioning system, and use the dual-path visual spatial positioning system to detect the real-time image of the towed pod; Step 3: Transmit the detected real-time image to the visual recognition model to obtain the spatial position information of the towed pod; Step 4: Fuse the differential GPS on the aircraft with the spatial position information of the towed pod, and determine the location of the radiation source using the three-station time difference direction finding positioning algorithm.
2. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 1 is characterized in that: In step 1, a visual recognition model is constructed using the YOLOv8 visual recognition network, and the steps include: Step 101: data collection; The two CMOS sensors on the simulated aircraft are used to collect images of the towed pod and obtain a dual-channel image dataset. Step 102: data labeling; Labelimg is used to annotate the collected dual-channel image dataset, and the annotated data is divided into training set, validation set and test set; Step 103: Optimize the YOLOv8 visual recognition network using the training set, validation set, and test set to obtain a visual recognition model.
3. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 2 is characterized in that: Step 103 of optimizing the YOLOv8 visual recognition network includes: The training set is used to train in the training mode embedded in the YOLOv8 visual recognition network, the validation set is used to verify the trained YOLOv8 visual recognition network, and the verified YOLOv8 visual recognition network is tested with the test set to obtain a visual recognition model.
4. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 3 is characterized in that: In step 2, the steps of building a dual-path visual space positioning system and using the dual-path visual space positioning system to detect the real-time image of the towed pod specifically include: Step 201: After converting the visual recognition model into RKNN format, deploy it on the edge AI computing platform; Step 202: CMOS sensors are installed at two wing tips and two wing roots of the aircraft respectively, and the CMOS sensors are connected to the edge AI computing platform to form a dual-path visual spatial positioning system for dynamic detection of the towed pod; Step 203: The real-time image of the towed pod is collected through the CMOS sensor and transmitted to the edge AI computing platform.
5. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 4 is characterized in that: In step 203, the MEMS sensor is embedded in the CMOS sensor, and the MEMS sensor is used to eliminate dynamic jitter of the real-time image collected by the CMOS sensor, thereby performing stabilization processing on the real-time image.
6. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 5 is characterized in that: Step 3: The step of obtaining the spatial position information of the towed pod includes: Step 301: using a visual recognition model to identify the position information of the towed pod in the real-time image; Step 302: Transmit the position information of the towed pod to the back-end processing module of the edge AI computing platform to calculate the spatial position of the towed pod.
7. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 6 is characterized in that: In step 4, the step of determining the location of the radiation source includes: Step 401: The three-station time difference method direction finding positioning algorithm measures the time difference between the radiation source reaching the main receiving station and two auxiliary receiving stations, wherein the main receiving station is a differential GPS antenna mounted on the aircraft fuselage, and the auxiliary receiving station is an antenna mounted on the towed pod; Step 402: The position of the radiation source is solved using the measured time difference between the main receiving station and the two auxiliary receiving stations. The solving process is as follows: (1) Calculate the distance difference between the radiation source and the main receiving station and the two auxiliary receiving stations. The expression is as follows: ①; In the formula, is the electromagnetic wave propagation speed, is the time difference measurement between the sites, , are the distance difference between the radiation source and the auxiliary receiving station, and the distance difference between the radiation source and the main receiving station, respectively, and the position coordinates of the radiation source are , the two-dimensional coordinates of the main receiving station and the auxiliary receiving station are ;in represents the main station, and the rest represent auxiliary receiving stations. 、 Main Receiving Station 、 Direction coordinate value; (2) By linearizing formula ①, we can obtain the linear equations for the radiation source position, which are expressed as follows: ②; ③; ④; In the formula, let , , are the absolute distances of the auxiliary receiving station and the main receiving station relative to the origin, , The main receiving station and the auxiliary receiving station are , Relative distance difference in direction; (3) Solve the linear equations of the radiation source and obtain the position coordinates of the radiation source.
8. A high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning, characterized in that: The system is implemented based on the high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 7, and the system includes: The system building module deploys the constructed visual recognition model on the edge AI computing platform to build a dual-path visual spatial positioning system; The detection module uses a dual-path visual spatial positioning system to detect the real-time image of the towed pod; A recognition module, which recognizes the real-time image of the towed pod and obtains the relative position information of the towed pod in the image; The back-end processing module uses the relative position information to calculate the spatial position of the towed pod; The calculation module is used to fuse the differential GPS on the aircraft with the spatial position information of the towed pod to calculate the position of the radiation source.
9. The high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning according to claim 8 is characterized in that: The detection module includes: A real-time acquisition submodule is used to acquire real-time images of the towed pod; The real-time de-jitter submodule is used to perform stabilization processing on the collected real-time images.
10. The high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning according to claim 8 is characterized in that: The computing module includes: The time difference measurement submodule is used to measure the time difference between the radiation source arriving at the main receiving station and the two auxiliary receiving stations; The position solving submodule obtains the distance difference between the radiation source reaching the main receiving station and the two auxiliary receiving stations through the time difference, and solves the position coordinates of the radiation source.
Citation Information
Patent Citations
Three-station time-difference-measuring stereoscopic positioning method
CN105044669A
Multi-sensor data fusion obstacle detection method for power line inspection unmanned aerial vehicle
CN108319982A
Near space aircraft relative pose measurement method and device based on binocular vision
CN111551151A
Pod attitude correction method and device based on three-dimensional scene model, and unmanned aerial vehicle
CN113415433A
Unmanned aerial vehicle height estimation method based on visual and inertial navigation information fusion neural network
CN114719848A
