High-precision signal time difference direction finding positioning method and system combined with dual-path vision pod spatial positioning

Abstract

The present invention is a high-precision signal time difference direction finding positioning method and system combined with dual-path visual pod spatial positioning, belonging to the field of unmanned aerial vehicle visual positioning technology. In view of the current problem that it is difficult to obtain the accurate position of the towed pod, the method of the present invention includes: constructing a visual recognition model; deploying the visual recognition model on an edge AI computing platform, building a dual-path visual spatial positioning system, and using the dual-path visual spatial positioning system to detect the real-time image of the towed pod; transmitting the detected real-time image to the visual recognition model to obtain the spatial position information of the towed pod; fusing the differential GPS on the aircraft with the spatial position information of the towed pod, and determining the position of the radiation source through a three-station time difference direction finding positioning algorithm. The method of the present invention can obtain the position information of the towed pod in real time, providing a guarantee for further accurate positioning of the ground radiation source signal.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle visual positioning, and in particular to a high-precision signal time difference direction finding positioning method and system combined with dual-path visual pod spatial positioning. Background Art

[0002] Due to the large number of signal frequency bands and complex types, in order to achieve high-precision wide-band signal reconnaissance, manned / unmanned reconnaissance aircraft need to pay extra weight and size costs, which seriously restricts the high maneuverability advantage of air-based platforms. Therefore, how to achieve wide-band signal reconnaissance as much as possible on a single small or medium-sized unmanned aerial platform is of great significance to improving signal reception efficiency.

[0003] Due to the characteristics of electromagnetic signals, the propagation attenuation of lower-frequency electromagnetic waves in the atmosphere is lower than that of high-frequency electromagnetic waves. Low-frequency electromagnetic waves are widely used in many scenarios such as communications and emergency rescue. However, the wavelength of low-frequency electromagnetic waves is longer than that of high-frequency electromagnetic waves, which makes it more difficult to find their direction and locate. Due to the constraint of the long wavelength of low-frequency electromagnetic waves, the size of air-based drones cannot be too small. Taking the typical American Global Hawk drone as an example, its wingspan reaches 35m, which meets the carrying requirements of low-frequency electromagnetic waves, but its unit price exceeds hundreds of millions of dollars and cannot be applied on a large scale. Small and medium-sized platforms are faced with the problem of small size and inability to sink to the low-frequency band to achieve single-machine reconnaissance. At present, towed antennas are not limited by the internal space of the aircraft and can reach a larger electrical size, enabling them to receive and send low-frequency electromagnetic waves, which solves this problem better.

[0004] Target positioning technology based on direction finding information is an important research direction in the field of passive positioning and is also one of the most common positioning methods. When the target and the direction finding station are in the same plane (i.e., considering two-dimensional spatial positioning), it is necessary to use the azimuth information obtained by the direction finding station to estimate the two-dimensional position coordinates of the target. However, when the target and the direction finding station are not in the same plane (i.e., considering three-dimensional spatial positioning), it is necessary to use the azimuth and elevation information obtained by the direction finding station to estimate the three-dimensional position coordinates of the target.

[0005] Although there have been a lot of research results on towed antennas and multi-band direction-finding and positioning technologies for UAVs / UAVs, the combination of the two and their application to small and medium-sized UAV platforms to detect and receive low-frequency electronic signals is still in the preliminary research stage. In addition, the towed antenna is towed by the traction wire rope and floats in the airflow. As the signal receiving end, its position jitter error will have a great impact on the detection result. It is difficult to obtain the accurate position of the towed pod, which will eventually cause a large deviation in the position determination of the ground radiation source signal. Summary of the invention

[0006] In view of the defects existing in the above-mentioned prior art, the present invention aims to propose a high-precision signal time difference direction finding positioning method and system combined with dual-path vision pod spatial positioning, with the purpose of using small and medium-sized carrier aircraft to detect low-frequency electronic signals, fully considering the impact of aircraft wake on towed pods, and constructing a high-precision positioning method for towed pod positioning that integrates differential GPS and dual-path vision, further reducing the jitter error in the operation of the towed pod, and obtaining the precise position of the ground radiation source with the improved three-station time difference direction finding positioning technology, thereby solving the problems existing in the background technology.

[0007] The above technical objectives of the present invention are achieved through the following technical solutions:

[0008] The technical solution of the first aspect disclosed in the present invention is: a high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning, comprising the following steps:

[0009] Step 1: Use the YOLOv8 visual recognition network to build a visual recognition model;

[0010] Step 2: Deploy the visual recognition model on the edge AI computing platform, build a dual-path visual spatial positioning system, and use the dual-path visual spatial positioning system to detect the real-time image of the towed pod;

[0011] Step 3: Transmit the detected real-time image to the visual recognition model to obtain the spatial position information of the towed pod;

[0012] Step 4: Fuse the differential GPS on the aircraft with the spatial position information of the towed pod, and determine the location of the radiation source using the three-station time difference direction finding positioning algorithm.

[0013] As a further preferred solution of the above technical solution, in step 1, a visual recognition model is constructed using a YOLOv8 visual recognition network, and the steps include:

[0014] Step 101: data collection;

[0015] The two CMOS sensors on the simulated aircraft are used to collect images of the towed pod and obtain a dual-channel image dataset.

[0016] Step 102: data labeling;

[0017] Labelimg is used to annotate the collected dual-channel image dataset, and the annotated data is divided into training set, validation set and test set;

[0018] Step 103: Optimize the YOLOv8 visual recognition network using the training set, validation set, and test set to obtain a visual recognition model.

[0019] A further preferred embodiment of the above scheme is as follows: Step 103 of optimizing the YOLOv8 visual recognition network includes:

[0020] The training set is used to train in the training mode embedded in the YOLOv8 visual recognition network, the validation set is used to verify the trained YOLOv8 visual recognition network, and the verified YOLOv8 visual recognition network is tested with the test set to obtain a visual recognition model.

[0021] As a further preferred solution of the above technical solution, in step 2, the step of building a dual-path visual space positioning system and using the dual-path visual space positioning system to detect the real-time image of the towed pod specifically includes:

[0022] Step 201: After converting the visual recognition model into RKNN format, deploy it on the edge AI computing platform;

[0023] Step 202: CMOS sensors are installed at two wing tips and two wing roots of the aircraft respectively, and the CMOS sensors are connected to the edge AI computing platform to form a dual-path visual spatial positioning system for dynamic detection of the towed pod;

[0024] Step 203: The real-time image of the towed pod is collected through the CMOS sensor and transmitted to the edge AI computing platform.

[0025] Based on the above technical solution, further, in step 203, the MEMS sensor is embedded in the CMOS sensor, and the MEMS sensor is used to eliminate dynamic jitter of the real-time image collected by the CMOS sensor, and stabilize the real-time image.

[0026] As a further preferred solution of the above technical solution, the step of obtaining the spatial position information of the towed pod in step 3 includes:

[0027] Step 301: using a visual recognition model to identify the position information of the towed pod in the real-time image;

[0028] Step 302: Transmit the position information of the towed pod to the back-end processing module of the edge AI computing platform to calculate the spatial position of the towed pod.

[0029] As a further preferred embodiment of the above technical solution, in step 4, the step of determining the position of the radiation source includes:

[0030] Step 401: The three-station time difference method direction finding positioning algorithm measures the time difference between the radiation source reaching the main receiving station and two auxiliary receiving stations, wherein the main receiving station is a differential GPS antenna mounted on the aircraft fuselage, and the auxiliary receiving station is an antenna mounted on the towed pod;

[0031] Step 402: The position of the radiation source is solved using the measured time difference between the main receiving station and the two auxiliary receiving stations. The solving process is as follows:

[0032] (1) Calculate the distance difference between the radiation source and the main receiving station and the two auxiliary receiving stations. The expression is as follows:

[0033] ①;

[0034] In the formula, is the electromagnetic wave propagation speed, is the time difference measurement between the sites, , are the distance difference between the radiation source and the auxiliary receiving station, and the distance difference between the radiation source and the main receiving station, respectively, and the position coordinates of the radiation source are , the two-dimensional coordinates of the main receiving station and the auxiliary receiving station are ;in represents the main receiving station, and the rest represent auxiliary receiving stations, and N The value is 2. 、 Main Receiving Station 、 Direction coordinate value;

[0035] (2) By linearizing formula ①, we can obtain the linear equations for the radiation source position, which are expressed as follows:

[0036] ②;

[0037] ③;

[0038] ④;

[0039] In the formula, let , , are the absolute distances of the auxiliary receiving station and the main receiving station relative to the origin, , The main receiving station and the auxiliary receiving station are , Relative distance difference in direction;

[0040] (3) Solve the linear equations of the radiation source and obtain the position coordinates of the radiation source.

[0041] The technical solution of the second aspect disclosed in the present invention is: a high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning, the system is implemented based on a high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning, the system comprises:

[0042] The system building module deploys the constructed visual recognition model on the edge AI computing platform, builds a dual-path visual spatial positioning system detection module, and uses the dual-path visual spatial positioning system to detect the real-time image of the towed pod;

[0043] A recognition module, which recognizes the real-time image of the towed pod and obtains the relative position information of the towed pod in the image;

[0044] The back-end processing module uses the relative position information to calculate the spatial position of the towed pod;

[0045] The calculation module is used to fuse the differential GPS on the aircraft with the spatial position information of the towed pod to calculate the position of the radiation source.

[0046] As a further preferred implementation of the above solution, the detection module includes:

[0047] A real-time acquisition submodule is used to acquire real-time images of the towed pod;

[0048] The real-time de-jitter submodule is used to perform stabilization processing on the collected real-time images.

[0049] As a further preferred implementation of the above solution, the calculation module includes:

[0050] The time difference measurement submodule is used to measure the time difference between the radiation source arriving at the main receiving station and the two auxiliary receiving stations;

[0051] The position solving submodule obtains the distance difference between the radiation source reaching the main receiving station and the two auxiliary receiving stations through the time difference, and solves the position coordinates of the radiation source.

[0052] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0053] 1. By installing two CMOS sensors on the wings on both sides of the aircraft, dynamic real-time image detection is performed on the towed pod at the tail of the wing, and the real-time image is transmitted to the visual recognition model based on the YOLOv8 network framework for recognition, the relative position information of the pod in the image is identified, and the spatial position of the relative position information is solved to obtain the spatial position information of the towed pod.

[0054] 2. In order to minimize the non-periodic oscillation of the wing caused by the towed pod, a MEMS sensor is embedded in the CMOS sensor to realize active de-jitter processing of the direct-collected images, and stabilize the real-time collected images to improve the stability of the collected images, effectively improving the accuracy of obtaining the towed pod position information.

[0055] 3. The present invention integrates differential GPS with a dual-path visual spatial positioning system, simultaneously obtains the spatial position information of the aircraft itself and the towed pod, and calculates the accurate position of the radiation source signal through a three-station time difference direction finding positioning algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a configuration diagram of the towed pod of the present invention;

[0057] Figure 2 This is a performance comparison chart of YOLOv8 and multiple versions of YOLO of the present invention;

[0058] Figure 3 An image acquisition system constructed in a laboratory for the present invention;

[0059] Figure 4 To build directory structures for different categories of data sets;

[0060] Figure 5 The confusion matrix generated by the training results of YOLOv8n and YOLOv8s of the present invention, wherein a is the confusion matrix of YOLOv8n and b is the confusion matrix of YOLOv8s;

[0061] Figure 6 The normalized confusion matrix generated by the training results of YOLOv8n and YOLOv8s of the present invention, wherein a is the normalized confusion matrix of YOLOv8n, and b is the normalized confusion matrix of YOLOv8s;

[0062] Figure 7 It is the evaluation index confidence curve diagram of the present invention, where a is the evaluation index confidence curve of YOLOv8n, and b is the evaluation index confidence curve of YOLOv8s;

[0063] Figure 8 It is a precision curve diagram of the present invention, where a is the precision curve of YOLOv8n, and b is the precision curve of YOLOv8s;

[0064] Fig. 9 It is a recall rate curve diagram of the present invention, where a is the recall rate curve of YOLOv8n, and b is the recall rate curve of YOLOv8s;

[0065] Fig.10It is a precision-recall curve diagram of the present invention, where a is the precision-recall curve of YOLOv8n, and b is the precision-recall curve of YOLOv8s;

[0066] Fig.11 Label image, where a is the label image of YOLOv8n and b is the label image of YOLOv8s;

[0067] Fig.12 is the label correlation graph, where a is the label correlation graph of YOLOv8n, and b is the label correlation graph of YOLOv8s;

[0068] Fig.13 is the training result, where a is the training result of YOLOv8n and b is the training result of YOLOv8s;

[0069] Fig.14 It is the test set image recognition result of the present invention;

[0070] Fig.15 OrangePi5 is a development board used to deploy the model of the present invention;

[0071] Fig.16 The model conversion process of the present invention;

[0072] Fig.17 This is a comparison of the target detection results based on Pytorch, ONNX and RKNN format models of the present invention;

[0073] Fig.18 It is a dual-path visual space positioning structure diagram of the present invention;

[0074] Fig.19 Output graph for visual recognition;

[0075] Fig. 20 is the dual-path visual recognition result of the present invention, where a is the recognition result of the left CMOS sensor; b is the recognition result of the right CMOS sensor;

[0076] Fig.21 To restore the spatial trajectory of the towed pod using the model of the present invention;

[0077] Fig. 22 The fluid domain grid diagram (a) and the solid domain grid diagram (b) of the present invention;

[0078] Fig.23 This is a transient change diagram of the position of the towed antenna pod obtained by simulation of the present invention;

[0079] Fig.24 The displacement variation characteristics of the towed pod in the flow direction (a), vertical direction (b), and span direction (c);

[0080] Fig.25 A CMOS+MEMS data acquisition system for eliminating jitter according to the present invention;

[0081] Fig.26 is the MEMS sensor data of the present invention, wherein a is the pitch axis gyroscope data, and b is the roll axis gyroscope data;

[0082] Fig. 27 is the original image two-axis jitter estimation value of the optical flow method of the present invention, wherein a is the pitch axis jitter estimation value of the optical flow method, and b is the roll axis jitter estimation value of the optical flow method;

[0083] Fig.28 is the estimated value of the two-axis jitter of the optical flow method after the image stabilization of the present invention, wherein a is the estimated value of the pitch axis jitter of the optical flow method, and b is the estimated value of the roll axis jitter of the optical flow method;

[0084] Fig.29 This is a comparison diagram of displacement trajectories before and after image stabilization of the present invention, where a is before image stabilization and b is after image stabilization;

[0085] Fig.30 is a comparison diagram of spatial displacement trajectories, where a is a comparison diagram of spatial displacement trajectories before image stabilization, and b is a comparison diagram of spatial displacement trajectories after image stabilization;

[0086] Fig.31 This is the schematic diagram of differential GPS;

[0087] Fig.32 It is a principle diagram of the direction finding and positioning using the three-station time difference method of the present invention;

[0088] Fig.33 Effects of introducing jitter error (a) and eliminating jitter error (b) on the lateral accuracy of the target for spanwise positioning of a towed pod;

[0089] Fig.34 Effects of introducing jitter error (a) and eliminating jitter error (b) on the lateral accuracy of the target for the axial positioning of a towed pod;

[0090] Fig.35 Effects of introducing jitter error (a) and eliminating jitter error (b) on the lateral accuracy of the target for the oblique positioning of a towed pod;

[0091] Fig.36 The influence of the spanwise positioning error of the towed pod on the lateral accuracy of the target before (a) and after (b) correction.

[0092] Fig.37 The influence of the axial positioning error of the towed pod on the lateral accuracy of the target before (a) and after (b) correction.

[0093] Fig.38 The influence of the towed pod’s oblique positioning error on the target’s lateral accuracy before (a) and after (b) correction. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0095] Reference Figure 1-Figure 38 The present invention provides a high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning, comprising the following steps:

[0096] Step 1: Use the YOLOv8 visual recognition network to build a visual recognition model;

[0097] Step 2: Deploy the visual recognition model on the edge AI computing platform, build a dual-path visual spatial positioning system, and use the dual-path visual spatial positioning system to detect the real-time image of the towed pod;

[0098] Step 3: Transmit the detected real-time image to the visual recognition model to obtain the spatial position information of the towed pod;

[0099] Step 4: Fuse the differential GPS on the aircraft with the spatial position information of the towed pod, and determine the location of the radiation source using the three-station time difference direction finding positioning algorithm.

[0100] Specifically, the present invention is based on the principle of deep learning target detection technology, and uses the newly released YOLOv8 visual recognition network framework to build a visual recognition model; YOLOv8 is based on the improved version of YOLO, which further improves flexibility and accuracy. YOLOv8 is fast, accurate, and easy to use, making it an excellent choice for completing a wide range of target detection and tracking, instance segmentation, image classification, and pose estimation tasks, such as Figure 2 ,From the performance comparison between YOLOv8 and multiple versions of YOLO, it can be found that YOLOv8 has significantly reduced the number of parameters under the same performance compared with the previous YOLO framework, and the inference speed of the model has also been greatly enhanced under the same number of parameters.

[0101] YOLOv8 is similar to the backbone network of YOLOv5, and has been changed on the CSP layer, called the C2f module. The C2f module (a cross-stage partial bottleneck with two convolutions) combines high-level features and contextual information to improve detection accuracy. YOLOv8 uses an anchor-free model with a decoupled head to independently handle objectivity, classification, and regression tasks, which allows each branch to focus on the design of its task, improving the overall accuracy of the model. The output layer of YOLOv8 uses the sigmoid function as the activation function of the target score, indicating the probability that the bounding box contains an object. It uses the softmax function for the probability of the class, indicating the probability of an object belonging to each possible class. YOLOv8 uses CIoU and DFL loss functions to represent the bounding box loss, and uses binary cross entropy to represent the classification loss. In addition, YOLOv8 also provides the semantic segmentation model of YOLOv8-Seg, whose backbone is a CSPDarknet53 feature extractor, which has achieved state-of-the-art results on various object detection and semantic segmentation benchmarks while maintaining high efficiency.

[0102] When building a visual recognition model, there are three main steps: data collection, data labeling, and model optimization:

[0103] (1) Data collection:

[0104] like Figure 3 As shown in the figure, a dual-channel image acquisition simulator is built in the laboratory to simulate the perspective of the CMOS sensor on the aircraft to acquire images of the towed pod. In real flight conditions, two CMOS sensors are required on each side of the aircraft's wings to identify and track the towed pod from two directions. Therefore, a two-angle dual-channel image acquisition simulator is built on the ground. The simulator consists of two left and right CMOS image sensors. Each of the left and right image sensors acquires 300 pictures at different angles to provide input data for the next step of data annotation.

[0105] (2) Data annotation:

[0106] Since the YOLOv8 visual recognition network framework used is a supervised learning model, it is necessary to annotate the collected data. Data annotation can help deep learning algorithms understand the meaning of data, such as the location, color, shape, and size of objects. The present invention uses Labelimg to annotate the 600 collected images, and divides the annotated data into training set, validation set, and test set in a ratio of 7:2:1. Figure 4 The directory structure shown builds the dataset.

[0107] (3) Model optimization:

[0108] The optimization of the visual recognition model includes model training, model verification and model testing. The YOLOv8 visual recognition network framework used in the present invention has a built-in model training mode. After the data set is built, training can be started with one click in CLI mode. Table 1 shows a comparison of the five basic training models provided by YOLOv8. According to the performance of the training platform and deployment platform used in this application, the present invention trains two models, YOLOv8n and YOLOv8s.

[0109] ;

[0110] like Figure 5 Middle a, Figure 5 As shown in b, the confusion matrix is ​​a tabular tool for evaluating the effectiveness of a classification model. It summarizes the sample results based on the categories predicted by the model and the actual categories (true values). When dealing with binary classification problems, the confusion matrix is ​​usually in the form of 2x2, covering four basic elements: true positive examples (True Positive, TP), true negative examples (True Negative, TN), false positive examples (False Positive, FP) and false negative examples (False Negative, FN). The normalized confusion matrix (as shown in Figure 1) obtained from the training results of YOLOv8n and YOLOv8s of the present invention is Figure 6 By dividing the value of each element by the actual number of samples in its corresponding category, the values ​​in the matrix are converted into percentages reflecting the classification accuracy. This allows for direct comparison of the classification accuracy between different categories and clarification of the advantages and disadvantages of different models.

[0111] like Figure 7 Middle a, Figure 7 Middle b, Figure 8 Middle a, Figure 8 Middle b, Fig. 9 Middle a, Fig. 9 As shown in b, during the training of the YOLOv8n and YOLOv8s models, the F1 curve depicts the changes in the F1 scores under different classification thresholds. The horizontal axis represents the confidence level and the vertical axis represents the F1 score. It can be obtained that the confidence of the YOLOv8n model is 0.822, while the confidence of the YOLOv8s model is 0.791; the accuracy curve shows the trend of accuracy increasing with confidence. Figure 8 Middle a, Figure 8Figure b shows that the detection accuracy of the two models increases with the increase of confidence; the recall rate curve represents the relationship between recall rate and confidence, showing that the recall rate decreases with the increase of confidence; the PR curve shows the relationship between the accuracy and recall rate of the model under different classification thresholds. The closer the PR curve is to the upper right corner of the coordinate system, the better the performance of the model is, the more accurately it can identify positive samples, and the higher the accuracy is. The closer it is to the left side, the weaker the model's ability to recognize positive samples is and the lower the recall rate is. An important feature of the PR curve is that as the classification threshold is adjusted, the accuracy and recall rate will change accordingly.

[0112] Generally, if the model can maintain high precision and high recall at the same time, the PR curve will be at a higher position. If the model tends to have high precision or high recall, the curve will shift toward low precision or low recall accordingly. The PR curve helps to evaluate the performance of the model at different thresholds and assists in selecting the appropriate threshold to balance precision and recall. The model can be quantitatively evaluated by comparing the area under the PR curve (i.e., average precision, AP). The larger the AP value, the better the model performance. Fig.10 Middle a, Fig.10 As shown in (b), the YOLOv8n and YOLOv8s models can show good performance while maintaining high precision and recall.

[0113] Fig.11 Middle a, Fig.11 The label image in b represents the category and bounding box information of each detected target, which mainly contains four types of information: target category, target position, target size, and target rotation angle; Fig.12 Middle a, Fig.12 The b in the figure is used to show the correlation between the predicted labels of YOLOv8n and YOLOv8s during the training process. The label correlation diagram is a color matrix diagram that shows the relationship between the data labels in the training set. It can reveal the behavior and performance of the target detection algorithm during the training phase, as well as the interaction between different predicted labels. By analyzing the label correlation diagram, we can deeply understand the strength of the algorithm's ability to distinguish between categories, as well as its accuracy in predicting different categories. In addition, comparing the label correlation diagrams of different algorithms or data sets helps to evaluate the algorithm performance and data set quality.

[0114] like Fig.13 Middle a, Fig.13As shown in b, after the training of the two models YOLOv8n and YOLOv8s is completed, the three columns on the left show multiple loss values ​​​​in the training and verification process, all of which are kept below 1, meeting the requirements of the performance indicators; the two columns on the right show several key performance indicators, including accuracy (measures the proportion of samples correctly predicted by the model. In target detection, if the predicted bounding box matches the true bounding box, the prediction is considered accurate), recall rate (measures the ability of the model to identify all true positive samples. In target detection, if the true bounding box matches the predicted bounding box, the sample is considered to be successfully recalled), mAP50 (i.e., average precision, the average precision calculated over multiple categories, mAP50 refers to the average precision at a 50% intersection-over-union (IoU) threshold), and mAP50-95 (the average precision calculated within the IoU threshold range of 50% to 95%. By taking the average of the mAP values ​​in this range, the performance of the model at different IoU thresholds can be more accurately evaluated). The intersection over union (IoU) is an indicator for evaluating the overlap between the predicted bounding box and the true bounding box. The higher the IoU value, the higher the similarity between the two bounding boxes. Usually, when IoU is greater than 0.5, the target is considered to be detected. By comparing the performance indicators, it can be found that the recognition performance difference between the YOLOv8n and YOLOv8s models on the used datasets is not much.

[0115] After the YOLOv8n and YOLOv8s models are trained, the trained visual recognition model is verified using the validation set, and then the visual recognition model is tested using the test set. The test set image recognition results are as follows: Fig.14 As shown, it can be seen that the visual recognition model of the present invention can perform good recognition on the test set images.

[0116] After the visual recognition model is optimized, the deployment problem needs to be considered. The present invention deploys the visual recognition model in the domestic development board OrangePi5 in the form of edge AI. Fig.15 As shown in the figure, the Rockchip RK3588 computing chip in OrangePi5 is used to realize real-time inference of video data and real-time output of inference results. The inference function of the RK3588 chip is mainly realized by the built-in high-performance NPU module, and the peak computing performance of its NPU module reaches 6TOPS; the model optimized by the YOLOv8 network framework is based on the Pytorch framework. The NPU module of the RK3588 chip only supports the RKNN format. The Pytorch format needs to be converted to the RKNN format. The specific conversion path is as follows Fig.16 As shown in the figure, the Pytorch format is converted to ONNX format first, and then the ONNX format model is converted to RKNN model using rknn_toolkit2. The target detection results of Pytorch, ONNX and RKNN format models are compared. Fig.17 Middle a, Fig.17 Middle b, Fig.17 As shown in Figure c, it can be seen that the target can be identified and located in all three model formats.

[0117] The comparison of the inference speed of the two models trained by the present invention on the RTXA4500 and RK3588 platforms is shown in Table 2. It can be seen that both the YOLOv8n and YOLOv8s models can achieve inference performance of a space target update rate of more than 30FPS after format conversion and are deployed on the RK3588 platform, meeting the application requirements of the present invention.

[0118] ;

[0119] Usually, single-channel vision technology can only measure the movement of the target on a two-dimensional plane. If it is necessary to measure the movement of the target in three-dimensional space through visual technology, at least two-channel vision information is required. In the present invention, in order to better detect the real-time position changes of the towed pod at the rear end of the aircraft, two CMOS sensors are respectively arranged on the wings on both sides of the aircraft to build a dual-channel visual space positioning system for the towed pod. Specifically: a CMOS sensor is installed at the wing tip and wing root on each side, and the CMOS sensor is connected to the airborne edge AI computing platform. The CMOS sensors on both sides of the wings perform real-time image acquisition of the towed pod, and transmit the acquired real-time image to the edge AI computing platform. By running the visual recognition model carried on the edge AI computing platform, the relative position information of the target (towed pod) in the picture is identified, and the identified detection information is transmitted to the back-end processing module on the edge AI computing platform to complete the spatial position solution of the towed pod, such as Fig.18 This is a structural diagram of the dual-path visual space positioning system of the present invention. The coordinates of the two CMOS sensors on the aircraft are known, and the CMOS sensors are used to collect real-time data of the towed pod.

[0120] by Fig.19 For example, the data output by visual recognition is the coordinates of the four vertices of a rectangular frame centered on the target object. The accurate coordinates of the target object in the picture are obtained by mathematical calculation. Table 3 shows the parameters of the CMOS sensor used in the present invention.

[0121] ;

[0122] Calculated from the CMOS size and focal length parameters in Table 3, the field of view of the CMOS sensor using a non-wide-angle lens is:

[0123] ;

[0124] In the normal use of CMOS sensors, considering the platform computing performance problem, it is necessary to use the CMOS sensor with reduced resolution. In the present invention, the output image of the CMOS sensor is defined as , the characteristic length of each pixel can be calculated from the image resolution and sensor size x .

[0125] ;

[0126] ;

[0127] Then, the following formula can be used to calculate the azimuth and elevation of the spherical coordinate system space straight line with the CMOS sensor as the origin and the target as the end point. , transforming the search for the spatial target position into the search for the intersection of two spatial lines.

[0128] ;

[0129] Will Fig. 20 Middle a, Fig. 20 The dual-path visual detection results in b are imported into the spatial positioning algorithm based on the YOLOv8 network to restore the trajectory of the towed pod in space (such as Fig.21 ).

[0130] Table 4 shows the sensor measurement resolution calculated based on the CMOS sensor viewing angle, output image resolution and the distance between the towed pod and the CMOS sensor. It can be found that the resolution of the CMOS sensor output image determines the basic measurement error.

[0131] ;

[0132] However, in a real operating environment, the aircraft is affected by the non-steady-state wake field, and the towed pod will produce irregular oscillations in the aircraft wake field. The oscillations of the towed pod are transmitted to the connected wing through the towing cable, which will cause irregular oscillations on the wing. At the same time, due to the instability of the aircraft's incoming flow, the wing will also produce irregular oscillations. This unexpected oscillation will further affect the accuracy of the image captured by the onboard CMOS sensor (mainly affecting the wing tip) of the towed pod. In order to better eliminate the influence of irregular oscillations on the load on the wing, the present invention uses a two-way fluid-solid coupling method to analyze the influence of the towed pod on the wing, and extracts the oscillation data of key points as subsequent input.

[0133] When considering cable constraints, the entire large deformation component is actually affected by the cable. Usually the cable material is a steel cable, which plays a traction and fixing role after the towed pod is released. Because the terminal position is a heavier antenna cabin, the steel cable will also deform with the disturbance of the airflow.

[0134] Considering the interaction between flow field grid and solid grid data, the flow field grid adopts tetrahedral grid, and the fluid domain calculation grid is as follows: Fig. 22 As shown in a, the mesh volume is about 800W. During simulation, the mesh of the structural area is simplified (such as Fig. 22 (b) The small internal features of the wing structure are ignored and it is equivalent to a solid structure. The total number of meshes in the solid computational domain is 4.2W.

[0135] During simulation, the fluid domain is solved first, and the fluid domain calculates the aerodynamic force applied to the fluid-solid coupling interface. Then, the aerodynamic force is transferred to the solid domain through the coupling module, the displacement of the solid domain is calculated, and then the displacement information of the solid domain is coupled to the fluid domain for iterative calculation. , time step , 20 iterations in the fluid domain, 25 iterations in the solid domain, 3 iterations outside the coupling, a total of 392040 iterations in the fluid domain, and 490050 iterations in the solid domain. The final simulated position change of the towed antenna pod is shown in the figure below: Fig.23 Middle a, Fig.23 Middle b, Fig.23 Middle c, Fig.23 As shown in (d), the towed pod is affected by gravity, incoming aerodynamic force and the traction of the cable, causing irregular oscillations in the wake area of ​​the wing.

[0136] Fig.24 Middle a, Fig.24 Middle b, Fig.25 The displacement change of the towed pod is shown in c. Fig.24 From b, we can see that the towed pod maintains irregular oscillations of 10 cm in the vertical direction. Fig.24 From a, we can see that in the incoming flow direction, since the aerodynamic resistance of the pod is the largest in the incoming flow direction, the oscillation in this direction converges fastest, and the oscillation range converges to 2cm~4cm; Fig.24 As shown in Figure c, the spanwise direction of the pod is affected by the uneven velocity of the incoming flow on both sides, causing extremely irregular oscillations with an oscillation magnitude of 15 cm.

[0137] In the present invention, in order to reduce the impact of irregular vibration on the collected image, a MEMS sensor (such as Fig.25 ), which provides input data for image stabilization processing.

[0138] Figure 26-Figure 28 The pitch and roll axis data of the MEMS sensor embedded in the CMOS sensor are displayed. In order to show the degree of jitter of the direct-collected image, the image optical flow method is used to estimate the jitter of the pitch and roll axes of the direct-collected image (such as Fig. 27Middle a, Fig. 27 In (b), the image can be stabilized by subtracting the vibration data recorded by the MEMS sensor from the direct image. The stabilized image is then used to estimate the jitter of the pitch and roll axes of the image using the optical flow method. The results are shown in Fig.28 Middle a, Fig.28 As shown in b, compared with the jitter estimation value before processing, it can be found that the stability of the direct-collected image is greatly improved after jitter reduction.

[0139] Fig.29 The comparison of displacement trajectories before and after image stabilization is shown. According to the measurement, the target center displacement caused by jitter before image stabilization is about 10pix. The resolution is equivalent to introducing a measurement error of 20~30cm. The target center displacement after image stabilization has a residual jitter of about 2pix. The resolution is equivalent to introducing a measurement error of 5 cm; the recognition data before and after stabilization are respectively imported into the spatial positioning algorithm based on the YOLOv8 network to obtain the spatial positioning of the towed pod before and after image stabilization (such as Fig.30 Middle a, Fig.30 (b) It can be seen that the target jitter range after stabilization is smaller than the target jitter range before stabilization.

[0140] For electronic reconnaissance, the more accurate the receiving end position of the reference is, the more accurate the positioning of the radiation source signal can be, so it is very important to obtain the position of the mobile pod with high precision. Conventional positioning uses the differential GPS method, but for signal detection, due to the influence of errors, the accuracy of obtaining the pod position is still low. Therefore, the present invention uses differential GPS fusion dual-path vision for joint positioning, specifically: the position of the aircraft fuselage is initially obtained through differential GPS positioning, the position of the two towed pods is obtained using the dual-path vision space positioning system, and then the specific position of the ground radiation source is determined by the three-station time difference method direction finding positioning algorithm.

[0141] The core principle of differential GPS mainly relies on the spatial and temporal correlation of satellite clock errors, satellite orbit errors, ionospheric and tropospheric delays, such as Fig.31 As shown in the figure, if a base station sends the error it measures to the receiver of a mobile station (user) via a data link, the mobile station can use the error data received from the base station to correct its own distance measurement to the same satellite, thereby significantly improving its measurement and positioning accuracy.

[0142] Compared with the target spatial positioning method based on visual information, the spatial positioning accuracy of differential GPS is 10~20cm, which is lower than the 5cm positioning accuracy of dual-path visual spatial positioning. Since the edge AI computing platform adopted by the present invention can only achieve an image reasoning and spatial target position update rate of 30~40FPS, the position update rate of differential GPS is much higher than the target spatial positioning method based on vision, so the positioning data of differential GPS can be used to make up for the disadvantage of low dynamics of visual positioning; when the flight state is stable, the high-precision positioning of the target can only use the towed pod spatial position calculated by the visual method; but under the conditions of unstable flight state, non-precise positioning and high-speed direction finding and positioning of the target, the pod spatial position can be measured by combining vision and differential GPS methods.

[0143] The method of differential GPS fusion dual-path visual spatial positioning is as follows: first, receive the data of dual-path visual spatial positioning, and determine whether there is visual spatial positioning data. If the visual spatial positioning works well, visual spatial positioning is used first; if there is no visual spatial positioning data, switch to high-dynamic and low-precision differential GPS spatial positioning. At this time, the accuracy of radiation source direction finding and positioning will be reduced.

[0144] The present invention uses the three-station time difference method direction finding positioning algorithm to determine the position of the radiation source. The time difference method positioning is to measure the time difference between the radiation source arriving at different receiving points, and then use mathematical methods to analyze and solve the position of the target. The arrival time difference (TDOA) positioning algorithm measures the time difference between the radiation source arriving at different receiving points, multiplies it by the spatial propagation speed of electromagnetic waves to obtain the distance difference between the radiation source and each receiving point, and then uses the time difference positioning algorithm to obtain the position of the radiation source.

[0145] Assume that the time difference positioning system consists of a main receiving station and N auxiliary receiving stations. The two-dimensional coordinates of the receiving stations are , the position coordinates of the radiation source are , the distance between the radiation source and the receiving site can be expressed by the following formula:

[0146] ;

[0147] Right now: ;

[0148] make: ;

[0149] get, ;

[0150] The distance between the radiation source and the auxiliary receiving station and the main receiving station can be expressed by the following formula:

[0151] ①;

[0152] In the formula, is the electromagnetic wave propagation speed, is the time difference measurement between the sites, , are the distance difference between the radiation source and the auxiliary receiving station, and the distance difference between the radiation source and the main receiving station, respectively, and the position coordinates of the radiation source are , the two-dimensional coordinates of the main receiving station and the auxiliary receiving station are ;in represents the main receiving station, and the rest represent auxiliary receiving stations, and N The value is 2. 、 Main Receiving Station 、 Direction coordinate value;

[0153] In order to solve the linear equations, the above equation is linearized:

[0154] ;

[0155] Right now: ;

[0156] when , , subtracting the two equations, we get:

[0157] ;

[0158] In the formula, , ,make , , , are the absolute distances of the auxiliary receiving station and the main receiving station relative to the origin, , The main receiving station and the auxiliary receiving station are , The relative distance difference in direction; the above formula is a linear equation group, which can be solved by giving the initial value using the numerical iteration method to obtain the coordinates of the radiation source. The relative distance difference in direction; the above formula is a linear equation group, which can be solved by giving the initial value using the numerical iteration method to obtain the coordinates of the radiation source.

[0159] Taking the two-dimensional three-station time difference positioning algorithm as an example, the time difference positioning system consists of a main receiving station and at least two auxiliary receiving stations. Its working principle is as follows: Fig.32 As shown in Figure 1, when the positions of the main receiving station and each auxiliary receiving station are known and they receive pulse signals from the same radiation source, the arrival time difference between the stations can be measured. Fig.32The time difference between the main receiving station C and the auxiliary receiving station A is proportional to the distance difference between the radiation source and the two stations. A hyperbola L1 with these two positions as the focus can be drawn. Similarly, the time difference between the main receiving station C and another auxiliary receiving station B can determine another hyperbola L2. The position of the radiation source is the intersection of these two hyperbolas. In order to locate the radiation source on the plane, at least three receiving stations are required. The main receiving station not only directly receives the signal emitted by the radiation source, but also receives the signal forwarded by the auxiliary receiving station. The main receiving station processes these signals, calculates the time difference between the radiation source signal arriving at the main receiving station and the auxiliary receiving station, and obtains the position of the radiation source through the direction finding positioning algorithm, and then tracks and identifies the radiation source. The auxiliary receiving station is a receiving station located at a certain distance. It receives the signal emitted by the radiation source and forwards the signal to the main receiving station.

[0160] Example:

[0161] 1) The position of the towed pod is not updated without visual calculation:

[0162] From the bidirectional fluid-structure coupling simulation results, it can be seen that the towed pod produces non-periodic displacements of 10 cm in the x, y and z directions respectively under the stable towing state. The displacement data of the towed pod is introduced into the TDOA positioning algorithm model of the present invention, and the measured results are as follows: Fig.33 Middle a, Fig.33 Middle b, Fig.34 Middle a, Fig.34 Middle b, Fig.35 Middle a, Fig.35 The influence of the pod jitter on the positioning accuracy shown in b is obtained from the simulation results: the axial target positioning result is most affected by the towed pod jitter. When the towed pod is jittering, the positioning error of the axial moving target at a distance of 100 km from the receiving point is 800~1000m, the positioning error of the span-moving target is about 100m, and the positioning error of the oblique moving target is 200m.

[0163] Will Figure 33-Figure 35 The data in are converted to obtain the influence of the pod jitter on the target accuracy as shown in Table 5. It can be found that in the towed pod jitter state, the influence on the direction finding accuracy of the axial target is the greatest.

[0164] ;

[0165] 2) Update of visual calculation of towed pod position:

[0166] The dual-path visual spatial positioning system constructed by the present invention generates a non-periodic 10 cm displacement in the x, y and z directions of the towed pod, which is dynamically updated and quantized to 5 cm. The displacement is then introduced into the TDOA positioning algorithm model to obtain the following: Fig.36 Middle a, Fig.36 Middle b, Fig.37 Middle a, Fig.37 Middle b, Fig.38 Middle a, Fig.38 The impact of towed pod jitter on positioning accuracy shown in (b).

[0167] Will Figure 36-Figure 38 The data in are converted to obtain the influence of the towed pod position update on the target accuracy as shown in Table 6. It can be seen that the direction finding accuracy of the target in three directions is significantly improved after the towed pod position is updated, and the maximum improvement effect reaches 150%.

[0168] ;

[0169] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning, characterized in that: The following steps are involved: Step 1: Use the YOLOv8 visual recognition network to build a visual recognition model; Step 2: Deploy the visual recognition model on the edge AI computing platform, build a dual-path visual spatial positioning system, and use the dual-path visual spatial positioning system to detect the real-time image of the towed pod; Step 3: Transmit the detected real-time image to the visual recognition model to obtain the spatial position information of the towed pod; Step 4: Fuse the differential GPS on the aircraft with the spatial position information of the towed pod, and determine the location of the radiation source using the three-station time difference direction finding positioning algorithm.

2. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 1 is characterized in that: In step 1, a visual recognition model is constructed using the YOLOv8 visual recognition network, and the steps include: Step 101: data collection; The two CMOS sensors on the simulated aircraft are used to collect images of the towed pod and obtain a dual-channel image dataset. Step 102: data labeling; Labelimg is used to annotate the collected dual-channel image dataset, and the annotated data is divided into training set, validation set and test set; Step 103: Optimize the YOLOv8 visual recognition network using the training set, validation set, and test set to obtain a visual recognition model.

3. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 2 is characterized in that: Step 103 of optimizing the YOLOv8 visual recognition network includes: The training set is used to train in the training mode embedded in the YOLOv8 visual recognition network, the validation set is used to verify the trained YOLOv8 visual recognition network, and the verified YOLOv8 visual recognition network is tested with the test set to obtain a visual recognition model.

4. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 3 is characterized in that: In step 2, the steps of building a dual-path visual space positioning system and using the dual-path visual space positioning system to detect the real-time image of the towed pod specifically include: Step 201: After converting the visual recognition model into RKNN format, deploy it on the edge AI computing platform; Step 202: CMOS sensors are installed at two wing tips and two wing roots of the aircraft respectively, and the CMOS sensors are connected to the edge AI computing platform to form a dual-path visual spatial positioning system for dynamic detection of the towed pod; Step 203: The real-time image of the towed pod is collected through the CMOS sensor and transmitted to the edge AI computing platform.

5. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 4 is characterized in that: In step 203, the MEMS sensor is embedded in the CMOS sensor, and the MEMS sensor is used to eliminate dynamic jitter of the real-time image collected by the CMOS sensor, thereby performing stabilization processing on the real-time image.

6. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 5 is characterized in that: Step 3: The step of obtaining the spatial position information of the towed pod includes: Step 301: using a visual recognition model to identify the position information of the towed pod in the real-time image; Step 302: Transmit the position information of the towed pod to the back-end processing module of the edge AI computing platform to calculate the spatial position of the towed pod.

7. The high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 6 is characterized in that: In step 4, the step of determining the location of the radiation source includes: Step 401: The three-station time difference method direction finding positioning algorithm measures the time difference between the radiation source reaching the main receiving station and two auxiliary receiving stations, wherein the main receiving station is a differential GPS antenna mounted on the aircraft fuselage, and the auxiliary receiving station is an antenna mounted on the towed pod; Step 402: The position of the radiation source is solved using the measured time difference between the main receiving station and the two auxiliary receiving stations. The solving process is as follows: (1) Calculate the distance difference between the radiation source and the main receiving station and the two auxiliary receiving stations. The expression is as follows: ①； In the formula, is the electromagnetic wave propagation speed, is the time difference measurement between the sites, , are the distance difference between the radiation source and the auxiliary receiving station, and the distance difference between the radiation source and the main receiving station, respectively, and the position coordinates of the radiation source are , the two-dimensional coordinates of the main receiving station and the auxiliary receiving station are ;in represents the main receiving station, and the rest represent auxiliary receiving stations, and N The value is 2. 、 Main Receiving Station 、 Direction coordinate value; (2) By linearizing formula ①, we can obtain the linear equations for the radiation source position, which are expressed as follows: ②； ③； ④； In the formula, let , , are the absolute distances of the auxiliary receiving station and the main receiving station relative to the origin, , The main receiving station and the auxiliary receiving station are , Relative distance difference in direction; (3) Solve the linear equations of the radiation source and obtain the position coordinates of the radiation source.

8. A high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning, characterized in that: The system is implemented based on the high-precision signal time difference direction finding positioning method combined with dual-path visual pod spatial positioning according to claim 7, and the system includes: The system building module deploys the constructed visual recognition model on the edge AI computing platform to build a dual-path visual spatial positioning system; The detection module uses a dual-path visual spatial positioning system to detect the real-time image of the towed pod; A recognition module, which recognizes the real-time image of the towed pod and obtains the relative position information of the towed pod in the image; The back-end processing module uses the relative position information to calculate the spatial position of the towed pod; The calculation module is used to fuse the differential GPS on the aircraft with the spatial position information of the towed pod to calculate the position of the radiation source.

9. The high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning according to claim 8 is characterized in that: The detection module includes: A real-time acquisition submodule is used to acquire real-time images of the towed pod; The real-time de-jitter submodule is used to perform stabilization processing on the collected real-time images.

10. The high-precision signal time difference direction finding positioning system combined with dual-path visual pod spatial positioning according to claim 8 is characterized in that: The computing module includes: The time difference measurement submodule is used to measure the time difference between the radiation source arriving at the main receiving station and the two auxiliary receiving stations; The position solving submodule obtains the distance difference between the radiation source reaching the main receiving station and the two auxiliary receiving stations through the time difference, and solves the position coordinates of the radiation source.

Images