Vehicle, personnel and ship real-time three-dimensional projection system based on unmanned aerial vehicle visual identification

Abstract

The invention discloses a vehicle, personnel and ship real-time three-dimensional projection system based on unmanned aerial vehicle visual identification, and the method comprises the steps: obtaining an optimal visual identification model through a visual identification module, achieving the multi-target identification, and obtaining an identification result; target space positioning is carried out through a space positioning module according to the recognition result and GNSS data, target trajectory tracking is achieved in combination with a Kalman filtering algorithm, and a real-time space positioning result is obtained; and acquiring a target three-dimensional position through a three-dimensional projection rendering module according to the recognition result, the real-time space positioning result and the unmanned aerial vehicle attitude angle data, and realizing real-time three-dimensional projection display according to the target three-dimensional position and the target type by adopting an instantiation rendering technology. According to the invention, the problems of low multi-target joint recognition precision, contradiction between positioning precision and real-time performance and weak cross-scene adaptability in the prior art are solved, and the problem that the recognition / positioning precision and efficiency of multiple targets in a complex scene cannot be effectively realized is solved.

Description

Technical Field

[0001] This invention relates to the fields of UAV visual recognition, 3D spatial computing and geographic information visualization fusion technology, and in particular to a real-time 3D projection system for vehicles, personnel and ships based on UAV visual recognition. Background Technology

[0002] With the rapid development of low-altitude economy and intelligent sensing technology, drones are increasingly being used in urban governance (such as traffic flow monitoring), emergency dispatch (such as search and rescue at disaster sites), and coastal control (such as patrolling for illegal fishing vessels). However, existing systems still have the following key technical shortcomings: (1) Low accuracy of multi-target joint recognition: Traditional systems use a single YOLOv5 model. In urban high-rise building occlusion scenarios, the vehicle recognition accuracy drops to below 75%; in water reflection scenarios, the boat recognition miss rate exceeds 20%, which cannot meet the needs of simultaneous monitoring of multiple types of targets. (2) The contradiction between positioning accuracy and real-time performance: Some systems use pure GNSS positioning, and the positioning error exceeds 5 meters due to obstruction; some rely on visual measurement, and the positioning fails in scenes without feature points (such as open water), and the end-to-end delay generally exceeds 1000ms, making it difficult to support real-time decision-making. (3) Weak cross-scene adaptability: The existing system has not designed adaptation schemes for different scenes. For example, in low light scene (light intensity ≤100 lux), personnel recognition is completely ineffective; in complex terrain (such as mountainous and bridge areas), the three-dimensional projection is prone to coordinate offset, with an offset of more than 3 meters.

[0003] Therefore, in order to solve the above-mentioned technical problems, there is an urgent need to provide a real-time 3D projection system for vehicles, personnel and ships based on UAV visual recognition. Summary of the Invention

[0004] This invention provides a real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition to overcome the aforementioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition includes a data acquisition module, a data processing module, a visual recognition module, a spatial positioning module, and a 3D projection rendering module connected via a communication link. The data acquisition module is used to acquire multi-target image data for UAV visual recognition, GNSS data for target localization in the multi-target image data, and UAV attitude angle data acquired by sensors. Furthermore, the multi-target image data includes, but is not limited to, labeled images of different types of vehicles, people in different postures / ages / clothing, and different tonnage / water scenes in different environmental scenarios; and the labeled images include visible light images and infrared images. The data processing module is used to preprocess multi-target image data to obtain a preprocessed image; The visual recognition module is used to obtain the optimal visual recognition model based on the preprocessed image, based on the constructed lightweight target detection model, and then to achieve multi-target recognition to obtain recognition results; The recognition result includes the target recognition type and its corresponding target bounding box; The spatial positioning module is used to locate the target in space based on the identification results and GNSS data, and to track the target trajectory by combining the Kalman filter algorithm to obtain real-time spatial positioning results; The 3D projection rendering module is used to obtain the 3D position of the target based on the recognition results, real-time spatial positioning results, and UAV attitude angle data, and uses instantiation rendering technology to realize real-time 3D projection display based on the 3D position and target type of the target.

[0006] Furthermore, the method for preprocessing multi-target image data is as follows: S100: Based on the Gaussian filtering algorithm, the visible light image in the multi-target image data is filtered to obtain the visible light filtered image; S101: Obtain the intrinsic parameters of the camera used to capture visible light images on the UAV by Zhang Zhengyou calibration method, and perform distortion correction on the visible light filtered image based on the camera intrinsic parameters using the fisheye lens distortion method, and obtain the distortion-corrected image. S102: Using the IMU sampling timestamp of the UAV attitude angle data acquired by the sensor as a reference, the distortion-corrected image and the infrared image are time-aligned to obtain a pre-processed image.

[0007] Furthermore, the constructed lightweight target detection model includes an input layer, a backbone network, a neck network, and a detection head; The input layer is used to input the preprocessed image into the backbone network; The backbone network adopts Networks are used to extract target feature vectors at multiple scales; Furthermore, the target features of the target feature vector include contour features in the distortion-corrected image and target pixel features in the infrared image; The neck network adopts A network structure used to fuse multi-scale target feature vectors to obtain a fused feature vector. The detection head is used to obtain the target recognition type and target bounding box corresponding to the detected target based on the fused feature vector; The expression for the detection head is: P(detect) = σ(W) X+b); where P(detect) represents the target detection probability; σ represents Activation function; W represents the weight matrix; X represents the fused feature vector; b represents the bias term.

[0008] Furthermore, the method for obtaining the optimal visual recognition model based on the preprocessed image includes the following steps: S200: Divide the preprocessed image into a training set and a test set according to a preset ratio: S201: Using the distortion-corrected image and infrared image as feature data and the target category as label data, train the lightweight target detection model based on the training set to obtain the trained lightweight target detection model. S202: Based on The loss function is used to confirm whether the output of the trained lightweight object detection model has converged based on the test set. If so, the lightweight object detection model trained at this time will be taken as the optimal visual recognition model; If not, the parameters of the trained lightweight object detection model are adaptively adjusted based on the backpropagation method, and step S201 is repeated until the weight parameters of the trained lightweight object detection model with converged output are confirmed to be the optimal weight parameters, and the lightweight object detection model is reconstructed to obtain the optimal visual recognition model.

[0009] Furthermore, the method for obtaining real-time spatial positioning results specifically includes: S300: Based on the identification results and GNSS data, perform target spatial positioning to obtain the current target location; And the expression for target spatial localization is: X=(x- ) / f*Z,Y=(y- ) / f*Z,Z=f B / d In the formula: X represents the east coordinate of the target in the WGS84 coordinate system; Y represents the north coordinate of the target in the WGS84 coordinate system; Z represents the vertical coordinate in the WGS84 coordinate system; f represents the focal length of the camera in the UAV acquisition module; , The coordinates of the camera principal point obtained by Zhang Zhengyou's calibration method are given; B represents the baseline length in the multi-view geometry obtained from GNSS data; d represents the parallax; and x and y represent the center coordinates of the target bounding box. S301: Based on the Kalman filter algorithm, target trajectory tracking is achieved according to the current target position to obtain real-time spatial positioning results.

[0010] Furthermore, the expression for obtaining the target's three-dimensional position based on the recognition results, real-time spatial positioning results, and UAV attitude angle data is as follows: P=K[R|t] X_w In the formula: P represents the projected coordinates of the target on the image plane; X_w represents the three-dimensional spatial coordinate vector of the target and X_w=[X, Y, Z, 1]^T; T represents the transpose; t represents the translation vector determined by the difference between the GNSS data and the initial position of the UAV; R represents the 3×3 orthogonal matrix obtained from the UAV attitude angle data, i.e., the rotation matrix; K represents the camera intrinsic parameter matrix and K=[[f, 0, ], [0, f, ], [0, 0, 1]].

[0011] Furthermore, it also includes an AI-assisted decision-making module: The AI-assisted decision-making module is used to construct an auxiliary decision-making mechanism based on the acquired real-time spatial positioning results, so as to achieve real-time tracking and response to the target; And the aforementioned decision support mechanism is as follows: S400: Based on the sampling period, obtain the set of historical trajectory points and the corresponding first motion velocity features according to the historical trajectory data of the target; Simultaneously, based on the sampling period and the real-time spatial positioning results, a set of real-time trajectory points and corresponding second motion velocity features are obtained. S401: The density-based DBSCAN clustering algorithm is used to obtain the first trajectory point density of the corresponding historical trajectory point set and the second trajectory point density of the corresponding real-time trajectory point set; S403: Set a trajectory point density threshold based on the first trajectory point density, and set a velocity change threshold based on the first motion velocity characteristic; and use the trajectory point density threshold, velocity change threshold, and preset deviation area as target anomaly determination conditions; S404: Based on the second trajectory point density, the second motion velocity characteristics, and the target's current location area, determine whether the target anomaly determination conditions are met; If any one of the following indicators is met: the second trajectory point density is less than or equal to the trajectory point density threshold, the velocity mutation value of the second motion velocity feature is greater than or equal to the velocity mutation threshold, or the target's current location area is outside the preset deviation area, then the target is determined to be in an abnormal mode and S405 is executed. Otherwise, the target is determined to be in normal mode; S405: Retrieve the exception data corresponding to the exception mode: Furthermore, the abnormal data includes the target ID, abnormality type, and real-time location coordinates; The monitoring terminal, drone flight control system, and audible and visual alarm devices are connected to the AI-assisted decision-making module via a multi-terminal linkage interface. The current target's status is monitored in real time via a monitoring terminal; Real-time target tracking is achieved by adjusting the flight attitude of the drone through the drone flight control system; The alarm signal is triggered synchronously by the audible and visual alarm device and sent to the pre-configured audible and visual alert device on the target for warning.

[0012] Beneficial Effects: This invention provides a real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition. The system utilizes a visual recognition module to acquire an optimal visual recognition model, enabling multi-target recognition and obtaining recognition results. A spatial positioning module performs target spatial positioning based on the recognition results and GNSS data, and combines this with a Kalman filter algorithm to track target trajectories and obtain real-time spatial positioning results. A 3D projection rendering module obtains the target's 3D position based on the recognition results, real-time spatial positioning results, and UAV attitude angle data, and uses instantiated rendering technology to achieve real-time 3D projection display based on the target's 3D position and type. This invention addresses the problems of low accuracy in multi-target joint recognition, the contradiction between positioning accuracy and real-time performance, and weak cross-scene adaptability in existing technologies through a design approach of "hardware adaptation + algorithm optimization + multi-technology fusion." It not only improves the recognition accuracy and efficiency of multiple targets in complex scenes but also enhances target positioning accuracy through multi-source positioning fusion. Finally, the low-latency 3D rendering engine configured in the 3D projection rendering module achieves accurate spatial visualization. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] This embodiment provides a real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition, such as... Figure 1 As shown, it includes a data acquisition module, a data processing module, a visual recognition module, a spatial positioning module, and a three-dimensional projection rendering module that realize data interaction connection through a 5G communication link (transmission rate ≥100Mbps and latency ≤20ms). The data acquisition module is used to acquire multi-target image data for UAV visual recognition, GNSS data for target localization in the multi-target image data, and UAV attitude angle data acquired by sensors. Specifically, the hardware configuration of the UAV includes: a 4K high-definition visible light camera (resolution 3840×2160, frame rate 25fps, focal length 8mm-25mm), a 10Hz refresh rate infrared thermal imager (detection band 8-14μm, resolution 640×512), and an IMU (sampling rate 100Hz, attitude angle error ≤0.1°). The UAV acquires and outputs 640×640 size images. The multi-target image data includes, but is not limited to, labeled images corresponding to different types of vehicles, human figures with different postures / ages / clothing, and different tonnage / water scenes in different environmental scenarios; and the labeled images include visible light images and infrared images. The data processing module is used to preprocess multi-target image data to obtain a preprocessed image; Specifically, the method for preprocessing multi-target image data is as follows: S100: The visible light image in the multi-target image data is filtered based on the Gaussian filtering algorithm to obtain the visible light filtered image; wherein the visible light image is filtered using a 5×5 Gaussian filter with a filter kernel of [1,4,6,4,1;4,16,24,16,4;6,24,36,24,6;4,16,24,16,4;1,4,6,4,1] / 256 to remove salt-and-pepper noise and achieve Gaussian filtering. S101: The intrinsic parameters of the camera used to capture visible light images on the UAV are obtained by Zhang Zhengyou calibration method, and the distortion correction of the visible light filtered image is realized according to the camera intrinsic parameters based on the fisheye lens distortion method, and the distortion correction image is obtained; in this embodiment, the distortion error after correction is ≤0.5 pixels; S102: Using the IMU sampling timestamp of the UAV attitude angle data acquired by the sensor as a reference, the distortion-corrected image and the infrared image are time-aligned with each other, and the time error is ≤10ms to achieve frame synchronization, thereby acquiring the pre-processed image. In this embodiment, the pre-processed image data (H.265 encoded format, bitrate 10Mbps) is sent to the visual recognition module through a preset data transmission module.

[0017] The visual recognition module is used to obtain the optimal visual recognition model based on the preprocessed image, based on the constructed lightweight target detection model, so as to achieve multi-target recognition and obtain recognition results; the recognition results include the target recognition type and its corresponding target bounding box; The lightweight target detection model constructed in this embodiment includes an input layer, a backbone network, a neck network, and a detection head; this embodiment adopts an improved lightweight target detection deep neural network model with ≤5M model parameters and an inference speed ≥30fps; The input layer is used to input the preprocessed image into the backbone network; The backbone network adopts The network reduces the number of convolutional layers by 50% to extract multi-scale target feature vectors; Furthermore, the target features of the target feature vector include contour features in the distortion-corrected image and target pixel features in the infrared image; The neck network adopts The network structure is designed to enhance cross-layer feature fusion, which is used to fuse target feature vectors at multiple scales and obtain fused feature vectors. The detection head is used to separate classification and regression tasks based on fused feature vectors, thereby achieving multi-target recognition to obtain the target recognition type and target bounding box corresponding to the detected target; the detection head adopts a decoupled head and the expression of the detection head is: P(detect)=σ(W X+b); where P(detect) represents the target detection probability; σ represents Activation function; W represents the weight matrix, with dimensions 1×(64×64×3), and the initial values ​​can be obtained through... The dataset was pre-trained and optimized iteratively using stochastic gradient descent during training, with a learning rate of 0.001. X represents the fused feature vector with dimensions of 64×64×3. b represents the bias term with an initial value of 0.1, updated synchronously with the weight matrix during training. The domain is (-∞, +∞), and the range is (0, 1). In this embodiment, model inference is performed using an edge computing chip (15 TOPS computing power), outputting the target category (vehicle / person / ship), bounding box coordinates (accuracy ≤ 1 pixel), and detection probability P(detect). Filtering: when P(detect) ≥ 0.85, it is determined to be a valid target, and the result is transmitted to the spatial positioning module.

[0018] Specifically, the method for obtaining the optimal visual recognition model based on the preprocessed image is as follows: S200: Divide the preprocessed image into a training set and a test set according to a preset ratio: Specifically, the dataset contains labeled images of vehicles (100,000 images, covering cars, trucks, and buses), people (80,000 images, showing different poses and clothing), and boats (50,000 images, showing different tonnages and water scenes). 30% of the data consists of complex scenes with occlusion, reflections, and low light. Training parameters: initial learning rate 0.001, cosine annealing with learning rate decay, batch size 16, 100 epochs, and loss function [missing information]. .

[0019] S201: Using the distortion-corrected image and infrared image as feature data and the target category as label data, train the lightweight target detection model based on the training set to obtain the trained lightweight target detection model. S202: Based on The loss function is used to confirm whether the output of the trained lightweight object detection model has converged based on the test set. If so, the lightweight object detection model trained at this time will be taken as the optimal visual recognition model; If not, the parameters of the trained lightweight object detection model are adaptively adjusted based on the backpropagation method, and step S201 is repeated until the weight parameters of the trained lightweight object detection model with converged output are confirmed to be the optimal weight parameters, and the lightweight object detection model is reconstructed to obtain the optimal visual recognition model.

[0020] The spatial positioning module is used to locate the target in space based on the identification results and GNSS data, and to track the target trajectory by combining the Kalman filter algorithm to obtain real-time spatial positioning results; The method for obtaining real-time spatial positioning results described in this embodiment specifically includes: S300: Based on the identification results and GNSS data, perform target spatial positioning to obtain the current target position with an error ≤ 1 meter; the expression for target spatial positioning is: X=(x- ) / f*Z,Y=(y- ) / f*Z,Z=f B / d In the formula: X represents the east coordinate of the target in the WGS84 coordinate system; Y represents the north coordinate of the target in the WGS84 coordinate system; Z represents the vertical coordinate in the WGS84 coordinate system; f represents the focal length of the camera in the UAV acquisition module; , The coordinates of the camera principal point obtained by Zhang Zhengyou's calibration method (calibration error ≤ 0.5 pixels) are given; B represents the baseline length in the multi-view geometry obtained from GNSS data (GNSS data: Beidou-3 + GPS dual-mode positioning, sampling rate 1Hz, positioning accuracy ≤ 1 meter, outputting UAV position coordinates (WGS84 coordinate system)). The baseline length is the UAV position distance between two adjacent frames of GNSS data, with a calculation accuracy ≤ 0.1 meters; d represents the parallax, calculated using the 5×5 window SAD algorithm, with a matching error ≤ 1 pixel; x, y represent the center coordinates of the target bounding box, in pixels. S301: Based on the Kalman filter algorithm, target trajectory tracking is achieved according to the current target position to obtain real-time spatial positioning results. In this embodiment, the Kalman filter algorithm is used to smooth the three-dimensional coordinates of 5 consecutive frames, reducing random errors. The coordinate fluctuation after filtering is ≤0.3 meters.

[0021] The 3D projection rendering module is used to obtain the target's 3D position based on the recognition results, real-time spatial positioning results, and UAV attitude angle data. The expression for obtaining the target's 3D position is: P=K[R|t] X_w In the formula: P represents the projected coordinates of the target on the image plane; X_w represents the three-dimensional spatial coordinate vector of the target and X_w=[X, Y, Z, 1]^T; T represents the transpose; t represents the translation vector determined by the difference between the GNSS data and the initial position of the UAV, with a dimension of 3×1; R represents the 3×3 orthogonal matrix obtained from the UAV attitude angle data, i.e., the rotation matrix; K represents the camera intrinsic parameter matrix and K=[ [f, 0, ], [0, f, [R|t] represents the extrinsic parameter matrix, where R is calculated from the IMU attitude angles, and t is the difference between the UAV's initial position and its current GNSS position; In this embodiment, instantiation rendering technology is used to achieve real-time 3D projection display based on the target's 3D position and type. The instantiation rendering process is as follows: A. Map Loading: Loads WGS84 coordinate system electronic maps with an accuracy ≥ 1:2000, supporting terrain elevation data (resolution ≤ 5 meters); B. Spatiotemporal Fusion: Aligns the target's 3D coordinates X_w with the map coordinates, with a time synchronization error ≤ 10ms; C. Visual Output: Using ES 3.2 renders and generates a 3D model of the target (vehicle / personnel / ship feature model), projects it to the corresponding location on the map, and achieves a rendering frame rate ≥ 20fps with an end-to-end latency ≤ 500ms. Based on The ES 3.2 interface optimizes the rendering process, using texture compression and instantiation rendering techniques to control 3D projection latency to within 500ms, while also supporting high-precision electronic map loading to ensure the accuracy of spatial visualization.

[0022] The system described in this embodiment also includes an AI-assisted decision-making module: The AI-assisted decision-making module is used to construct an auxiliary decision-making mechanism based on the acquired real-time spatial positioning results to achieve real-time tracking and response to the target. In this embodiment, data transmission adopts 5G NR communication, with an uplink rate of ≥100Mbps (transmitting images and recognition results, etc.), a downlink rate of ≥50Mbps (transmitting control commands), a packet loss rate of ≤0.1%, and supports packet loss retransmission (retransmission timeout of 100ms). And the aforementioned decision support mechanism is as follows: S400: Based on the sampling period, obtain the set of historical trajectory points and the corresponding first motion velocity features according to the historical trajectory data of the target; Simultaneously, based on the sampling period and the real-time spatial positioning results, a set of real-time trajectory points and corresponding second motion velocity features are obtained. S401: The density-based DBSCAN clustering algorithm is used to obtain the first trajectory point density of the corresponding historical trajectory point set and the second trajectory point density of the corresponding real-time trajectory point set; S403: Set a trajectory point density threshold based on the first trajectory point density; for example: trajectory point density ≤ 0.1 points / square meter; set a velocity change threshold based on the first motion velocity characteristic; for example: velocity change ≥ 10 m / s; and use the trajectory point density threshold, velocity change threshold, and preset deviation area (e.g., defined by polygon coordinates) as the target anomaly judgment conditions. S404: Based on the second trajectory point density, the second motion velocity characteristics, and the target's current location area, determine whether the target anomaly determination conditions are met; If any one of the following indicators is met: the second trajectory point density is less than or equal to the trajectory point density threshold, the velocity mutation value of the second motion velocity feature is greater than or equal to the velocity mutation threshold, or the target's current location area is outside the preset deviation area, then the target is determined to be in an abnormal mode and S405 is executed. Otherwise, the target is determined to be in normal mode; S405: Retrieve the exception data corresponding to the exception mode: Furthermore, the abnormal data includes the target ID, abnormality type, and real-time location coordinates; The monitoring terminal, drone flight control system, and audible and visual alarm devices are connected to the AI-assisted decision-making module via a multi-terminal linkage interface. The current target's status is monitored in real time via a monitoring terminal; Real-time target tracking is achieved by adjusting the flight attitude of the drone through the drone flight control system; An alarm signal is triggered synchronously by an audible and visual alarm device and sent to the target's pre-configured audible and visual alert device for warning. In this embodiment, data acquisition is performed: the target's historical trajectory for 5 minutes is stored (sampling interval of 1 second), and the velocity (≤30m / s) and acceleration (≤5m / s²) are calculated in real time. Anomaly detection: DBSCAN clustering algorithm. =5 meters, min_samples=3, determine sparse trajectory, sudden velocity change and regional deviation as abnormal; Linkage control: When an anomaly occurs, an early warning message is generated and sent to the monitoring terminal, UAV flight controller (adjust flight speed ≤15m / s to track the target) and audible and visual alarm device (triggering buzzer and light) within 100ms.

[0023] The beneficial effects of the system described in this embodiment are as follows: (1) Significantly improved multi-target recognition performance: The system described in this embodiment improves the recognition accuracy by more than 20% in occluded scenes, and the recognition accuracy in low light scenes increases from 60% to 88%, with a comprehensive recognition accuracy of ≥92%, meeting the needs of cross-scene monitoring of multiple types of targets; (2) Positioning and real-time balance optimization: By integrating GNSS and visual measurement, the positioning error is reduced from 3-5 meters in the existing technology to ≤1 meter. At the same time, through edge computing and 5G transmission, the end-to-end latency is reduced from more than 1000ms to ≤500ms, balancing accuracy and real-time performance. By integrating Beidou-3 + GPS dual-mode GNSS and multi-view geometric visual measurement, and through an adaptive weight allocation algorithm (weight 0.7 when GNSS signal is good, weight 0.3 when GNSS signal is blocked), the positioning error is ≤1 meter, and the positioning accuracy within 3 meters can still be maintained in the absence of GNSS signal. (3) Enhanced scene adaptability: It supports complex environments with light intensity of 0-100000 lux and visibility of ≥100 meters. It can work stably in urban shading, water reflection and low light scenes, and the scene coverage is improved by 40% compared with the existing system; (4) High flexibility in engineering applications: The modular design supports independent replacement of each module. For example, the visual recognition model can be replaced with YOLOv9-Lite, and the spatial positioning module can add LiDAR data fusion to adapt to different scenario requirements. (5) Improved decision support capabilities: Built-in AI-assisted decision-making mechanism with an anomaly identification accuracy of ≥95% and linkage response delay of ≤100ms, which can quickly trigger multi-terminal collaborative actions and improve emergency response efficiency.

[0024] The application scenarios of the system described in this implementation include the following: (a) Regulation of the low-altitude economy: In drone flight control areas, the system described in this embodiment can identify drones flying illegally (extended target types), personnel at ground take-off and landing points, and support vehicles. The system displays the target location and trajectory in three dimensions. When illegal flight is detected, the drone interception equipment is triggered to achieve low-altitude safety control. The system has been tested in a provincial low-altitude economic demonstration zone, and the accuracy rate of illegal identification is ≥96%.

[0025] (II) Smart City Security: In urban transportation hubs (such as railway station squares), the density of people (error in identifying the number of people ≤5%) and the traffic flow (statistical error ≤3%) are monitored. The congestion situation in the area is displayed by 3D projection. When the density of people exceeds 10 people / square meter or the traffic congestion exceeds 500 meters, the warning information is automatically pushed to the traffic command center to dispatch police forces to guide traffic. In the test, the congestion warning response time was ≤30 seconds.

[0026] (III) Emergency Command: At earthquake disaster sites, infrared thermal imagers are used to identify trapped personnel (with an identification rate of ≥89% in low-light and rubble-covered scenarios). Combined with the projection of personnel positions onto 3D terrain maps, this guides rescue teams to quickly locate the individuals. At the same time, it monitors the routes of rescue vehicles and boats to prevent secondary disasters. In a simulated earthquake rescue drill, the personnel location time was reduced from 30 minutes using traditional methods to 5 minutes.

[0027] (iv) Coastal Control: In nearshore waters, the system monitors the types of passing vessels (fishing boats, cargo ships, and illegal vessels with an accuracy rate of ≥93%) and their navigation trajectories. When a vessel is found to deviate from its designated route or illegally anchor, it triggers coastal radar tracking and linkage with law enforcement vessels. At the same time, a 3D projection displays the relative position of the vessel to the coastline to assist law enforcement decision-making. In a test in a coastal city, the efficiency of intercepting illegal vessels was improved by 50%.

[0028] The system described in this embodiment can achieve a multi-target cross-scene recognition accuracy of ≥92%, a positioning error of ≤1 meter, and a three-dimensional projection delay of ≤500ms, meeting the high requirements of fields such as low-altitude economic supervision and smart city security.

[0029] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition, characterized in that, It includes a data acquisition module, a data processing module, a visual recognition module, a spatial positioning module, and a 3D projection rendering module connected via a communication link; The data acquisition module is used to acquire multi-target image data for UAV visual recognition, GNSS data for target localization in the multi-target image data, and UAV attitude angle data acquired by sensors. Furthermore, the multi-target image data includes, but is not limited to, labeled images of different types of vehicles, people in different postures / ages / clothing, and different tonnage / water scenes in different environmental scenarios; and the labeled images include visible light images and infrared images. The data processing module is used to preprocess multi-target image data to obtain a preprocessed image; The visual recognition module is used to obtain the optimal visual recognition model based on the preprocessed image, based on the constructed lightweight target detection model, and then to achieve multi-target recognition to obtain recognition results; The recognition result includes the target recognition type and its corresponding target bounding box; The spatial positioning module is used to locate the target in space based on the identification results and GNSS data, and to track the target trajectory by combining the Kalman filter algorithm to obtain real-time spatial positioning results; The 3D projection rendering module is used to obtain the 3D position of the target based on the recognition results, real-time spatial positioning results, and UAV attitude angle data, and uses instantiation rendering technology to realize real-time 3D projection display based on the 3D position and target type of the target.

2. The real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to claim 1, characterized in that, The method for preprocessing multi-target image data is as follows: S100: Based on the Gaussian filtering algorithm, the visible light image in the multi-target image data is filtered to obtain the visible light filtered image; S101: Obtain the intrinsic parameters of the camera used to capture visible light images on the UAV by Zhang Zhengyou calibration method, and perform distortion correction on the visible light filtered image based on the camera intrinsic parameters using the fisheye lens distortion method, and obtain the distortion-corrected image. S102: Using the IMU sampling timestamp of the UAV attitude angle data acquired by the sensor as a reference, the distortion-corrected image and the infrared image are time-aligned to obtain a pre-processed image.

3. A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to claim 2, characterized in that, The constructed lightweight target detection model includes an input layer, a backbone network, a neck network, and a detection head; The input layer is used to input the preprocessed image into the backbone network; The backbone network adopts Networks are used to extract target feature vectors at multiple scales; Furthermore, the target features of the target feature vector include contour features in the distortion-corrected image and target pixel features in the infrared image; The neck network adopts A network structure used to fuse multi-scale target feature vectors to obtain a fused feature vector. The detection head is used to obtain the target recognition type and target bounding box corresponding to the detected target based on the fused feature vector; The expression for the detection head is: P(detect) = σ(W) X+b); where P(detect) represents the target detection probability; σ represents Activation function; W represents the weight matrix; X represents the fused feature vector; b represents the bias term.

4. A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to claim 3, characterized in that, The method for obtaining the optimal visual recognition model based on preprocessed images specifically includes the following steps: S200: Divide the preprocessed image into a training set and a test set according to a preset ratio: S201: Using the distortion-corrected image and infrared image as feature data and the target category as label data, train the lightweight target detection model based on the training set to obtain the trained lightweight target detection model. S202: Based on The loss function is used to confirm whether the output of the trained lightweight object detection model has converged based on the test set. If so, the lightweight object detection model trained at this time will be taken as the optimal visual recognition model; If not, the parameters of the trained lightweight object detection model are adaptively adjusted based on the backpropagation method, and step S201 is repeated until the weight parameters of the trained lightweight object detection model with converged output are confirmed to be the optimal weight parameters, and the lightweight object detection model is reconstructed to obtain the optimal visual recognition model.

5. A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to claim 4, characterized in that, The method for obtaining real-time spatial positioning results specifically includes: S300: Based on the identification results and GNSS data, perform target spatial positioning to obtain the current target location; And the expression for target spatial localization is: X=(x- ) / f*Z,Y=(y- ) / f*Z,Z=f B / d In the formula: X represents the east coordinate of the target in the WGS84 coordinate system; Y represents the north coordinate of the target in the WGS84 coordinate system; Z represents the vertical coordinate in the WGS84 coordinate system; f represents the focal length of the camera in the UAV acquisition module; , The coordinates of the camera principal point obtained by Zhang Zhengyou's calibration method are given; B represents the baseline length in the multi-view geometry obtained from GNSS data; d represents the parallax; and x and y represent the center coordinates of the target bounding box. S301: Based on the Kalman filter algorithm, target trajectory tracking is achieved according to the current target position to obtain real-time spatial positioning results.

6. A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to claim 5, characterized in that, The expression for obtaining the target's three-dimensional position based on the recognition results, real-time spatial positioning results, and UAV attitude angle data is as follows: P=K[R|t] X_w In the formula: P represents the projected coordinates of the target on the image plane; X_w represents the three-dimensional spatial coordinate vector of the target and X_w=[X,Y, Z, 1]^T; T represents the transpose; t represents the translation vector determined by the difference between the GNSS data and the initial position of the UAV; R represents the 3×3 orthogonal matrix obtained from the UAV attitude angle data, i.e., the rotation matrix; K represents the camera intrinsic parameter matrix and K=[ [f,0, ], [0, f, ], [0, 0, 1] ].

7. A real-time 3D projection system for vehicles, personnel, and vessels based on UAV visual recognition according to claim 6, characterized in that, It also includes an AI-assisted decision-making module: The AI-assisted decision-making module is used to construct an auxiliary decision-making mechanism based on the acquired real-time spatial positioning results, so as to achieve real-time tracking and response to the target; And the aforementioned decision support mechanism is as follows: S400: Based on the sampling period, obtain the set of historical trajectory points and the corresponding first motion velocity features according to the historical trajectory data of the target; Simultaneously, based on the sampling period and the real-time spatial positioning results, a set of real-time trajectory points and corresponding second motion velocity features are obtained. S401: The density-based DBSCAN clustering algorithm is used to obtain the first trajectory point density of the corresponding historical trajectory point set and the second trajectory point density of the corresponding real-time trajectory point set; S402: Set a trajectory point density threshold based on the first trajectory point density, and set a velocity change threshold based on the first motion velocity characteristic; and use the trajectory point density threshold, velocity change threshold, and preset deviation area as target anomaly determination conditions; S403: Based on the second trajectory point density, the second motion velocity characteristics, and the target's current location area, determine whether the target anomaly determination conditions are met; If any one of the following indicators is met: the second trajectory point density is less than or equal to the trajectory point density threshold, the velocity mutation value of the second motion velocity feature is greater than or equal to the velocity mutation threshold, or the target's current location area is outside the preset deviation area, then the target is determined to be in an abnormal mode and S404 is executed. Otherwise, the target is determined to be in normal mode; S404: Retrieve the exception data corresponding to the exception mode: Furthermore, the abnormal data includes the target ID, abnormality type, and real-time location coordinates; The monitoring terminal, drone flight control system, and audible and visual alarm devices are connected to the AI-assisted decision-making module via a multi-terminal linkage interface. The current target's status is monitored in real time via a monitoring terminal; Real-time target tracking is achieved by adjusting the flight attitude of the drone through the drone flight control system; The alarm signal is triggered synchronously by the audible and visual alarm device and sent to the pre-configured audible and visual alert device on the target for warning.

Images