Unmanned aerial vehicle optical image unexploded ordnance detection method and system based on pseudo label generation

Abstract

The invention provides an unmanned aerial vehicle optical image unexploded ordnance detection method and system based on pseudo label generation, and the system comprises a ground station processing module, an unmanned aerial vehicle and ground station communication module, an airborne calculation module, and an unmanned aerial vehicle platform. The operation route is transmitted to the airborne calculation module through the unmanned aerial vehicle and ground station communication module; the airborne calculation module uploads an operation route to the unmanned aerial vehicle platform, controls the unmanned aerial vehicle to execute a first search task, receives images acquired by the unmanned aerial vehicle in real time, and transmits the images acquired in real time to the ground station processing module through the unmanned aerial vehicle and ground station communication module; according to the invention, real-time target identification can be carried out on the image shot by the unmanned aerial vehicle, the target latitude and longitude are calculated, and the target latitude and longitude and the image are filed.

Description

Technical Field

[0001] This invention belongs to the field of unmanned surface unexploded ordnance detection technology, specifically, it relates to a method and system for unexploded ordnance detection based on UAV optical images generated by pseudo-tags. Background Technology

[0002] Existing drone platforms possess significant advantages such as flexible flight altitude, diverse perspectives, and large imaging range, but they also bring a series of challenges. The scale of the target in the image changes drastically, and its posture exhibits complex and diverse states. At the same time, they are also affected by many uncertain factors such as lighting, weather, and background interference.

[0003] For example, patent document CN117761795A discloses a UAV-borne multi-system unexploded ordnance detection system and method, including: a UAV platform, a terrain adaptive module, an airborne detection module, an onboard identification computer, and a ground control terminal; the onboard identification computer communicates with the ground control terminal, receives control commands in real time, and sends UAV flight information, multispectral image data, and electromagnetic induction data and magnetic detection data collected by the airborne detection module during the detection process to the ground control terminal; the ground control terminal displays information on the remote control of the UAV and the status of the unexploded ordnance. This invention adopts a multi-detection system fusion detection scheme to solve the problems of insufficient detection capability of existing single unexploded ordnance detection systems, low detection rate of deeply buried ordnance, and high false alarm rate.

[0004] For example, patent document CN117516480A discloses a precise unexploded ordnance (UOD) pose detection system and method based on vehicle-machine collaboration. This system utilizes a CNN-based UAV and ground robot collaborative pose estimation technology to acquire precise UOD pose information. The UAV first conducts remote reconnaissance, actively searching for the target through global path planning. After detecting the target, it sends its location information to a ground station. The ground station guides the ground robot to the vicinity of the target location. The target's point cloud data is obtained by combining images acquired by the ground robot's built-in depth camera with third-view images provided by the UAV for pose estimation. Through real-time data communication, a three-in-one data sharing and joint control system involving the ground station, UAV, and ground robot is achieved, ensuring the stability and comprehensive visualization of UOD disposal operations.

[0005] However, traditional deep learning object detection methods heavily rely on large-scale, accurately labeled data as supervision information. However, obtaining a large number of high-quality labeled samples from a drone's perspective requires significant manpower and time, resulting in high labeling costs. Furthermore, the types of objects in the environment are constantly evolving, and their shapes are continuously changing, making it difficult for existing models to adapt to new situations. Continuous labeling updates and model training are essential.

[0006] Unexploded ordnance (UFO) detection missions often operate in complex and dangerous environments, with extremely high time requirements, necessitating the rapid discovery and location of potential threat targets within a limited timeframe. In unknown environments, there may be types of munitions not covered in the dataset, making it difficult for traditional deep learning target detection algorithms to generalize well in real-world combat scenarios.

[0007] In view of the above problems, there is an urgent need to design a process that can quickly understand the environmental situation, automatically complete target labeling, and iterate to generate an unexploded ordnance detection model that is adapted to the current environment in a short period of time. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the present invention aims to provide a UAV optical image unexploded ordnance detection system based on pseudo-tag generation, comprising: a ground station processing module, a UAV-ground station communication module, an airborne computing module, and a UAV platform; the ground station processing module manually sets the operation area and generates an initial search operation route, which is transmitted to the airborne computing module via the UAV-ground station communication module; the airborne computing module uploads the operation route to the UAV platform, controls the UAV to perform the initial search mission, and receives images acquired in real time by the UAV; the airborne computing module transmits the real-time acquired images to the ground station processing module via the UAV-ground station communication module.

[0009] The ground station processing module processes the image into an orthophoto map and calculates the target area of ​​interest within the operational area. It then creates a flight path containing only the target area of ​​interest and transmits this flight path to the onboard computing module via the UAV-Ground Station communication module. The onboard computing module uploads the flight path to the UAV platform, controls the UAV to perform the photography task, and receives images acquired in real time by the UAV. The onboard computing module then transmits the acquired images to the ground station processing module via the UAV-Ground Station communication module. The ground station processing module uses a visual language model to coarsely annotate the captured images and fuses the coarsely annotated data with existing data to train a deep learning target detection model for the current task. This target detection model is then uploaded to the onboard computing module via the UAV-Ground Station communication module. The onboard computing module loads the target detection model, processes the real-time video stream transmitted from the UAV to the onboard computing module, draws bounding boxes around the locations of unexploded ordnance targets in the video stream, encodes the images beyond the bounding boxes, and transmits the encoded video stream, along with the tilt angle of the UAV's optical payload and the UAV's GPS coordinates, to the ground station processing module via the UAV-Ground Station communication module. The ground station processing module receives the processing results, displays them on the user interface, and archives them.

[0010] The ground station processing module includes: a flight path planning unit, an orthophoto generation unit, an image pseudo-labeling unit, a YOLOv11 deep learning target detection model training unit, and a unit for displaying and archiving real-time UAV detection results.

[0011] The flight path planning unit includes: initial search flight path planning and detailed image capture flight path planning for the area of ​​interest. Initial search flight path planning automatically plans the flight path using an S-shaped parallel scanning method after setting the search area, and controls the flight path spacing based on the camera's field of view and a preset overlap rate to ensure sufficient overlap between adjacent aerial images. The area of ​​interest image capture flight path planning first scans the orthophoto image into regions, then uses a pre-trained convolutional neural network to determine the area of ​​interest according to a set threshold, setting all areas of interest as waypoints that the UAV must pass through. The UAV's flight path planning problem is modeled as a traveling salesman problem, solved using a genetic algorithm. Through chromosome encoding, selection, crossover, and mutation operations, iterative optimization yields an approximately optimal flight path covering all task points. When the UAV flies to each waypoint, it captures images of the area of ​​interest according to preset values, using different tilt angles of the optical payload, different circular flight paths, and a specified frequency. No image capture is performed when the UAV flies between waypoints. The unit for generating orthophotos includes: Component M101: Performs lens distortion and radiation correction on the acquired images; Component M102: Implements sparse 3D reconstruction based on feature matching and solves the camera exterior orientation by adjusting the binding. Component M103: A digital surface model is generated using dense stereo matching based on optimized camera parameters; Component M104: Utilizes digital surface models involving both inside and outside the camera to perform orthorectification on images, as well as color balancing and automatic stitching, to output georeferenced digital orthophotos.

[0012] The image pseudo-labeling unit includes: using a visual language model to process the image output of the photographed flight path of the region of interest to generate pseudo-label boxes, and using a convolutional neural network classification model to determine whether the target in the label box is an unexploded ordnance; Furthermore, the image is labeled with pseudo-label units, including: Component M201: Scales the image to be detected so that its length and width meet the multiples of the block size set in the visual language model and are close to the original size, and extracts the multi-scale feature representation of the image through the visual encoder; Component M202: Embeds and vectorizes the prompt words; Component M203: Enables feature fusion of images and prompts in cross-modal interaction components; Component M204: Outputs predicted statements in the output layer of the visual language model; Component M205: Post-processes the predicted statement to generate ground truth values ​​and class labels for target boxes used to train the target detection model. It supports manually setting confidence thresholds and manually modifying pseudo-labels. The unit for training the YOLOv11 deep learning object detection model includes: Component M301: Counts the number of samples with pseudo-labels and randomly extracts a specified proportion of sample data from the database and uses it together with the pseudo-label data as the training dataset. Component M302: Sets a higher probability for executing data augmentation strategies for single images on pseudo-labeled data; Component M303: The fused dataset is divided into training and validation sets proportionally, fine-tuned on the YOLOv11 model trained on a large-scale general drone dataset, and the model with the highest average detection accuracy in the validation set is saved. Component M304: It is manually determined whether the model generated in the previous step needs to be used to re-label the images of the flight path output of the region of interest, and then retrain the YOLOv11 model. Component M305: Uploads the trained deep learning model to the recording and computing module and archives the training logs and the model; The unit for displaying and archiving real-time detection results from UAVs includes: real-time display of video streams, calculation of target GPS coordinates, and data archiving; Specifically, the unit displays and archives the real-time detection results from the UAV. It uses the GStreamer framework to decode and render the video stream sent by the onboard computing module in real time, ensuring low-latency display at different resolutions and bitrates. The coordinates of the target at image points are converted into direction vectors in the camera coordinate system using the camera intrinsic parameter matrix. Then, the direction vectors in the camera coordinate system are converted to geographic coordinate vectors using the optical payload gimbal attitude matrix and the UAV's GPS coordinates. The intersection of the vectors with the digital surface model is used to obtain the actual GPS coordinates of the detected target. The video stream and GPS data stream are aligned according to their timestamp information, and the aligned data is stored in the database for archiving. The optical payload gimbal attitude includes pitch angle, roll angle, and yaw angle.

[0013] The UAV-Ground Station Communication Module includes: a high-bandwidth wireless module that can be modified to switch frequency bands to communicate with the ground station processing module, supporting TCP, IP, or UDP protocols. Video streams use RTP or RTSP protocols based on GStreamer for low-latency transmission, while operation commands, status feedback, and deep learning models use TCP or HTTPS to ensure reliable transmission; and a UART and USB interface for communication with the onboard computing module. Control commands are transmitted bidirectionally via the UART interface, while video streams and deep learning models are transmitted at high speed via the USB interface. The airborne computing module includes: a UART and USB interface for transmitting video, deep learning models, and control commands to the UAV and ground station communication module; a MIPI CSI data bus for receiving video streams transmitted from the UAV; an RS-422 interface for sending flight control commands and camera operation commands to the communication module; and the ability to receive operation commands from the ground station. The airborne computing module also incorporates an embedded GPU for deep learning inference and a dedicated video hardware codec chip to reduce the computational burden on the CPU. Furthermore, the airborne computing module includes: controlling the UAV's automatic flight and image acquisition; the airborne computing module acts as a relay to transmit images acquired by the UAV; during the real-time unexploded ordnance detection phase, it loads a deep learning target detection model to decode the data transmitted by the UAV, detects the target position and draws prediction boxes on the video frames, encodes and compresses the processed video stream and transmits it to the UAV-ground station communication module; and it transmits time-stamped optical payload gimbal attitude information and UAV GPS information to the UAV-ground station communication module. The drone platform specifically refers to an optical payload with a built-in video hardware encoding module that compresses the acquired raw images into H.264 or H.265 video streams in real time; the drone receives flight path configuration items to complete automatic operations; the drone platform has built-in GPS and IMU sensors to record the drone's GPS information and the optical payload's gimbal attitude information in real time; the optical payload supports multiple shooting modes, including timed shooting, continuous shooting, and area scanning; and the video stream and sensor data are time-stamped synchronously.

[0014] This invention proposes a method for unexploded ordnance detection based on UAV optical images generated by pseudo-tags, and employs a UAV optical image unexploded ordnance detection system based on pseudo-tags, including: Step S1: Identify the region of interest and acquire optical data; Step S2: After completing the optical data acquisition of the area of ​​interest, the ground station processing module executes pseudo-labels and trains the YOLOv11 model; Step S3: After completing the YOLOv11 model training, perform a real-time unexploded ordnance optical target detection task and summarize and archive the detection results.

[0015] Step S1 includes: Step S101: Generate an S-shaped parallel flight path file according to the manually planned search range and the preset overlap rate of aerial images, and upload the flight path file to the UAV to perform the aerial photography task; Step S102: Perform geometric correction and projection processing on the aerial images collected by the UAV to generate orthophotos with georeferenced information; Step S103: Use a pre-trained convolutional neural network to divide the orthophoto into regions by sliding window to determine whether there is a region of interest, and save the GPS coordinates of the region of interest; Step S104: Use a genetic algorithm to find the solution to the optimal trajectory planning problem for the waypoint set consisting of all regions of interest, generate a route file that takes pictures around the regions of interest and flies quickly between the regions of interest, and upload the route file to the UAV to perform the data acquisition task; Step S105: The UAV completes the data acquisition task and transmits the acquired data to the ground station processing module.

[0016] Step S2 includes: Step S201: Scale the optical image of the region of interest with the principle that the length and width meet the multiples of the block size set in the visual language model and the scaled image maintains the minimum deviation from the original resolution. Extract the multi-scale feature representation of the image through the visual encoder. Step S201: Embedded and vectorized the prompt words, fused them with the features extracted by the visual encoder, and output the predicted sentence in the output layer of the visual-language model; Step S202: Post-process the predicted statement to generate pseudo-labeled boxes and category labels for training the object detection model; Step S203: Crop the image according to the coordinates of the generated pseudo-label boxes, and use a convolutional neural network classification model trained in the field of unexploded ordnance to determine whether the target in the cropped image is an unexploded ordnance target, and retain pseudo-label boxes with a confidence threshold. Step S204: Determine whether the pseudo-annotation box needs to be manually modified according to requirements; Step S205: Count the number of samples annotated with pseudo-label boxes, randomly extract a predetermined proportion of sample data from the database and use it together with the pseudo-label box data as the training dataset, and divide the dataset into training set and validation set according to the proportion. Step S206: In the data augmentation stage, the dataset is divided into training set and validation set proportionally; Step S207: Train the YOLOv11 model. The pre-trained model is a YOLOv11 model trained on a large-scale UAV dataset. The model with the highest average detection accuracy in the validation set is saved. Step S208: Determine whether it is necessary to use the model generated in the previous step to re-label the images of the fine-grained photographic flight path output of the region of interest and retrain the YOLOv11 model. If not, update the pseudo-labels and repeat step S205. Step S209: Upload the trained YOLOv11 model to the recording and calculation module, and archive the training logs and the model.

[0017] Step S3 includes: Step S301: Load the YOLOv11 model onto the airborne computing module and set it to inference mode; Step S302: The airborne computing module decodes the data transmitted by the UAV, detects the target position and draws a prediction box on the video frame. The processed video stream is encoded and compressed using the Gstreamer framework and then transmitted to the ground station processing module. At the same time, the UAV's GPS information and optical payload attitude information are also transmitted. Step S303: The ground station processing module decodes the video stream containing the predicted target bounding box, aligns the video stream timestamp with the UAV GPS information and the optical payload attitude information timestamp; Step S304: The ground station processing module calculates the actual GPS coordinates of the target based on the optical payload intrinsic and extrinsic parameters matrix and the digital surface model; Step S305: The ground station processing module displays the real-time video stream containing the predicted target bounding box and the target's coordinate information, and archives the target's latitude and longitude and the image.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention can quickly and automatically complete the task of detecting unexploded ordnance on the ground based on optical images in unknown operating environments.

[0019] 2. This invention can autonomously analyze areas in unknown environments where unexploded ordnance exists and automatically formulate a scheme for collecting optical data of unexploded ordnance.

[0020] 3. This invention can complete the annotation of newly acquired unexploded ordnance optical datasets in scenarios with no or minimal human assistance, greatly reducing the time and labor costs required for supervised learning. Attached Figure Description

[0021] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the system composition of the UAV optical image unexploded ordnance detection system based on pseudo-tag generation proposed in this invention; Figure 2 This is a schematic diagram of the flight path planning process for the unexploded ordnance detection system for UAV optical images based on pseudo-tag generation proposed in this invention. Figure 3 This is a schematic diagram of the orthophoto generation process of the UAV optical image unexploded ordnance detection system based on pseudo-tag generation proposed in this invention. Figure 4 This is a schematic diagram of the pseudo-tag labeling process for the unexploded ordnance detection system based on pseudo-tag generation for UAV optical images proposed in this invention; Figure 5 This is a schematic diagram of the model training process for the unexploded ordnance detection system based on pseudo-label generation of UAV optical images proposed in this invention; Figure 6 This is a schematic diagram of the real-time target detection process of the airborne computing module of the unexploded ordnance detection system for UAV optical images based on pseudo-tag generation proposed in this invention. Figure 7 This is a schematic diagram illustrating the real-time display and archiving process of the unexploded ordnance detection system based on pseudo-tag generation for UAV optical images proposed in this invention. Detailed Implementation

[0022] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0023] This invention proposes a UAV optical image unexploded ordnance detection system based on pseudo-tag generation, comprising: a ground station processing module, a UAV-ground station communication module, an airborne computing module, and a UAV platform. The interaction relationships between the modules are shown in the attached figure. Figure 1 As shown.

[0024] The ground station processing module manually sets the operation area and generates the initial search operation route. The operation route is transmitted to the airborne computing module through the communication module between the UAV and the ground station. The airborne computing module uploads the operation route to the UAV platform, controls the UAV to perform the initial search mission, and receives images collected in real time by the UAV. The airborne computing module transmits the real-time collected images to the ground station processing module through the communication module between the UAV and the ground station. Furthermore, the ground station processing module processes the image into an orthophoto map and calculates the target area of ​​interest within the operational area. It then creates a photographing flight path containing only the target area of ​​interest and transmits this flight path to the onboard computing module via the UAV-Ground Station communication module. The onboard computing module uploads the operational flight path to the UAV platform, controls the UAV to perform the photographing task, and receives images acquired in real-time by the UAV. The onboard computing module then transmits the captured images to the ground station processing module via the UAV-Ground Station communication module. The ground station processing module uses a visual language model to coarsely annotate the photographed images, fuses the coarsely annotated data with existing database data to train a deep learning target detection model for the current task, and then uploads the target detection model to the onboard computing module via the UAV-Ground Station communication module. The onboard computing module loads the target detection model, processes the real-time video stream transmitted from the UAV to the onboard computing module, draws bounding boxes around the locations of unexploded ordnance targets in the video stream, encodes the images beyond the bounding boxes, and transmits the encoded video stream, along with the tilt angle of the UAV's optical payload and the UAV's GPS coordinates, to the ground station processing module via the UAV-Ground Station communication module. The ground station processing module receives the processing results, displays them on the user interface, and archives them.

[0025] The ground station processing module includes: a flight path planning unit, an orthophoto generation unit, an image pseudo-labeling unit, a YOLOv11 deep learning target detection model training unit, and a unit for displaying and archiving real-time UAV detection results.

[0026] Among them, as attached Figure 2 As shown, the flight path planning unit includes: initial search flight path planning and detailed image capture flight path planning for the area of ​​interest. The initial search flight path planning automatically plans the flight path according to the S-shaped parallel scanning method after setting the search area, and controls the flight path spacing according to the camera field of view and the preset overlap rate to ensure that adjacent aerial images have sufficient overlap. The detailed image capture flight path planning for the area of ​​interest first scans the orthophoto image into regions, and uses a pre-trained convolutional neural network to determine the area of ​​interest according to the set threshold. All areas of interest are set as waypoints that the UAV must pass through. The UAV's flight path planning problem is modeled as a traveling salesman problem and solved using a genetic algorithm. Through chromosome encoding, selection, crossover and mutation operations, iterative optimization is performed to obtain an approximately optimal flight path covering all task points. When the UAV flies to each flight waypoint, it captures images of the area of ​​interest according to preset values ​​with different tilt angles of optical payload, different circular flight paths and specified frequencies. The UAV does not capture images when flying between waypoints.

[0027] Generate orthophoto units, as shown in the attached figure. Figure 3 As shown, it includes: Component M101: Performs lens distortion and radiation correction on the acquired images; Component M102: Implements sparse 3D reconstruction based on feature matching and solves the camera exterior orientation by adjusting the binding. Component M103: A digital surface model is generated using dense stereo matching based on optimized camera parameters; Component M104: Utilizes digital surface models involving both inside and outside the camera to perform orthorectification on images, as well as color balancing and automatic stitching, to output georeferenced digital orthophotos.

[0028] The image pseudo-labeling unit includes: using a visual language model to process the image output of the photographic flight path of the region of interest to generate pseudo-label boxes, and using a convolutional neural network classification model trained in the field of unexploded ordnance to determine whether the target in the label box is an unexploded ordnance.

[0029] Furthermore, the image is labeled with pseudo-label units, including: Component M201: Scales the image to be detected so that its length and width meet the multiples of the block size set in the visual language model and are close to the original size, and extracts the multi-scale feature representation of the image through the visual encoder; Component M202: Embeds and vectorizes the prompt words; Component M203: Enables feature fusion of images and prompts in cross-modal interaction components; Component M204: Outputs predicted statements in the output layer of the visual language model; Component M205: Post-processes the predicted statement to generate ground truth values ​​and class labels for target boxes that can be used to train the target detection model. It supports manually setting the confidence threshold and manually modifying the pseudo-labels.

[0030] Training YOLOv11 deep learning object detection model units includes: Component M301: Counts the number of samples with pseudo-labels and randomly extracts a specified proportion of sample data from the database and uses it together with the pseudo-label data as the training dataset. Component M302: Sets a higher probability for executing data augmentation strategies for single images on pseudo-labeled data; Component M303: The fused dataset is divided into training and validation sets proportionally, fine-tuned on the YOLOv11 model trained on a large-scale general drone dataset, and the model with the highest average detection accuracy in the validation set is saved. Component M304: It is manually determined whether the model generated in the previous step needs to be used to re-label the images of the flight path output of the region of interest, and then retrain the YOLOv11 model. Component M305: Uploads the trained deep learning model to the recording and computing module and archives the training logs and the model.

[0031] The unit for displaying and archiving real-time detection results from UAVs includes: real-time display of video streams, calculation of target GPS coordinates, and data archiving; Specifically, the unit displays and archives the real-time detection results from the UAV. It uses the GStreamer framework to decode and render the video stream sent by the onboard computing module in real time, ensuring low-latency display at different resolutions and bitrates. The coordinates of the target at image points are converted into direction vectors in the camera coordinate system using the camera intrinsic parameter matrix. Then, the direction vectors in the camera coordinate system are converted to geographic coordinate vectors using the optical payload gimbal attitude matrix and the UAV's GPS coordinates. The intersection of the vectors with the digital surface model is used to obtain the actual GPS coordinates of the detected target. The video stream and GPS data stream are aligned according to their timestamp information, and the aligned data is stored in the database for archiving. The optical payload gimbal attitude includes pitch angle, roll angle, and yaw angle.

[0032] The UAV-Ground Station Communication Module includes: a high-bandwidth wireless module that uses a modifiable frequency band to communicate with the ground station processing module, supporting TCP / IP or UDP protocols. The video stream uses the RTP / RTSP protocol based on GStreamer to achieve low-latency transmission, while operation commands, status feedback, and deep learning models use TCP or HTTPS to ensure reliable transmission; and a UART and USB interface to communicate with the onboard computing module. Control commands are transmitted bidirectionally via the UART interface, while video streams and deep learning models are transmitted at high speed via the USB interface.

[0033] The airborne computing module includes: a UART and USB interface for transmitting video, deep learning models, and control commands to the UAV and ground station communication module; a MIPI CSI data bus for receiving video streams transmitted from the UAV; an RS-422 interface for sending flight control commands and camera operation commands to the communication module; and the ability to receive operation commands from the ground station. The airborne computing module also incorporates an embedded GPU for deep learning inference and an independent video hardware codec chip to reduce the computational burden on the CPU.

[0034] Furthermore, the airborne computing module includes: controlling the UAV's automatic flight and image acquisition; the airborne computing module acts as a relay to transmit images acquired by the UAV; during the real-time unexploded ordnance detection phase, it loads a deep learning target detection model to decode the data transmitted by the UAV, detects the target position and draws a prediction box on the video frame, encodes and compresses the processed video stream and transmits it to the UAV-ground station communication module; and it transmits time-stamped optical payload gimbal attitude information and UAV GPS information to the UAV-ground station communication module.

[0035] The drone platform specifically refers to an optical payload with a built-in video hardware encoding module that compresses the acquired raw images into H.264 / H.265 video streams in real time; the drone can receive flight path configuration items to complete automatic operations and has autonomous obstacle avoidance capabilities; it has built-in sensors such as GPS and IMU to record the drone's GPS information and the optical payload's gimbal attitude information in real time; the optical payload supports multiple shooting modes, including timed shooting, continuous shooting, and area scanning; and the video stream and sensor data are synchronized and time-stamped.

[0036] This invention proposes a method for unexploded ordnance detection based on UAV optical images generated by pseudo-tags, and employs a UAV optical image unexploded ordnance detection system based on pseudo-tags, including: Step S1: First, identify the region of interest and acquire optical data; Step S1 includes: Step S101: Generate an S-shaped parallel flight path file according to the manually planned search range and the preset overlap rate of aerial images, and upload the flight path file to the UAV to perform the aerial photography task; Step S102: Perform geometric correction and projection processing on the aerial images collected by the UAV to generate orthophotos with georeferenced information; Step S103: Use a pre-trained convolutional neural network (such as a ResNet+RPN network trained using all unexploded ordnance data in the database) to divide the orthophoto into regions by sliding window to determine whether there is a region of interest, and save the GPS coordinates of the region of interest. Step S104: Use a genetic algorithm to find the solution to the optimal trajectory planning problem for the waypoint set consisting of all regions of interest, generate a route file that takes pictures around the regions of interest and flies quickly between the regions of interest, and upload the route file to the UAV to perform the data acquisition task; Step S105: The UAV completes the data acquisition task and transmits the acquired data to the ground station processing module; Step S2: After completing the optical data acquisition of the area of ​​interest, the ground station processing module executes pseudo-labels and trains the YOLOv11 model; Among them, as attached Figure 4 and 5 As shown, step S2 includes: Step S201: Scale the optical image of the region of interest with the principle that the length and width meet the multiples of the block size set in the visual language model and the scaled image maintains the minimum deviation from the original resolution. Extract the multi-scale feature representation of the image through the visual encoder. Step S201: Embedded and vectorized the prompt words, fused them with the features extracted by the visual encoder, and output the predicted sentence in the output layer of the visual-language model; Step S202: Post-process the predicted statement to generate pseudo-labeled boxes and category labels that can be used to train the object detection model; Step S203: Crop the image according to the coordinates of the generated pseudo-label boxes, and use a convolutional neural network classification model trained in the field of unexploded ordnance to determine whether the target in the cropped image is an unexploded ordnance target, and retain pseudo-label boxes with a confidence threshold. Step S204: Determine whether the pseudo-annotation box needs to be manually modified according to requirements; Step S205: Count the number of samples annotated with pseudo-label boxes, randomly extract a predetermined proportion of sample data from the database and use it together with the pseudo-label box data as the training dataset, and divide the dataset into training set and validation set according to the proportion. Step S206: In the data augmentation stage, the dataset is divided into training and validation sets proportionally to increase the probability of data augmentation for samples containing pseudo-labels. Step S207: Train the YOLOv11 model. The pre-trained model is a YOLOv11 model trained on a large-scale UAV dataset. The model with the highest average detection accuracy in the validation set is saved. Step S208: Determine whether it is necessary to use the model generated in the previous step to re-label the images of the fine-grained photographic flight path output of the region of interest and retrain the YOLOv11 model. If not, update the pseudo-labels and repeat step S205. Step S209: Upload the trained YOLOv11 model to the recording and calculation module, and archive the training logs and the model.

[0037] Step S3: After completing the YOLOv11 model training, a real-time unexploded ordnance optical target detection task can be performed and the detection results can be summarized and archived. Among them, as attached Figure 6 and attached Figure 7 As shown, step S3 includes: Step S301: Load the YOLOv11 model onto the airborne computing module and set it to inference mode; Step S302: The airborne computing module decodes the data transmitted by the UAV, detects the target position and draws a prediction box on the video frame. The processed video stream is encoded and compressed using the Gstreamer framework and then transmitted to the ground station processing module. At the same time, the UAV's GPS information and optical payload attitude information are also transmitted. Step S303: The ground station processing module decodes the video stream containing the predicted target bounding box, aligns the video stream timestamp with the UAV GPS information and the optical payload attitude information timestamp; Step S304: The ground station processing module calculates the actual GPS coordinates of the target based on the optical payload intrinsic and extrinsic parameters matrix and the digital surface model; Step S305: The ground station processing module displays the real-time video stream containing the predicted target bounding box and the target's coordinate information, and archives the target's latitude and longitude and the image.

[0038] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0039] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A UAV optical image unexploded ordnance detection system based on pseudo-tag generation, characterized in that, include: Ground station processing module, UAV-ground station communication module, airborne computing module, UAV platform; The ground station processing module manually sets the operation area and generates the initial search operation route. The operation route is transmitted to the airborne computing module through the UAV-ground station communication module. The airborne computing module uploads the operation route to the UAV platform, controls the UAV to execute the initial search mission, and receives images collected in real time by the UAV. The airborne computing module transmits the real-time collected images to the ground station processing module through the UAV-ground station communication module.

2. The unexploded ordnance detection system for UAV optical images based on pseudo-tag generation according to claim 1, characterized in that, The ground station processing module processes the image into an orthophoto map and calculates the target area of ​​interest within the operational area. It then creates a flight path containing only the target area of ​​interest and transmits this flight path to the onboard computing module via the UAV-Ground Station communication module. The onboard computing module uploads the flight path to the UAV platform, controls the UAV to perform the photography task, and receives images acquired in real time by the UAV. The onboard computing module then transmits the acquired images to the ground station processing module via the UAV-Ground Station communication module. The ground station processing module uses a visual language model to coarsely annotate the captured images and fuses the coarsely annotated data with existing data to train a deep learning target detection model for the current task. This target detection model is then uploaded to the onboard computing module via the UAV-Ground Station communication module. The onboard computing module loads the target detection model, processes the real-time video stream transmitted from the UAV to the onboard computing module, draws bounding boxes around the locations of unexploded ordnance targets in the video stream, encodes the images beyond the bounding boxes, and transmits the encoded video stream, along with the tilt angle of the UAV's optical payload and the UAV's GPS coordinates, to the ground station processing module via the UAV-Ground Station communication module. The ground station processing module receives the processing results, displays them on the user interface, and archives them.

3. The unexploded ordnance detection system for UAV optical images based on pseudo-tag generation according to claim 1, characterized in that, The ground station processing module includes: a flight path planning unit, an orthophoto generation unit, an image pseudo-labeling unit, a YOLOv11 deep learning target detection model training unit, and a unit for displaying and archiving real-time UAV detection results.

4. The unexploded ordnance detection system for UAV optical images based on pseudo-tag generation according to claim 3, characterized in that, in, The flight path planning unit includes: initial search flight path planning and detailed image capture flight path planning for the area of ​​interest. Initial search flight path planning automatically plans the flight path using an S-shaped parallel scanning method after the search area is set, and controls the flight path spacing based on the camera's field of view and a preset overlap rate to ensure sufficient overlap between adjacent aerial images. The area of ​​interest image capture flight path planning first scans the orthophoto image into regions, then uses a pre-trained convolutional neural network to determine the area of ​​interest according to a set threshold, setting all areas of interest as waypoints that the UAV must pass through. The UAV's flight path planning problem is modeled as a traveling salesman problem, solved using a genetic algorithm. Through chromosome encoding, selection, crossover, and mutation operations, iterative optimization yields an approximately optimal flight path covering all task points. When the UAV flies to each waypoint, it captures images of the area of ​​interest according to preset values, with different tilt angles of the optical payload, different circular flight paths, and a specified frequency. No image capture is performed when the UAV flies between waypoints. The unit for generating orthophotos includes: Component M101: Performs lens distortion and radiation correction on the acquired images; Component M102: Implements sparse 3D reconstruction based on feature matching and solves the camera exterior orientation by adjusting the binding. Component M103: A digital surface model is generated using dense stereo matching based on optimized camera parameters; Component M104: Utilizes digital surface models involving both inside and outside the camera to perform orthorectification on images, as well as color balancing and automatic stitching, to output georeferenced digital orthophotos.

5. The unexploded ordnance detection system for UAV optical images based on pseudo-tag generation according to claim 3, characterized in that, The image pseudo-labeling unit includes: using a visual language model to process the image output of the photographed flight path of the region of interest to generate pseudo-label boxes, and using a convolutional neural network classification model to determine whether the target in the label box is an unexploded ordnance; Furthermore, the image is labeled with pseudo-label units, including: Component M201: Scales the image to be detected so that its length and width meet the multiples of the block size set in the visual language model and are close to the original size, and extracts the multi-scale feature representation of the image through the visual encoder; Component M202: Embeds and vectorizes the prompt words; Component M203: Enables feature fusion of images and prompts in cross-modal interaction components; Component M204: Outputs predicted statements in the output layer of the visual language model; Component M205: Post-processes the predicted statement to generate ground truth values ​​and class labels for target boxes used to train the target detection model. It supports manually setting confidence thresholds and manually modifying pseudo-labels. The unit for training the YOLOv11 deep learning object detection model includes: Component M301: Counts the number of samples with pseudo-labels and randomly extracts a specified proportion of sample data from the database and uses it together with the pseudo-label data as the training dataset. Component M302: Sets a higher probability for executing data augmentation strategies for single images on pseudo-labeled data; Component M303: The fused dataset is divided into training and validation sets proportionally, fine-tuned on the YOLOv11 model trained on a large-scale general drone dataset, and the model with the highest average detection accuracy in the validation set is saved. Component M304: It is manually determined whether the model generated in the previous step needs to be used to re-label the images of the flight path output of the region of interest, and then retrain the YOLOv11 model. Component M305: Uploads the trained deep learning model to the recording and computing module and archives the training logs and the model; The unit for displaying and archiving real-time detection results from UAVs includes: real-time display of video streams, calculation of target GPS coordinates, and data archiving; Specifically, the unit displays and archives the real-time detection results from the UAV. It uses the GStreamer framework to decode and render the video stream sent by the onboard computing module in real time, ensuring low-latency display at different resolutions and bitrates. The coordinates of the target at image points are converted into direction vectors in the camera coordinate system using the camera intrinsic parameter matrix. Then, the direction vectors in the camera coordinate system are converted to geographic coordinate vectors using the optical payload gimbal attitude matrix and the UAV's GPS coordinates. The intersection of the vectors with the digital surface model is used to obtain the actual GPS coordinates of the detected target. The video stream and GPS data stream are aligned according to their timestamp information, and the aligned data is stored in the database for archiving. The optical payload gimbal attitude includes pitch angle, roll angle, and yaw angle.

6. The unexploded ordnance detection system for UAV optical images based on pseudo-tag generation according to claim 1, characterized in that, The UAV-Ground Station Communication Module includes: a high-bandwidth wireless module that can be modified to switch frequency bands to communicate with the ground station processing module, supporting TCP, IP, or UDP protocols. Video streams use RTP or RTSP protocols based on GStreamer for low-latency transmission, while operation commands, status feedback, and deep learning models use TCP or HTTPS to ensure reliable transmission; and a UART and USB interface for communication with the onboard computing module. Control commands are transmitted bidirectionally via the UART interface, while video streams and deep learning models are transmitted at high speed via the USB interface. The airborne computing module includes: a UART and USB interface for transmitting video, deep learning models, and control commands to the UAV and ground station communication module; a MIPI CSI data bus for receiving video streams transmitted from the UAV; an RS-422 interface for sending flight control commands and camera operation commands to the communication module; and the ability to receive operation commands from the ground station. The airborne computing module also incorporates an embedded GPU for deep learning inference and a dedicated video hardware codec chip to reduce the computational burden on the CPU. Furthermore, the airborne computing module includes: controlling the UAV's automatic flight and image acquisition; the airborne computing module acts as a relay to transmit images acquired by the UAV; during the real-time unexploded ordnance detection phase, it loads a deep learning target detection model to decode the data transmitted by the UAV, detects the target position and draws prediction boxes on the video frames, encodes and compresses the processed video stream and transmits it to the UAV-ground station communication module; and it transmits time-stamped optical payload gimbal attitude information and UAV GPS information to the UAV-ground station communication module. The drone platform specifically refers to an optical payload with a built-in video hardware encoding module that compresses the acquired raw images into H.264 or H.265 video streams in real time; the drone receives flight path configuration items to complete automatic operations; the drone platform has built-in GPS and IMU sensors to record the drone's GPS information and the optical payload's gimbal attitude information in real time; the optical payload supports multiple shooting modes, including timed shooting, continuous shooting, and area scanning; and the video stream and sensor data are time-stamped synchronously.

7. A method for detecting unexploded ordnance from UAV optical images based on pseudo-tag generation, characterized in that, The unexploded ordnance detection system based on pseudo-tag generation of UAV optical images, as described in claim 1, comprises: Step S1: Identify the region of interest and acquire optical data; Step S2: After completing the optical data acquisition of the area of ​​interest, the ground station processing module executes pseudo-labels and trains the YOLOv11 model; Step S3: After completing the YOLOv11 model training, perform a real-time unexploded ordnance optical target detection task and summarize and archive the detection results.

8. The method for detecting unexploded ordnance from UAV optical images based on pseudo-tag generation according to claim 7, characterized in that, Step S1 includes: Step S101: Generate an S-shaped parallel flight path file according to the manually planned search range and the preset overlap rate of aerial images, and upload the flight path file to the UAV to perform the aerial photography task; Step S102: Perform geometric correction and projection processing on the aerial images collected by the UAV to generate orthophotos with georeferenced information; Step S103: Use a pre-trained convolutional neural network to divide the orthophoto into regions by sliding window to determine whether there is a region of interest, and save the GPS coordinates of the region of interest; Step S104: Use a genetic algorithm to find the solution to the optimal trajectory planning problem for the waypoint set consisting of all regions of interest, generate a route file that takes pictures around the regions of interest and flies quickly between the regions of interest, and upload the route file to the UAV to perform the data acquisition task; Step S105: The UAV completes the data acquisition task and transmits the acquired data to the ground station processing module.

9. The method for detecting unexploded ordnance from UAV optical images based on pseudo-tag generation according to claim 7, characterized in that, Step S2 includes: Step S201: Scale the optical image of the region of interest with the principle that the length and width meet the multiples of the block size set in the visual language model and the scaled image maintains the minimum deviation from the original resolution. Extract the multi-scale feature representation of the image through the visual encoder. Step S201: Embedded and vectorized the prompt words, fused them with the features extracted by the visual encoder, and output the predicted sentence in the output layer of the visual-language model; Step S202: Post-process the predicted statement to generate pseudo-labeled boxes and category labels for training the object detection model; Step S203: Crop the image according to the coordinates of the generated pseudo-label boxes, and use a convolutional neural network classification model trained in the field of unexploded ordnance to determine whether the target in the cropped image is an unexploded ordnance target, and retain pseudo-label boxes with a confidence threshold. Step S204: Determine whether the pseudo-annotation box needs to be manually modified according to requirements; Step S205: Count the number of samples annotated with pseudo-label boxes, randomly extract a predetermined proportion of sample data from the database and use it together with the pseudo-label box data as the training dataset, and divide the dataset into training set and validation set according to the proportion. Step S206: In the data augmentation stage, the dataset is divided into training set and validation set proportionally; Step S207: Train the YOLOv11 model. The pre-trained model is a YOLOv11 model trained on a large-scale UAV dataset. The model with the highest average detection accuracy in the validation set is saved. Step S208: Determine whether it is necessary to use the model generated in the previous step to re-label the images of the fine-grained photographic flight path output of the region of interest and retrain the YOLOv11 model. If not, update the pseudo-labels and repeat step S205. Step S209: Upload the trained YOLOv11 model to the recording and calculation module, and archive the training logs and the model.

10. The method for detecting unexploded ordnance from UAV optical images based on pseudo-tag generation according to claim 7, characterized in that, Step S3 includes: Step S301: Load the YOLOv11 model onto the airborne computing module and set it to inference mode; Step S302: The airborne computing module decodes the data transmitted by the UAV, detects the target position and draws a prediction box on the video frame. The processed video stream is encoded and compressed using the Gstreamer framework and then transmitted to the ground station processing module. At the same time, the UAV's GPS information and optical payload attitude information are also transmitted. Step S303: The ground station processing module decodes the video stream containing the predicted target bounding box, aligns the video stream timestamp with the UAV GPS information and the optical payload attitude information timestamp; Step S304: The ground station processing module calculates the actual GPS coordinates of the target based on the optical payload intrinsic and extrinsic parameters matrix and the digital surface model; Step S305: The ground station processing module displays the real-time video stream containing the predicted target bounding box and the target's coordinate information, and archives the target's latitude and longitude and the image.

Images