Unexploded ordnance drop point prediction and refined search system and method

Abstract

The invention belongs to the technical field of drop point prediction and refined search, and provides a drop point prediction and refined search system and method based on unexploded ordnance. The method comprises the steps that an unexploded trajectory simulation visual model is established to predict an unexploded falling point distribution area, a comprehensive cost map of a target area is generated in combination with earth surface information, and an intelligent vehicle conducts autonomous navigation and gets close to the target area according to unexploded falling point distribution information. An intelligent vehicle generates an optimal search path through a comprehensive cost map to perform global search on a target area, detects a target in real time by using data acquired by a depth camera and combining a pre-trained unexploded shell deep learning model, and transmits the data to a ground station after determining the accurate position of an unexploded shell. By adopting the method provided by the invention, the falling point position of the unexploded ordnance is predicted more accurately and efficiently, the huge manpower and financial cost of an unexploded ordnance recovery task is reduced, and the potential harm of the unexploded ordnance to ground search personnel is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of impact point prediction and refined search, and in particular to a system and method based on the impact point prediction and refined search of unexploded bombs. Background Art

[0002] Unexploded Ordnance (UXO) refers to explosives such as artillery shells, bombs, mines, grenades, etc. that fail to detonate or explode normally during wars, conflicts or training activities. These UXOs may remain in various areas such as battlefields, conflict areas or military training grounds. UXOs may detonate at any time, causing harm or fatal threats to nearby personnel. Whether it is military personnel, civilians or rescue workers, they are at risk of being harmed by unstable explosives. At the same time, UXOs may contain toxic chemicals or radioactive substances, which are released during the detonation or corrosion process and cause environmental pollution. These harmful substances may enter the soil, water sources and air, causing long-term damage and pollution to ecosystems, farmland and wildlife. It has a huge impact on society, the environment and the economy. Timely discovery, removal and elimination of UXOs, as well as strengthening relevant education and protective measures, are crucial to ensuring people's safety and promoting sustainable development in affected areas.

[0003] With the rapid progress and development of artificial intelligence and simulation modeling technology, it has had a wide and far-reaching impact on people's lives. At the same time, artificial intelligence and simulation modeling technology also play an important role in the research and development and optimization of military equipment and weapon systems. Through simulation and emulation, the performance, accuracy and effectiveness of various combat equipment can be evaluated, and military science and technology researchers can be helped to optimize and improve them. Artificial intelligence can also be applied to autonomous weapon systems and unmanned systems to improve the level of automation and intelligence, so that they have higher combat effectiveness and the ability to perform tasks. Traditional unexploded bomb drop prediction and search methods usually rely on historical combat data, intelligence analysis, and factors such as topography for inference and speculation. Based on known combat locations, troop activity trajectories and combat behaviors, military analysts will try to predict the possible drop areas of unexploded bombs. Then, deminers will conduct searches and inspections in these predicted areas, searching through visual, auditory and metal detectors and other equipment to find potential unexploded bombs. Although traditional unexploded bomb drop prediction and search methods can detect and eliminate unexploded bombs to a certain extent, their low efficiency, high missed detection rate and safety risks limit their scope of application and effectiveness. If simulation modeling technology can be applied to the field of unexploded bomb drop point prediction, it will improve the accuracy, efficiency and safety of unexploded bomb detection to better protect people's lives and social stability.

[0004] In recent years, with the continuous development of machine vision technology, point cloud recognition technology has been increasingly applied to key fields such as industrial manufacturing and robot navigation. The key steps of point cloud recognition technology include feature extraction, feature description and classification recognition. First, by extracting features such as geometric shape, color, texture, etc. from the point cloud data, each point is represented as a feature vector with certain semantic information. Then, these feature vectors are described and learned using machine learning algorithms or deep learning models to establish a model of the object or structure. Finally, according to the learned model, the new point cloud data is classified and identified to determine the category of the object or structure. However, so far, point cloud recognition technology has rarely been used in the field of unexploded bomb search. If point cloud recognition technology can be applied to the field of unexploded bomb search, breaking through the traditional unexploded bomb drop point prediction and refined search methods, the search efficiency of unexploded bombs can be improved.

[0005] Therefore, it is very necessary to invent a system and method based on the prediction and refinement of the landing point of unexploded bombs. Summary of the invention

[0006] Based on the above problems, the present invention proposes a system and method based on prediction and refined search of unexploded bomb landing points.

[0007] The technical solution of the present invention is as follows: a system based on prediction and refined search of unexploded bomb landing points, comprising an intelligent vehicle, a ground station and a digital twin model of the intelligent vehicle; the ground station is integrated with an unexploded bomb trajectory landing point prediction simulation module, and the unexploded bomb trajectory landing point prediction simulation module predicts the landing position of the unexploded bomb according to the unexploded bomb environment and launch parameters; the intelligent vehicle is integrated with a ROS system, deploys an unexploded bomb point cloud training model, and is loaded with a depth camera, a radar, an RTK communication module and an autonomous navigation mobile chassis; when the intelligent vehicle autonomously navigates to the predicted landing point distribution area of ​​the unexploded bomb trajectory landing point prediction simulation module, the intelligent vehicle performs a refined search on the predicted landing point distribution area and combines the real-time data collected by the depth camera to accurately locate the unexploded bomb; then the precise position of the unexploded bomb is transmitted to the ground station; the human-computer interaction page on the ground station displays the precise position information of the unexploded bomb in real time; the digital twin model of the intelligent vehicle is used to monitor, simulate and optimize the performance of the intelligent vehicle in real time by creating a virtual model corresponding to the intelligent vehicle and its operating environment in the real world.

[0008] A method based on prediction and refined search of unexploded bomb drop points, the specific steps are as follows:

[0009] Step S1, creating an unexploded bomb sample point cloud dataset and training it, and detecting unexploded bombs in the point cloud using the PointNet++ algorithm;

[0010] Step S2, using Matlab software to establish an unexploded bomb trajectory fall point prediction simulation module to predict the fall point area of ​​the unexploded bomb;

[0011] Step S3, the smart car uses the predicted landing area obtained in step S2 to autonomously navigate to the target area, autonomously make decisions and plan to conduct a refined search of the target area;

[0012] Step S4, the central controller of the smart car deploys the unexploded bomb point cloud training model, and autonomously identifies the unexploded bomb through the data collected in real time by the depth camera to determine the precise location of the unexploded bomb;

[0013] Step S5, transmitting the precise location of the unexploded bomb obtained in step S4 to the ground station, and the human-computer interaction page on the ground station displays the precise location information of the unexploded bomb in real time;

[0014] Step S6, build a virtual vehicle model of the smart car through the Unity engine digital modeling technology; during the autonomous navigation of the smart car, ensure the real-time transmission of sensor data on the smart car and the digital twin virtual vehicle model through the CoAP communication protocol; the control strategy optimization module formulates a staged optimization control strategy for the internal environment of the smart car to improve the optimization strategy.

[0015] The operations of preparing a point cloud dataset of unexploded bomb samples and training and detecting unexploded bombs in the point cloud using the PointNet++ algorithm are as follows:

[0016] S1.1, using the depth camera to collect the point cloud data of the unexploded bomb, and using the Point-Cloud-Viewer software to annotate the point cloud of the unexploded bomb;

[0017] S1.2, the statistical filtering method is used to denoise the unexploded bomb point cloud data, and the random sampling and centralization methods are used to downsample and normalize the point cloud data to improve the training effect and speed of the unexploded bomb model.

[0018] S1.3, divide the unexploded bomb dataset into training set, validation set and test set; select Pointnet++ model to build the unexploded bomb recognition network model, and use the prepared unexploded bomb point cloud data to pre-train the built network model; during the training process, use the cross entropy loss function to measure the difference between the model's prediction and the true label, and use the stochastic gradient descent algorithm to update the model's parameters to minimize the loss function;

[0019] S1.4, use the validation set to tune the unexploded bomb model and adjust the learning rate to improve the performance and generalization ability of the model; use the test set to evaluate the trained unexploded bomb model and calculate the accuracy of the model on unseen unexploded bomb point cloud data;

[0020] S1.5, deploy the trained unexploded bomb identification network model to the central controller of the smart car, so that it can accurately detect the unexploded bombs in the point cloud.

[0021] The specific steps of establishing the unexploded bomb drop point prediction simulation module and predicting the drop point coordinates of the unexploded bomb include:

[0022] S2.1, collect data related to the flight trajectory of the unexploded bomb, including the size, shape, mass, and environmental conditions of the unexploded bomb;

[0023] S2.2, using Newton's laws of motion Combined with the information of the initial velocity v0, mass, angle of attack, etc. of the unexploded shell, a mathematical model of the center of mass motion of the unexploded shell is established, and the equation of motion of the unexploded shell rotating around the center is obtained. Where G is the momentum moment of the unexploded bomb, and M represents the total external torque on the unexploded bomb;

[0024] S2.3, the increase of air pressure will lead to the increase of air density, thus reducing the range. Use the fluid mechanics equation and the gas state equation to derive the relationship between the change of air pressure and density with altitude in the atmosphere, calculate the flight resistance and gas weight of the unexploded bomb, and further study the ballistic characteristics;

[0025] S2.4, considering the change of the velocity of the center of mass with time, combined with the magnitude of the drag and gas weight, the Runge-Kutta method is used to calculate the flight velocity module of the unexploded bomb;

[0026] S2.5, integrate the mathematical model of center of mass motion, the size of drag and gas weight, and the module of flight speed of unexploded bombs, and build the overall ballistic simulation model through the Simulink module in Matlab;

[0027] S2.5, input the initial parameters of the unexploded bomb, display the motion trajectory of the unexploded bomb in real time during the simulation process, and predict the coordinates of the landing point of the unexploded bomb.

[0028] The intelligent vehicle uses the predicted landing point distribution area obtained by S2 to autonomously navigate to the target area, make autonomous decisions, and plan the strategy for a refined search of the target area as follows:

[0029] S3.1, the smart car receives the predicted landing point range from the landing point prediction simulation model, performs RTK positioning with the northeast sky as the coordinate system, obtains the surrounding environment information through the laser radar, and imports it into the cost map synchronously, combines the global path planning A* algorithm and the local path planning algorithm DWA, and autonomously moves to the vicinity of the landing point prediction area; before the smart car starts navigating, a path connecting the starting point and the target point as a whole is generated by considering the environmental map, the starting point and the target point information; the smart car starts to move, and under the guidance of the global path, the DWA local path planning algorithm is used to generate a feasible local path in the current environment in real time to avoid obstacles and ensure safe arrival at the target;

[0030] S3.2, the intelligent vehicle autonomously detects unexploded bombs, uses the adaptive Monte Carlo positioning algorithm AMCL and combines it with the global cost map to conduct a refined search of the predicted drop point distribution area.

[0031] The central controller of the smart car deploys the unexploded bomb point cloud training model, and autonomously identifies the unexploded bomb through the data collected in real time by the depth camera to determine the precise location of the unexploded bomb. The specific steps include:

[0032] S4.1, the depth camera collects point cloud data of the target area in real time and transmits the collected data to the central controller of the smart car through a serial port connection;

[0033] S4.2, the central controller of the smart car deploys the unexploded bomb point cloud training model in advance, detects the unexploded bombs in the data through the PointNet++ algorithm, and determines the precise location of the unexploded bomb through the 3D model feature matching algorithm.

[0034] The precise location of the unexploded bomb obtained by S4 is transmitted to the ground station. The human-computer interaction page on the ground station displays the precise location information of the unexploded bomb in real time. The operation method is as follows:

[0035] S5.1, the central controller of the smart car sends the precise location of the unexploded bomb to the ground station through the RTK communication module;

[0036] S5.2, the human-computer interaction page on the ground station displays in real time the video data collected by the depth camera, the predicted simulation posture of the unexploded bomb, and the precise location information of the unexploded bomb.

[0037] The virtual vehicle model of the smart car is constructed through the Unity engine digital modeling technology; during the autonomous navigation of the smart car, the CoAP communication protocol is used to ensure the real-time transmission of the sensor data on the smart car and the digital twin virtual vehicle model; the control strategy optimization module formulates a phased optimization control strategy for the internal environment of the smart car to improve the optimization strategy; the specific operation methods are as follows:

[0038] S6.1, use 3D modeling software Blender to create a 3D model of the smart car, build the appearance, import the model into the Unity project, select physical properties that match the actual smart car for the virtual vehicle model to achieve a realistic appearance display; adjust material properties to optimize visual effects; add physical components to the virtual vehicle in the Unity project, set the vehicle's power system, suspension system and tires to simulate the real driving behavior of the smart car when it is running; add sensor simulation components to the virtual vehicle model, and configure its parameters and behaviors to achieve simulation and visualization of sensor data; finally, achieve free movement of the virtual vehicle model;

[0039] S6.2, set up a CoAP server in the system of the digital twin virtual vehicle model, configure the IP address and port of the server, receive sensor data from the smart car, and ensure a stable network connection; configure the CoAP client in the control system of the smart car, set the target server address and port of the client, determine the format of the sensor data as JSON, ensure the consistency of the data when sending and receiving, and send the sensor data to the CoAP server of the virtual vehicle model; implement the data collection function in the sensor system of the smart car, read the sensor data regularly and prepare to send; implement the data receiving and parsing function on the CoAP server side, parse the received data into a format suitable for the virtual model and update it to the digital twin virtual vehicle model, to ensure that the model reflects the actual situation of the smart car;

[0040] S6.3, in the control strategy optimization module of the smart car, first use the CoAP server to obtain the sensor data of the real smart car and the surrounding conditions of the vehicle in real time, including the location of obstacles, road conditions and the current state of the vehicle; taking into account the existence of obstacles, by adding a collision module to the virtual vehicle model and setting a threshold, when the coefficient detected by the virtual vehicle model collision module is less than the preset threshold, a first instruction is generated, and when the coefficient detected by the virtual vehicle model collision module is greater than the preset threshold, a second instruction is generated; when the first instruction is generated, the virtual vehicle model drives normally; when the second instruction is generated, the virtual vehicle model stops immediately and issues a warning, and sends the warning to the real smart car through the CoAP server; after receiving the warning signal, the real smart car re-plans the path until the first instruction is generated to safely pass the obstacle; the whole process continuously performs data collection, path planning, obstacle avoidance and strategy adjustment through the main loop, and finally realizes efficient and safe driving of the smart car in a dynamic environment.

[0041] In the control strategy optimization module, the sensor data and road conditions of the real smart car are also obtained in real time through the CoAP server, including the road friction coefficient, slope, and the position and distance information of the obstacles ahead, and a speed adjustment module is designed and a speed adjustment threshold is set: when the road friction coefficient is lower than the preset value, a first instruction is generated, and when the road friction coefficient is higher than the preset value, a second instruction is generated; when the first instruction is generated, the virtual vehicle model reduces the driving speed; when the second instruction is generated, the virtual vehicle model increases the driving speed; when the slope exceeds the set value, an instruction to increase the speed is generated; the virtual vehicle model adjusts the driving speed according to these instructions, and sends the adjustment results to the real smart car through the CoAP server; after receiving the speed adjustment instruction, the real smart car adjusts its driving speed in real time and performs dynamic path planning to ensure safe and efficient driving under various road conditions; the whole process continuously performs data collection, speed adjustment and path planning through the main loop to ensure that the smart car can flexibly respond to various challenges in a dynamic environment.

[0042] Beneficial effects of the present invention: The system and method based on the prediction and refined search of the unexploded bomb landing point provided by the present invention can quickly and accurately predict the landing point of the unexploded bomb, and efficiently perform refined search for the unexploded bomb. The system uses the Simulink simulation software in Matlab to simulate the launch trajectory of the unexploded bomb, and obtains the predicted landing point distribution area of ​​the unexploded bomb. The point cloud model of the unexploded bomb is trained in advance by the deep learning method. The advantages of the strong adaptability and high safety of the smart car are used, and the depth camera is combined to provide additional visual information of the unexploded bomb. It can successfully complete the prediction and refined search tasks of the unexploded bomb landing point, and improve the efficiency, reliability and safety of the prediction and refined search of the unexploded bomb landing point. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the unexploded bomb drop point prediction and refined search system of the present invention;

[0044] Figure 2 A path planning diagram for the smart car of the present invention;

[0045] Figure 3 The present invention obtains a three-dimensional point cloud data map of unexploded bombs. DETAILED DESCRIPTION

[0046] A method based on prediction and refined search of unexploded bomb drop points is applied to the intelligent system based on prediction and refined search of unexploded bomb drop points.

[0047] Enter the Simulink simulation environment and establish a simulation model of the external ballistic motion system. Take the unexploded bomb as the research object, and use MATLAB to establish the mathematical model of the center of mass motion of the unexploded bomb according to the physical parameters, initial velocity and launch angle of the unexploded bomb. Consider the influence of factors such as gravity, air resistance and wind force on the trajectory, and add resistance function and air weight function modules to the model. Use the Runge-Kutta numerical method to numerically solve the ballistic model and obtain the velocity module of the unexploded bomb. Integrate the mathematical model of the center of mass motion, the resistance function and air weight function modules and the velocity module to build an overall ballistic simulation model. Use the Geographic Information System (GIS) toolbox or terrain data processing tool in MATLAB to import terrain or landform data. According to the imported terrain data, use MATLAB to process and model the data and generate a numerical representation of the terrain information. Consider the influence of the shape of the terrain, obstacles and other geographical elements on the trajectory landing point. Integrate these elements into the trajectory simulation model and perform interactive calculations with the terrain information. Add the unexploded bomb trajectory data visualization module to obtain a model that can predict the distribution area of ​​the trajectory landing point;

[0048] The point cloud data related to unexploded bombs is collected through the depth camera to ensure that the point cloud data contains the location and shape information of the unexploded bombs and is distinguished from the point cloud data of other environments or obstacles. The collected point cloud data is preprocessed by denoising, downsampling and normalization to improve data quality and accuracy. The location and shape information of the unexploded bombs are manually annotated in the point cloud data. The annotated point cloud data set is divided into training set, validation set and test set using the holdout method. The Pointnet++ algorithm is used to train the model of the unexploded bomb point cloud data. According to the results of the validation set, the model is optimized and adjusted in terms of hyperparameters, network results, etc. using the depth information method. The trained model is used to predict the new unexploded bomb point cloud data so that the model can accurately output the shape prediction results of the unexploded bomb.

[0049] The smart car receives the predicted landing area from the landing prediction simulation model and uses a combination of global path planning and local path planning to achieve autonomous navigation. In the first step, the global path planning is combined with the A* algorithm. Before the smart car starts navigating, a path that connects the starting point and the target point as a whole is generated by considering the environment map, the starting point and the target point. In the second step, once the smart car starts moving, under the guidance of the global path, the DWA (Dynamic Window Approach) local path planning algorithm is used to generate a feasible local path in the current environment in real time to avoid obstacles and ensure safe arrival at the target. In the third step, once the local path is obtained, under the guidance of the path tracking controller, the smart car moves along the set path. The path tracking controller usually considers factors such as the smart car dynamics model and sensor feedback to ensure that the smart car moves stably along the path.

[0050] Step 4: The smart car arrives at the predicted landing point distribution area of ​​the landing point prediction simulation model. Since the landing point prediction area is a relatively large circular area, the goal is to maximize the passing probability and find the optimal path from the starting point to the end point. Therefore, the adaptive Monte Carlo positioning algorithm (AMCL) is used in combination with the global cost map to perform a refined search for the predicted landing point area. While the smart car is moving steadily along the path, the depth camera acquires the depth image and color image of the scene in real time to generate point cloud data. The smart car central controller preprocesses the point cloud data, including denoising, filtering and downsampling, to improve the efficiency and accuracy of subsequent processing. Information features such as surface normals and curvature are extracted from the preprocessed point cloud data for better identification of unexploded bombs. The trained unexploded bomb point cloud learning model is used to perform target detection and segmentation on the point cloud data. Based on target detection and segmentation, the detected targets are further classified to identify unexploded bombs. The precise location of the unexploded bomb is determined by the 3D model feature matching algorithm, and the detected unexploded bomb location is marked on the color image so that the operator can see it clearly.

[0051] As a preferred embodiment, based on the above technical solution, the following improvements are made:

[0052] The unexploded bomb trajectory prediction simulation model predicts the distribution area of ​​the unexploded bomb. The smart car autonomously navigates to the target area and conducts a refined search strategy for the target area. The specific steps include:

[0053] (1) Establish an unexploded bomb trajectory prediction simulation module through the Simulink module in Matlab.

[0054] (2) Input the initial information of unexploded bombs and environmental information to predict the distribution area of ​​unexploded bomb landing points.

[0055] (3) The smart car autonomously navigates to the target area based on a combination of global path navigation (A* algorithm) and local path navigation (PWA algorithm).

[0056] (4) The intelligent vehicle uses the adaptive Monte Carlo localization algorithm (AMCL) and combines it with the global cost map to perform a refined search of the predicted landing area.

[0057] (5) The smart car determines the precise location of the unexploded bomb through a 3D feature matching algorithm.

[0058] As a preferred embodiment, based on the above technical solution, the following improvements are made:

[0059] The model training of unexploded bomb point cloud data using the Pointnet++ algorithm includes the following steps:

[0060] (1) Local spatial hierarchy: PointNet++ introduces a local spatial hierarchy to segment the unexploded bomb point cloud data layer by layer from global to local from bottom to top. This can capture richer local feature information in the unexploded bomb point cloud data.

[0061] (2) Sampling and organization: At each level, PointNet++ uses sampling and organization strategies to generate new point sets. By splitting larger point sets into smaller local regions, it can better handle irregular and uneven unexploded bomb point cloud data.

[0062] (3) Local feature learning: For each local area of ​​unexploded bombs, PointNet++ uses a multi-layer perceptron (MLP) network to learn local features. By stacking multiple MLP layers, features of points in the local area can be extracted and learned, and rich local feature representations of unexploded bombs can be obtained.

[0063] (4) Upsampling and aggregation: After learning the local features of the unexploded bombs, PointNet++ uses upsampling and aggregation strategies to aggregate the local features of the unexploded bombs. This can aggregate local features from low levels to high levels, and achieve global feature learning of the overall point cloud data of the unexploded bombs.

[0064] (5) Global feature learning: At the highest level, PointNet++ learns the global features of the unexploded bomb through fully connected layers and pooling operations. This can obtain a more descriptive global feature representation of the entire unexploded bomb point cloud data.

[0065] (6) Classification and segmentation: Using the final unexploded bomb global feature representation, PointNet++ can perform classification and segmentation tasks for point cloud data. By inputting the unexploded bomb global features into the classifier or segmenter, label prediction and boundary segmentation of the unexploded bomb point cloud data can be achieved.

[0066] As a preferred embodiment, based on the above technical solution, the following improvements are made:

[0067] Determining the precise location of unexploded bombs using a 3D model feature matching algorithm includes the following steps:

[0068] (1) Use a depth camera to collect point cloud data of the area where the unexploded bomb is located.

[0069] (2) The central controller of the smart car models the target area and generates its 3D model.

[0070] (3) Extract features such as surface texture and edge information from the collected point cloud data and the established point cloud model.

[0071] (4) Describe the extracted features for matching.

[0072] (5) Match the features in the collected point cloud data with the features in the established unexploded bomb point cloud model.

[0073] (6) Estimate the posture information of the unexploded bomb relative to the sensor based on the matched feature points or feature descriptors.

[0074] (7) Combining the attitude information and the position information of the sensor, the precise position of the unexploded bomb in the global coordinate system is estimated.

[0075] As a preferred embodiment, based on the above technical solution, the following improvements are made:

[0076] The smart car sends the precise location of the unexploded bomb to the ground station, and the human-computer interaction interface of the ground station displays the precise location of the unexploded bomb, which also includes the following steps:

[0077] (1) The smart car transmits the real-time video stream of unexploded bombs to the ground station, and the human-computer interaction interface of the ground station displays the collected unexploded bomb videos.

[0078] (2) The human-computer interaction interface of the ground station displays the simulated ballistic trajectory and predicted landing point distribution area of ​​the unexploded bomb trajectory simulation prediction model.

[0079] (3) The human-computer interaction interface of the ground station displays the position information of the smart car in real time.

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] The purpose of the present invention is to provide a system and method based on the prediction and refined search of the landing point of unexploded bombs, which has accurate positioning of the landing point of unexploded bombs, high detection efficiency, strong portability and stability, overcomes the disadvantages of the difficult and high consumption cost of general unexploded bomb recovery tasks, and improves the efficiency of the prediction and refined search of the landing point of unexploded bombs.

[0082] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0083] like Figure 1 As shown, this example is applied to the unexploded bomb drop point prediction and refined search system and method, including the following steps:

[0084] Step 1: Use the Simulink simulation module in Matlab to establish an unexploded bomb trajectory drop point prediction simulation module.

[0085] Step 2: Input the initial information of the unexploded bomb, such as the initial launch position, launch speed, launch angle, and environmental information such as air assistance and air weight, into the unexploded bomb trajectory landing point prediction simulation module.

[0086] Step 3: Use the Runge-Kutta method to determine the ballistic motion equation of the unexploded bomb and obtain the predicted location of the unexploded bomb.

[0087] Step 4: The smart car receives the predicted coordinates from the landing point prediction simulation model and autonomously navigates to the predicted landing point range using a combination of the global path planning A* algorithm and the local path planning DWA (Dynamic Window Approach) algorithm.

[0088] Step 5: The smart car arrives at the distribution area of ​​the predicted unexploded bomb drop points, and uses the adaptive Monte Carlo positioning algorithm combined with the optimal search path generated by the global cost map to conduct a refined search of the target area.

[0089] Step 6: The depth camera on the smart car acquires the depth image of the scene in real time and generates point cloud data. The central controller of the smart car pre-processes the point cloud data and identifies the unexploded bomb through the Pointnet++ algorithm. If no unexploded bomb is identified, return to step 5 and conduct a refined search of the target area again.

[0090] Step 7: After identifying the unexploded bomb, the precise location of the unexploded bomb is determined through the 3D model feature matching algorithm, and the precise location is transmitted to the ground station platform through RTK. The ground station platform displays the position information of the smart car in real time, the video data monitored by the depth camera in real time, and the precise position information of the unexploded bomb detected by the central controller of the smart car, realizing the prediction of the landing point of the unexploded bomb and refined search operations.

[0091] The Simulink simulation module diagram for predicting the fall point of unexploded bombs includes the following steps:

[0092] Step 1: Use Simulink sub-packaging technology to build function modules such as addition, multiplication, resistance, and air weight.

[0093] Step 2: Use the XY Graph module to display the projectile motion trajectory curve.

[0094] Step 3: Use the Scope module to display the trajectory inclination angle and speed change curve.

[0095] Step 4: Determine the input-output relationship between modules, connect the modules, and obtain the simulation model.

[0096] The visualization interface diagram of the unexploded bomb drop prediction is designed using Matlab APP Designer, which includes the following steps:

[0097] Step 1: Input the initial parameters, initial velocity, elevation angle, pitch angle, ammunition shape and other coefficients, and solve them to obtain the external ballistic curve under these parameters.

[0098] Step 2: Obtain the predicted landing area of ​​the shell based on the horizontal distance, slant distance and height.

[0099] Step 3: Import the set Model Name model into the system through the Load_Model button.

[0100] Step 4: Import other parameters through the Import_Value button.

[0101] Step 5: Click the Simulink button to simulate the projectile center of mass motion simulation model in the APP interface.

[0102] The autonomous navigation unit of the smart car is a key component of the refined search system for unexploded bombs, mainly including mobile chassis, radar module, depth camera, RTK module, central control cabinet, etc. Its performance directly affects the efficiency of the search. The smart car can autonomously plan paths, avoid obstacles and reach the destination safely in unknown or dynamic environments. The smart car uses technologies such as the global positioning system (GPS), inertial navigation system (IMU), and laser radar SLAM (Simultaneous Localization and Mapping) for positioning to obtain the current location information of the smart car. The data of the surrounding environment is obtained through sensors such as laser radar and cameras, and a map is built in real time and updated so that the smart car can accurately understand the changes in the surrounding environment. Based on the acquired map information and target location, a path planning algorithm is used to generate a feasible path, and obstacle avoidance requirements and dynamic environmental changes are taken into account. Surrounding obstacles are detected using sensors such as laser radar and cameras, and obstacle avoidance decisions are made in combination with obstacle avoidance algorithms to ensure the safe passage of the smart car.

[0103] like Figure 2 As shown in the figure, it is a path planning diagram of the intelligent vehicle of the present invention. The intelligent vehicle adopts a method combining the global path planning A* algorithm and the local path planning DWA algorithm for autonomous navigation. The A* algorithm can find the shortest path or the optimal path in an environment based on the whole map, and can take into account obstacles and constraints in the environment; the PWA algorithm can generate a smooth and feasible local path in a real-time scenario to help the unmanned vehicle navigate safely and efficiently in a complex environment.

[0104] like Figure 3 As shown, the present invention obtains the three-dimensional point cloud data map of unexploded bombs, including the following steps:

[0105] Step 1: The depth camera acquires the point cloud image of the scene in real time.

[0106] Step 2: The central controller of the smart car performs denoising, downsampling, normalization and other operations on the on-site point cloud data.

[0107] Step 3: The central controller of the smart car processes the on-site point cloud information through the Pointnet++ algorithm to identify unexploded bombs.

[0108] Step 4: Transmit the detected unexploded bomb image back to the human-computer interaction page of the ground station.

[0109] The invention provides a system and method based on the prediction and refined search of unexploded bombs, which has accurate positioning of the landing point, high detection efficiency, strong portability and stability, and overcomes the disadvantages of the difficult task and high consumption cost of general unexploded bomb recovery. The unexploded bomb landing point prediction model is obtained by using the simulinik simulation software in Matlab software, and the ballistic simulation model of the unexploded bomb is obtained by using the Runge-Kutta method, so as to obtain the predicted distribution area of ​​the unexploded bomb landing point, and the global path planning A* algorithm and the local path planning DWA (Dynamic Window Approach) algorithm are used to make the smart car move autonomously to the vicinity of the predicted landing point distribution area of ​​the unexploded bomb, and the optimal search path generated by the adaptive Monte Carlo positioning algorithm and the global cost map is used to perform a refined search on the target area, and the depth camera on the smart car identifies the unexploded bomb through the Pointnet++ algorithm, and the precise position of the unexploded bomb is determined by the 3D model feature matching algorithm, and finally, the precise position of the unexploded bomb is transmitted to the ground station platform through the RTK communication module. The invention provides a system based on the prediction and refined search of unexploded bombs, which has high accuracy and reliability and good adaptability to complex environments.

Claims

1. A system based on prediction and refined search of unexploded bomb drop points, characterized in that: The unexploded bomb landing point prediction and refined search system includes an intelligent vehicle, a ground station and a digital twin model of the intelligent vehicle; the ground station is integrated with an unexploded bomb trajectory landing point prediction simulation module, and the unexploded bomb trajectory landing point prediction simulation module predicts the landing position of the unexploded bomb based on the unexploded bomb environment and launch parameters; the intelligent vehicle is integrated with a ROS system, deploys an unexploded bomb point cloud training model, and is equipped with a depth camera, a radar, an RTK communication module and an autonomous navigation mobile chassis; when the intelligent vehicle autonomously navigates to the predicted landing point distribution area of ​​the unexploded bomb trajectory landing point prediction simulation module, the intelligent vehicle conducts a refined search for the predicted landing point distribution area and combines the real-time data collected by the depth camera to accurately locate the unexploded bomb; then the precise position of the unexploded bomb is transmitted to the ground station; the human-computer interaction page on the ground station displays the precise position information of the unexploded bomb in real time; the digital twin model of the intelligent vehicle creates a virtual model corresponding to the intelligent vehicle and its operating environment in the real world, which is used to monitor, simulate and optimize the performance of the intelligent vehicle in real time.

2. A method based on prediction and refined search of unexploded bomb drop points, characterized in that: Based on the unexploded bomb drop point prediction and refined search system according to claim 1, the specific steps are as follows: Step S1, creating an unexploded bomb sample point cloud dataset and training it, and detecting unexploded bombs in the point cloud using the PointNet++ algorithm; Step S2, using Matlab software to establish an unexploded bomb trajectory fall point prediction simulation module to predict the fall point area of ​​the unexploded bomb; Step S3, the smart car uses the predicted landing area obtained in step S2 to autonomously navigate to the target area, autonomously make decisions and plan to conduct a refined search of the target area; Step S4, the central controller of the smart car deploys the unexploded bomb point cloud training model, and autonomously identifies the unexploded bomb through the data collected in real time by the depth camera to determine the precise location of the unexploded bomb; Step S5, transmitting the precise location of the unexploded bomb obtained in step S4 to the ground station, and the human-computer interaction page on the ground station displays the precise location information of the unexploded bomb in real time; Step S6, build a virtual vehicle model of the smart car through the Unity engine digital modeling technology; during the autonomous navigation of the smart car, ensure the real-time transmission of sensor data on the smart car and the digital twin virtual vehicle model through the CoAP communication protocol; the control strategy optimization module formulates a staged optimization control strategy for the internal environment of the smart car to improve the optimization strategy.

3. The method for predicting and refining the location of unexploded bombs according to claim 2 is characterized in that: The operations of preparing a point cloud dataset of unexploded bomb samples and training and detecting unexploded bombs in the point cloud using the PointNet++ algorithm are as follows: S1.1, using the depth camera to collect the point cloud data of the unexploded bomb, and using the Point-Cloud-Viewer software to annotate the point cloud of the unexploded bomb; S1.2, using statistical filtering method to denoise the unexploded bomb point cloud data, and using random sampling and centralization methods to downsample and normalize the point cloud data to improve the training effect and speed of the unexploded bomb model; S1.3, divide the unexploded bomb dataset into training set, validation set and test set; select Pointnet++ model to build the unexploded bomb recognition network model, and use the prepared unexploded bomb point cloud data to pre-train the built network model; during the training process, use the cross entropy loss function to measure the difference between the model's prediction and the true label, and use the stochastic gradient descent algorithm to update the model's parameters to minimize the loss function; S1.4, use the validation set to tune the unexploded bomb model and adjust the learning rate to improve the performance and generalization ability of the model; use the test set to evaluate the trained unexploded bomb model and calculate the accuracy of the model on unseen unexploded bomb point cloud data; S1.5, deploy the trained unexploded bomb identification network model to the central controller of the smart car, so that it can accurately detect the unexploded bombs in the point cloud.

4. The method for predicting and refining the location of unexploded bombs according to claim 2 is characterized in that: The specific steps of establishing the unexploded bomb drop point prediction simulation module and predicting the drop point coordinates of the unexploded bomb include: S2.1, collect data related to the flight trajectory of the unexploded bomb, including the size, shape, mass, and environmental conditions of the unexploded bomb; S2.2, using Newton's laws of motion Combined with the information of the initial velocity v0, mass, angle of attack, etc. of the unexploded shell, a mathematical model of the center of mass motion of the unexploded shell is established, and the equation of motion of the unexploded shell rotating around the center is obtained. Where G is the momentum moment of the unexploded bomb, and M represents the total external torque on the unexploded bomb; S2.3, the increase of air pressure will lead to the increase of air density, thus reducing the range. Use the fluid mechanics equation and the gas state equation to derive the relationship between the change of air pressure and density with altitude in the atmosphere, calculate the flight resistance and gas weight of the unexploded bomb, and further study the ballistic characteristics; S2.4, considering the change of the velocity of the center of mass with time, combined with the magnitude of the drag and gas weight, the Runge-Kutta method is used to calculate the flight velocity module of the unexploded bomb; S2.5, integrate the mathematical model of center of mass motion, the size of drag and gas weight, and the module of flight speed of unexploded bombs, and build the overall ballistic simulation model through the Simulink module in Matlab; S2.5, input the initial parameters of the unexploded bomb, display the motion trajectory of the unexploded bomb in real time during the simulation process, and predict the coordinates of the landing point of the unexploded bomb.

5. The method for predicting and refining the location of unexploded bombs according to claim 2 is characterized in that: The intelligent vehicle uses the predicted landing point distribution area obtained by S2 to autonomously navigate to the target area, make autonomous decisions, and plan the strategy for a refined search of the target area as follows: S3.1, the smart car receives the predicted landing point range from the landing point prediction simulation model, performs RTK positioning with the northeast sky as the coordinate system, obtains the surrounding environment information through the laser radar, and imports it into the cost map synchronously, combines the global path planning A* algorithm and the local path planning algorithm DWA, and autonomously moves to the vicinity of the landing point prediction area; before the smart car starts navigating, a path connecting the starting point and the target point as a whole is generated by considering the environmental map, the starting point and the target point information; the smart car starts to move, and under the guidance of the global path, the DWA local path planning algorithm is used to generate a feasible local path in the current environment in real time to avoid obstacles and ensure safe arrival at the target; S3.2, the intelligent vehicle autonomously detects unexploded bombs, uses the adaptive Monte Carlo positioning algorithm AMCL and combines it with the global cost map to conduct a refined search of the predicted drop point distribution area.

6. The method for predicting and refining the location of unexploded bombs according to claim 2 is characterized in that: The central controller of the smart car deploys the unexploded bomb point cloud training model, and autonomously identifies the unexploded bomb through the data collected in real time by the depth camera to determine the precise location of the unexploded bomb. The specific steps include: S4.1, the depth camera collects point cloud data of the target area in real time and transmits the collected data to the central controller of the smart car through a serial port connection; S4.2, the central controller of the smart car deploys the unexploded bomb point cloud training model in advance, detects the unexploded bombs in the data through the PointNet++ algorithm, and determines the precise location of the unexploded bomb through the 3D model feature matching algorithm.

7. The method for predicting and refining the location of unexploded bombs according to claim 2 is characterized in that: The precise location of the unexploded bomb obtained by S4 is transmitted to the ground station. The human-computer interaction page on the ground station displays the precise location information of the unexploded bomb in real time. The operation method is as follows: S5.1, the central controller of the smart car sends the precise location of the unexploded bomb to the ground station through the RTK communication module; S5.2, the human-computer interaction page on the ground station displays in real time the video data collected by the depth camera, the predicted simulation posture of the unexploded bomb, and the precise location information of the unexploded bomb.

8. The method for predicting and refining the location of unexploded bombs according to claim 2 is characterized in that: The virtual vehicle model of the smart car is constructed through the Unity engine digital modeling technology; during the autonomous navigation of the smart car, the CoAP communication protocol is used to ensure the real-time transmission of the sensor data on the smart car and the digital twin virtual vehicle model; the control strategy optimization module formulates a phased optimization control strategy for the internal environment of the smart car to improve the optimization strategy; the specific operation methods are as follows: S6.1, use 3D modeling software Blender to create a 3D model of the smart car, build the appearance, import the model into the Unity project, select physical properties that match the actual smart car for the virtual vehicle model to achieve a realistic appearance display; adjust material properties to optimize visual effects; add physical components to the virtual vehicle in the Unity project, set the vehicle's power system, suspension system and tires to simulate the real driving behavior of the smart car when it is running; add sensor simulation components to the virtual vehicle model, and configure its parameters and behaviors to achieve simulation and visualization of sensor data; finally, achieve free movement of the virtual vehicle model; S6.2, set up a CoAP server in the system of the digital twin virtual vehicle model, configure the IP address and port of the server, receive sensor data from the smart car, and ensure a stable network connection; configure the CoAP client in the control system of the smart car, set the target server address and port of the client, determine the format of the sensor data as JSON, ensure the consistency of the data when sending and receiving, and send the sensor data to the CoAP server of the virtual vehicle model; implement the data collection function in the sensor system of the smart car, read the sensor data regularly and prepare to send; implement the data receiving and parsing function on the CoAP server side, parse the received data into a format suitable for the virtual model and update it to the digital twin virtual vehicle model, to ensure that the model reflects the actual situation of the smart car; S6.3, in the control strategy optimization module of the smart car, firstly, the CoAP server is used to obtain the sensor data of the real smart car and the surrounding conditions of the vehicle in real time, including the location of obstacles, road conditions and the current state of the vehicle; considering the existence of obstacles, a collision module is added to the virtual vehicle model and a threshold is set. When the coefficient detected by the collision module of the virtual vehicle model is less than the preset threshold, a first instruction is generated, and when the coefficient detected by the collision module of the virtual vehicle model is greater than the preset threshold, a second instruction is generated; when the first instruction is generated, the virtual vehicle model drives normally; when the second instruction is generated, the virtual vehicle model stops immediately and issues a warning, and the warning is sent to the real smart car through the CoAP server; After receiving the warning signal, the real smart car replans the path until the first instruction is generated to safely pass the obstacle; The entire process continuously performs data collection, path planning, obstacle avoidance and strategy adjustment through the main loop, ultimately achieving efficient and safe driving of smart cars in dynamic environments.

9. The method for predicting and refining the location of unexploded bombs according to claim 8 is characterized in that: In the control strategy optimization module, the sensor data and road conditions of the real smart car are also obtained in real time through the CoAP server, including the road friction coefficient, slope, and the position and distance information of the obstacles ahead, and a speed adjustment module is designed and a speed adjustment threshold is set: when the road friction coefficient is lower than the preset value, a first instruction is generated, and when the road friction coefficient is higher than the preset value, a second instruction is generated; when the first instruction is generated, the virtual vehicle model reduces the driving speed; when the second instruction is generated, the virtual vehicle model increases the driving speed; when the slope exceeds the set value, an instruction to increase the speed is generated; the virtual vehicle model adjusts the driving speed according to these instructions, and sends the adjustment result to the real smart car through the CoAP server; After receiving the speed adjustment instruction, the real smart car adjusts its driving speed in real time and performs dynamic path planning to ensure safe and efficient driving under various road conditions; The entire process continuously performs data collection, speed adjustment and path planning through the main loop, ensuring that the smart car can flexibly respond to various challenges in a dynamic environment.

Images