A fast 3D modeling system for complex terrain in dark environments

Abstract

The present invention relates to the field of three-dimensional modeling technology, and in particular to a rapid three-dimensional modeling system for complex terrain in a dark light environment, including four modules: data acquisition and fusion, three-dimensional modeling and map generation, path planning and optimization, and system control and interface display. The data acquisition and fusion module carries a variety of sensors on a drone and uses an adaptive multimodal sensor fusion algorithm to process data; the three-dimensional modeling and map generation module generates a complex terrain three-dimensional model with geographic information coordinates by means of a dynamic hierarchical point cloud reconstruction algorithm and a three-dimensional reconstruction method based on a generative adversarial network GAN, and stores it in an octree structure; the path planning and optimization module performs feature recognition based on the three-dimensional model, and plans and optimizes the rescue route in combination with an intelligent path planning and optimization algorithm that integrates multiple factors. The present invention effectively solves the problem of rapid modeling and rescue path planning for complex terrain in a dark light environment, and improves the efficiency and success rate of rescue operations.

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional modeling, and in particular to a rapid three-dimensional modeling system for complex terrain in a dark environment. Background Art

[0002] In emergency rescue scenarios, time is life, especially in harsh conditions where low light and complex terrain are intertwined. Rescue work faces unprecedented challenges. The dim light makes it difficult for rescuers to see the entire terrain and obstacles to be identified, which greatly increases the danger of rescue operations. Complex terrain, such as mountains and ruins, not only makes it difficult for rescuers to move, but also makes it extremely difficult to determine the location of trapped people. Traditional rescue methods are often inefficient in this situation, and it is difficult to quickly and accurately formulate rescue strategies. If a three-dimensional model of the emergency rescue scene can be quickly constructed, it can provide effective data support for rescue.

[0003] The current 3D modeling technology has achieved certain results in conventional environments, but in dark environments, the accuracy of data acquisition by most sensors is affected. For example, when the light is insufficient, the image of an ordinary camera is blurred, and the texture information obtained is inaccurate, resulting in large modeling errors. In complex terrain, such as mountainous areas and ruins, the terrain is undulating and there are many obstacles. The existing modeling methods are difficult to quickly and comprehensively obtain terrain information and build high-precision 3D models. Traditional modeling technology is inefficient in processing large-scale data, and it is difficult to meet the needs of rapid modeling of complex terrain; in emergency rescue scenarios, time is life, and it is crucial to quickly and accurately grasp the terrain information of the rescue area. In the rescue of ruins after an earthquake, rescuers need to understand the structure, passages, and possible locations of survivors in the ruins. However, dark light and complex terrain increase the difficulty of rescue. The existing modeling technology cannot provide rescuers with clear and accurate 3D models in time, resulting in a lack of reliable basis for rescue route planning and delaying the rescue time. At the same time, the traditional modeling system lacks effective coordination with the rescue operation, cannot realize real-time data interaction and task allocation, and is difficult to meet the actual needs of emergency rescue work. Therefore, a fast 3D modeling system for complex terrain in a dark environment is proposed to address the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a rapid three-dimensional modeling system for complex terrain in a dark environment to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A fast 3D modeling system for complex terrain in a dark environment, comprising:

[0007] Data collection and fusion module: includes laser radar, infrared camera, depth camera and ultrasonic sensor mounted on the drone for collecting emergency rescue scenes, and uses adaptive multimodal sensor fusion algorithm to fuse the perception data collected by infrared sensor, low light sensor and laser radar to obtain fused perception data;

[0008] 3D modeling and map generation module: Based on the fused perception data provided by the data acquisition and fusion module, a 3D model of complex terrain with geographic information coordinates is generated through a dynamic hierarchical point cloud reconstruction algorithm and a 3D reconstruction method based on a generative adversarial network (GAN);

[0009] Path planning and optimization module: It performs feature recognition based on the complex terrain 3D model with geographic information coordinates, determines the coordinates of the rescue location and the mobile terminal, and plans the rescue route based on the recognition features, the coordinates of the rescue location and the mobile terminal and the attributes of the mobile terminal. It also uses a multi-factor integrated intelligent path planning and optimization algorithm to optimize the rescue route.

[0010] System control and interface display module:

[0011] It has a central command platform, which receives the three-dimensional model of complex terrain with geographic information coordinates for real-time display; it communicates data with the mobile terminal to receive the mobile terminal coordinates and send rescue routes. The mobile terminal coordinates are displayed in real time on the three-dimensional model of complex terrain with geographic information coordinates;

[0012] With mobile support, rescuers can receive real-time route guidance and task assignments through mobile devices, and interact with the central platform for data.

[0013] As a preferred solution, the calculation formula of the adaptive multimodal sensor fusion algorithm is:

[0014] ;

[0015] in, is the fused data, , and are the weights of infrared sensor, low light sensor and lidar data respectively, and ; The data collected by the infrared sensor is The data collected by the low-light sensor is Data collected for LiDAR; weights are calculated based on sensor reliability , , and environmental factors , , , the calculation formula is as follows:

[0016] ;

[0017] ;

[0018] ;

[0019] Among them, reliability Based on the historical performance data of the sensor, the environmental impact factor Based on the current dark environment, temperature, and humidity factors, the value range is 0 to 1, where 0 means no impact and 1 means complete impact.

[0020] As a preferred solution, in the dynamic hierarchical point cloud reconstruction algorithm, the hierarchical representation formula of the point cloud is:

[0021] ,in, is a point cloud collection, is the number of levels, For the Point clouds at each level Updated by the following formula:

[0022] ,in, For the The level Point cloud at the moment, For the The level Point cloud at the moment, For the The feature constraints at each level are For the The neighborhood information of each level, is the update rate, the value range is between 0 and 1; the update function The specific form is:

[0023] ,in, is a point in the neighborhood, Neighborhood points The weight of Neighborhood points The characteristic value of Based on the distance between points Calculate, that is , is a small positive number used to avoid the denominator being zero.

[0024] As a preferred solution, in the 3D reconstruction method based on the generative adversarial network GAN, the generator and the discriminator The objective function is:

[0025] ,in, For real data, is a random noise input, is the distribution of real data, is the distribution of noise, The noise is added to the generator Generating function for conversion to 3D data Judge the input for the discriminator is the probability of true data, is the regularization coefficient, is the regularization term of the generator, that is:

[0026] ,in, From the noise distribution The samples sampled in is the sample size, express Norm, used to penalize the gradient of the generator to prevent overfitting; yes About input The gradient of Indicates that the Random noise samples Input to the generator After that, the synthetic data output by the generator.

[0027] As a preferred solution, the cost function of the multi-factor fusion intelligent path planning and optimization algorithm is: ,in, is the total cost, is the path length, is the height difference of the path, is the risk factor of the path, is the smoothness of the path, is the time cost of the path; , , , and are the weight coefficients of path length, height difference, risk factor, smoothness and time cost respectively. ;

[0028] Path length Calculated by the Euclidean distance formula, for a series of points on the path :

[0029] ;

[0030] Height difference of the path Calculated as:

[0031] ,in, Indicates the path Points Coordinate value;

[0032] Risk Factors of Path According to the slope of the terrain Obstacle density and the roughness of the terrain Calculate, that is:

[0033] ,in, and is the corresponding weight, the slope Obstacle density is calculated by calculating the height difference and horizontal distance between adjacent points. The roughness of the terrain is calculated based on the number of obstacles and path length in the 3D model. Calculated through local curvature of point cloud and other information; smoothness of the path Based on the curvature of the path Calculate, that is: , Indicates the path The curvature at a point;

[0034] Time cost of the path According to the moving speed of rescue personnel or rescue equipment and path length calculate: .

[0035] As a preferred solution, feature recognition based on a complex terrain three-dimensional model with geographic information coordinates includes:

[0036] The feature extraction of the complex terrain 3D model with geographic information coordinates is performed using the following formula: ,in, is the extracted feature set, is the number of extracted features, For the Features include terrain slope, curvature, roughness, and obstacle information;

[0037] The feature extraction function is expressed as: ,in, is the feature extraction function, It is a point cloud collection of 3D models. Extract different features, for slope features :

[0038] , is the number of sampling points, For the The slope of each sampling point;

[0039] Then according to the feature set , use the following classification formula to identify the rescue location:

[0040] ,in, are the coordinates of the rescue location, is the classification function, For the The weight of the feature, is the bias term, which is trained with data and Optimize to make the classification results more accurate.

[0041] As a preferred solution, in the three-dimensional modeling and map generation module, the complex terrain three-dimensional model with geographic information coordinates is stored in an octree data structure to improve the storage efficiency and retrieval speed of the model; the division rule of the octree nodes is: recursively divide the spatial area into eight sub-areas, each sub-area corresponds to a node, until the preset division stop condition is met, which is that the number of point clouds in the sub-area is less than or equal to the set threshold.

[0042] It can be seen from the technical solutions provided by the present invention that the present invention provides a rapid three-dimensional modeling system for complex terrain in a dark environment, which has the following beneficial effects:

[0043] Accurate perception of multimodal data acquisition and fusion: The data acquisition and fusion module carries a variety of sensors on the drone, uses an adaptive multimodal sensor fusion algorithm, determines weights based on sensor reliability and environmental influencing factors, and fuses infrared, low-light, lidar and other data; this method overcomes the limitations of a single sensor, and can comprehensively collect information such as thermal radiation, distance, texture, etc. in dark and complex terrain, generate accurate perception data, and lay a solid foundation for subsequent modeling and rescue decision-making; for example, in a fire rescue scenario, the infrared sensor can identify the source of the fire, and the lidar can obtain the terrain, and after fusion, provide accurate situational awareness for the rescue command;

[0044] Efficient and accurate 3D modeling and map generation: The 3D modeling and map generation module adopts a dynamic hierarchical point cloud reconstruction algorithm and a 3D reconstruction method based on the generative adversarial network (GAN). The point cloud reconstruction algorithm optimizes the point cloud model by hierarchical representation and update, combining neighborhood and feature information. In the GAN method, the generator and the discriminator are trained adversarially to generate a high-fidelity 3D model. At the same time, the octree storage model is used to improve storage and retrieval efficiency. The complex terrain 3D models generated by these technologies have accurate geographic coordinates and rich details, providing intuitive and reliable geographic information for path planning and rescue command.

[0045] Intelligent and optimized rescue path planning: The path planning and optimization module identifies terrain features based on the three-dimensional model, determines the coordinates of the rescued and mobile terminals, plans the rescue route based on the attributes of the mobile terminal, and uses a multi-factor fusion algorithm for optimization; the cost function integrates path length, height difference, risk factor, smoothness and time cost, and balances various factors by adjusting weights; for example, when rescuing in mountainous areas, it considers factors such as slope and obstacles to plan a safe and efficient route, thereby improving the efficiency and safety of rescue operations;

[0046] Real-time interactive system control and display: The central command platform of the system control and interface display module displays the three-dimensional model in real time, communicates with the mobile terminal data, receives coordinates and sends the rescue route; the mobile terminal receives path guidance and task assignment, and realizes two-way data interaction; it enables the commander to grasp the rescue dynamics in real time and adjust the strategy in time, and the rescue personnel to clarify the tasks and routes, improve the rescue coordination and response speed, and ensure the efficient execution of the rescue operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The figure is a schematic diagram of the overall structure of a rapid three-dimensional modeling system for complex terrain in a dark environment according to the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0050] like Figure 1 As shown, an embodiment of the present invention provides a rapid three-dimensional modeling system for complex terrain in a dark environment, including a data acquisition and fusion module, a three-dimensional modeling and map generation module, a path planning and optimization module, and a system control and interface display module.

[0051] In this embodiment, the data acquisition and fusion module includes a laser radar, an infrared camera, a depth camera and an ultrasonic sensor mounted on the drone for collecting emergency rescue scenes, and uses an adaptive multimodal sensor fusion algorithm to fuse the perception data collected by the infrared sensor, the low-light sensor and the laser radar to obtain fused perception data;

[0052] Furthermore, the specific operation steps of the data acquisition and fusion module are as follows:

[0053] 1. Hardware Installation and Scene Data Collection

[0054] Drone equipment deployment:

[0055] First, according to the carrying capacity of the UAV and the size, weight and installation requirements of the sensors, select the appropriate installation location and method to fix the laser radar, infrared camera, depth camera and ultrasonic sensor on the UAV; ensure that these sensors will not interfere with each other during installation, and their field of view will not be blocked by the UAV's own structure or other sensors; for example, the laser radar can be installed on the top or bottom of the UAV to obtain terrain information at different angles; the infrared camera can be installed on the side or bottom of the fuselage as needed to observe thermal radiation sources in different directions; the depth camera is usually also installed at the bottom to accurately measure the depth information of the ground or target object; the ultrasonic sensor is generally installed near the front end or around the UAV for close-range obstacle detection;

[0056] After installation, each sensor needs to be calibrated. For LiDAR, its measurement range, angle accuracy and distance accuracy can be calibrated. You can use a target with a known distance and adjust its parameters through software settings or manually to ensure that the distance data it measures is accurate. The calibration of infrared cameras involves the temperature range calibration of thermal imaging. By aligning it with a standard heat source of known temperature, its internal parameters are adjusted to match the displayed thermal image with the actual temperature. The calibration of depth cameras requires the use of calibration tools such as checkerboards to calibrate the accuracy of its depth perception to ensure that the accuracy of depth measurement meets the requirements. Ultrasonic sensors need to adjust their detection range and sensitivity to ensure that they can accurately detect obstacles within a certain distance range.

[0057] Scene data collection:

[0058] The operator plans the flight route of the drone according to the scope of the emergency rescue scene and the area that needs to be monitored. Before the flight, it is necessary to ensure that the flight control system of the drone is working properly, including the positioning system (such as GPS or other high-precision positioning systems) can accurately obtain the location information of the drone, and the flight attitude control system can stably control the flight attitude of the drone.

[0059] When the drone flies along the predetermined route, various sensors start to work; the laser radar will emit laser beams to the surrounding environment at a certain frequency (for example, thousands of laser pulses per second), and calculate the distance to the target object based on the time difference of the laser beam reflecting back, thereby generating point cloud data, which can reflect the terrain of the scene, the location and shape of the object and other information;

[0060] The infrared camera will continuously capture the thermal radiation information in the scene and generate thermal images. In emergency rescue scenarios, it can help identify personnel, heating machinery and equipment, fire sources, etc. Different temperatures will be displayed in different colors on the thermal image, which is convenient for the subsequent location and analysis of personnel or potential sources of danger.

[0061] The depth camera measures the depth of objects in the scene through binocular vision or structured light principles to generate a depth map, in which each pixel contains the distance information from the point to the camera, which helps to establish the three-dimensional structure information of the scene;

[0062] The ultrasonic sensor works on the principle of ultrasonic propagation and reflection. It emits ultrasonic signals to the surroundings. When the signal encounters an obstacle, it will be reflected back. According to the propagation speed of ultrasonic waves and the reflection time difference, it can detect the distance of obstacles close to the drone (usually a few meters to more than ten meters). It is mainly used to prevent the drone from colliding with close obstacles during flight.

[0063] 2. Adaptive Multimodal Sensor Fusion Algorithm Execution

[0064] Sensor reliability and environmental factor assessment:

[0065] Sensor reliability assessment:

[0066] For each sensor, its reliability needs to be determined based on its historical performance data; for example, for an infrared sensor, if inaccurate temperature measurements or high image noise are common in past use, its reliability may be questioned. It may be rated as lower, such as 0.7 (assuming that the reliability range is 0 to 1, 1 means completely reliable); this can be calculated by statistically analyzing indicators such as the error rate and failure frequency of the sensor in different environments; for lidar, if it can work stably and with high measurement accuracy in various environments, its reliability is high. May be evaluated as 0.9; reliability of low-light sensors It depends on its performance in low-light environments, such as the stability of image resolution and sensitivity in low-light environments, which may be evaluated as 0.8;

[0067] Environmental Factor Assessment:

[0068] Evaluate the impact of environmental factors on sensor performance; in a dark environment, the performance of the infrared sensor may not be affected, but the performance of the low-light sensor may be improved. may be 0.2 (indicating that the environment has little impact on its performance), while It may be 0.1 (indicating that the environment promotes its performance); for lidar, its measurement accuracy will be affected in rainy and foggy weather. It may increase, such as 0.4; the evaluation of environmental factors can be determined based on empirical data or data from on-site environmental monitoring equipment. For example, environmental parameters can be measured by temperature and humidity sensors, light intensity sensors, etc., and environmental factors can be determined based on the relationship between these parameters and sensor performance;

[0069] Weight calculation:

[0070] Calculate the weight of each sensor data according to the formula mentioned above:

[0071] For infrared sensors, the weight ,in, Represents different sensors (infrared sensors , low light sensor , LiDAR ) reliability; Represents environmental factors, used to evaluate the environmental impact of different sensors (infrared sensors , low light sensor , LiDAR ) performance; Assumptions , , , , , , then calculate the denominator first:

[0072] Calculate the value of each sensor separately The values ​​are then added; for infrared sensors , ,but ; For low light sensors , , but ; For LiDAR , , but ; Add them together to get the denominator ;

[0073] Calculate infrared sensor weights

[0074] ;

[0075] Similarly, calculate the weight of the low light sensor

[0076] , ;

[0077] Calculating the weight of the LiDAR

[0078] , ;

[0079] Data Fusion:

[0080] Assume that the data collected by the infrared sensor It is a matrix containing thermal imaging information, data collected by the low-light sensor It is a low-light image matrix, the data collected by the lidar Is a point cloud data matrix; according to the fusion formula Perform data fusion;

[0081] For each data point or pixel, multiply its corresponding weight by the sensor data and add them together; for example, for information at a certain location, multiply the infrared sensor data at that location by , the data of the low light sensor at this position is multiplied by , the laser radar data at that location is multiplied by , and then add the results to get the fused data of the location; this can comprehensively consider the advantages of different sensors in different environments and obtain more comprehensive and accurate perception data;

[0082] The final fused perception data It will be a data set that integrates information from multiple sensors. It can be a new image matrix or three-dimensional point cloud data, which integrates thermal imaging, low-light images and distance information. It is more conducive to the accurate judgment and analysis of emergency rescue scenes, and provides strong support for subsequent emergency rescue operations, such as identifying dangerous areas, finding trapped people, and planning rescue routes.

[0083] In this embodiment, the 3D modeling and map generation module generates a complex terrain 3D model with geographic information coordinates based on the fused perception data provided by the data acquisition and fusion module through a dynamic hierarchical point cloud reconstruction algorithm and a 3D reconstruction method based on a generative adversarial network GAN;

[0084] In the dynamic hierarchical point cloud reconstruction algorithm, the hierarchical representation formula of the point cloud is:

[0085] ,in, is a point cloud collection, is the number of levels, For the Point clouds at each level; the point clouds at each level Are updated by the following formula:

[0086] , where Is the point cloud at the th level at time , Is the point cloud at the th level at time , Is the feature constraint of the th level, Is the neighborhood information of the th level, Is the update rate, with a value range between 0 and 1; the specific form of the update function Is:

[0087] , where Is the point within the neighborhood, Is the weight of the neighborhood point , Is the eigenvalue of the neighborhood point ; the weight Is calculated according to the distance between points, that is , Is a very small positive number used to avoid a zero denominator.

[0088] In the 3D reconstruction method based on the generative adversarial network GAN, the objective functions of the generator and the discriminator are:

[0089] , where Is the real data, Is the random noise input, Is the distribution of the real data, Is the distribution of the noise, Is the generation function for the generator to convert the noise into 3D data Is the probability for the discriminator to judge that the input is real data, Is the regularization coefficient, Is the regularization term of the generator, that is:

[0090] , where Is the sample sampled from the noise distribution , Is the number of samples, Represents Norm, used to penalize the gradient of the generator to prevent overfitting; yes About input The gradient of Indicates that the random noise samples Input to the generator After that, the synthetic data output by the generator;

[0091] Furthermore, the specific operation steps of the 3D modeling and map generation module are as follows:

[0092] Dynamic hierarchical point cloud reconstruction algorithm operation:

[0093] Point cloud hierarchical representation construction: The point cloud set is represented as ,in, is a point cloud collection, is the number of levels, For the This representation constructs a hierarchical structure of point cloud data and provides a framework for subsequent processing;

[0094] Point cloud update calculation: point clouds at all levels According to the formula Update; among them, For the The level Point cloud at the moment, For the The level Point cloud at the moment, For the The feature constraints at each level are For the The neighborhood information of each level, is the update rate, the value range is between 0 and 1; the update function ,in, is a point in the neighborhood, Neighborhood points The weight of Neighborhood points The characteristic value of Based on the distance between points Calculate, that is ( Yes With point The distance is a small positive number to avoid the denominator being zero); through this formula, combined with the characteristic constraint and neighborhood information , realize the dynamic update of point cloud and gradually optimize the point cloud model;

[0095] The 3D reconstruction method based on the generative adversarial network GAN operates as follows:

[0096] Objective function setting: The objective functions of the generator and discriminator are

[0097] ;in, For real data, is a random noise input, is the distribution of real data, is the distribution of noise, The noise is added to the generator Generating function for conversion to 3D data Judge the input for the discriminator is the probability of true data, is the regularization coefficient, is the regularization term of the generator ( From the noise distribution The samples sampled in is the sample size, express norm); This objective function provides the optimization direction for model training;

[0098] Model training and optimization: Through continuous iterative training, the generator and the discriminator compete with each other; the generator tries to generate more realistic 3D data to deceive the discriminator, while the discriminator strives to accurately distinguish between real data and generated data; during the training process, the parameters of the generator and the discriminator are adjusted according to the objective function to continuously improve the model performance and finally generate a 3D model that meets the requirements;

[0099] Model storage: The generated three-dimensional model of complex terrain with geographic information coordinates is stored in an octree data structure; the spatial area is recursively divided into eight sub-areas, each sub-area corresponds to a node, and the division is stopped when the number of point clouds in the sub-area is less than or equal to the set threshold; this storage method can improve the storage efficiency and retrieval speed of the model, and facilitate the subsequent call and processing of the model.

[0100] In this embodiment, the path planning and optimization module performs feature recognition based on the complex terrain three-dimensional model with geographic information coordinates, determines the coordinates of the rescue location and the mobile terminal, and plans the rescue route based on the recognition features, the coordinates of the rescue location and the mobile terminal coordinates and the mobile terminal attributes. At the same time, the intelligent path planning and optimization algorithm integrating multiple factors is used to optimize the rescue route.

[0101] Furthermore, the specific operation steps of the path planning and optimization module are as follows:

[0102] Feature recognition and coordinate determination:

[0103] Feature extraction: Extract features from complex terrain 3D models with geographic information coordinates. The formula is: ; Here is the extracted feature set, is the number of extracted features, For the Features, covering the slope, curvature, roughness, obstacle information and other key aspects of the terrain; taking slope feature extraction as an example, for the point cloud collection of the 3D model , slope characteristics The calculation method is ,in, is the number of sampling points, For the The slope of the sampling points; through the feature extraction function , depending on the value, you can get Extract the corresponding specific features from the data to provide basic data for subsequent analysis and decision-making;

[0104] Coordinate identification of the rescue location: Based on the extracted feature set , using the classification formula To identify the coordinates of the rescue location ; In this formula, is the classification function, For the The weight of the feature, is the bias term; through a large amount of training data and Optimization can make the classification results more accurate, so as to accurately determine the coordinates of the rescue location; at the same time, the coordinate information of the mobile terminal is obtained. These two coordinates will become the key starting and target location points for subsequent rescue route planning;

[0105] Preliminary planning of rescue routes: After determining the coordinates of the rescue site and the mobile terminal, the rescue route is preliminarily planned based on the identified features and the attributes of the mobile terminal. The attributes of the mobile terminal include the walking speed of the rescue personnel and the traffic capacity of the rescue vehicle, which determine the passability and mobility efficiency under different terrain conditions. During the planning process, based on the three-dimensional model of complex terrain, the slope of the terrain, the distribution of obstacles and other factors are fully considered to avoid areas with steep slopes that are difficult to climb and areas with dense obstacles that are inaccessible, and a theoretically feasible route is planned between the mobile terminal and the rescue site. Although this preliminarily planned route can connect the starting point and the end point, it may not be the best and needs further optimization.

[0106] Rescue route optimization:

[0107] Constructing cost function: Adopting intelligent path planning and optimization algorithm integrating multiple factors, the core of which is to construct cost function

[0108] ;in, Represents the total cost, which is a comprehensive indicator for evaluating the quality of the route; is the path length, using the Euclidean distance formula Calculation, the formula is based on a series of points on the path , accurately calculate the actual length of the path; is the height difference of the path, and the calculation formula is ,in, Indicates the path Points The coordinate value reflects the vertical fluctuation of the path. A large height difference may increase the difficulty and time of rescue. is the risk factor of the path, which is determined by the slope of the terrain Obstacle density and the roughness of the terrain Jointly decided, the calculation formula is ,in, and The potential risks faced by the path are evaluated by comprehensively considering these factors and giving corresponding weights; The smoothness of the path is calculated by calculating the curvature of the path To determine, the formula is , Indicates the path The curvature at each point and the smoothness of the path help the rescue operation proceed smoothly, reducing bumps and accidents; is the time cost of the path, based on the moving speed of the rescue personnel or rescue equipment and path length Calculation, the formula is ,Time cost is an important factor in measuring rescue efficiency; , , , and are the weight coefficients of path length, height difference, risk factor, smoothness and time cost, and satisfy ,By adjusting these weight coefficients, the importance of different factors can be flexibly adjusted according to the actual rescue needs;

[0109] Optimization calculation: By changing the node positions on the path, different path plans are generated, and the cost function value C corresponding to each plan is calculated; during the calculation process, new path combinations are continuously tried to traverse the possible path space; the C values ​​of different path plans are compared, and the path with the smallest C value is selected as the optimized rescue route; this optimization process seeks the best balance between multiple factors such as path length, height difference, risk factor, smoothness and time cost to ensure that the final rescue route can not only ensure the efficiency of the rescue operation and shorten the rescue time as much as possible, but also take into account safety, reduce the risks in the rescue process, and increase the probability of success of the rescue mission.

[0110] System control and interface display module:

[0111] It has a central command platform, which receives the three-dimensional model of complex terrain with geographic information coordinates for real-time display; it communicates data with the mobile terminal to receive the mobile terminal coordinates and send rescue routes. The mobile terminal coordinates are displayed in real time on the three-dimensional model of complex terrain with geographic information coordinates;

[0112] With mobile terminal support, rescuers can receive real-time route guidance and task assignments through mobile terminals, and interact with the central platform for data;

[0113] Furthermore, the system control and interface display module mainly realizes the complex terrain three-dimensional model display, data interaction between the mobile terminal and the central platform, and task guidance functions, providing support for rescue command and execution. The specific steps are as follows:

[0114] Central Command Platform Operation:

[0115] Model reception and display: The central command platform receives the complex terrain 3D model with geographic information coordinates generated by the 3D modeling and map generation module; obtains the model data through a special data interface with a specific data transmission protocol; after receiving the data, the complex terrain 3D model is displayed in real time on the command platform screen using graphics rendering technology; this enables the command personnel to intuitively and comprehensively understand the terrain of the rescue scene, including information such as mountains, rivers, and building distribution, providing a basic basis for command decision-making;

[0116] Data interaction between mobile terminals: The central command platform establishes a stable data communication link with the mobile terminal, for example, through 4G, 5G or dedicated wireless networks. On the one hand, the platform receives the coordinate information sent by the mobile terminal in real time. The mobile terminal obtains its own geographic location coordinates through built-in positioning modules, such as GPS, Beidou, etc., and sends the coordinate data to the central command platform at a certain time interval (such as once per second). After receiving the coordinates, the platform will mark and display them in real time on the complex terrain three-dimensional model, so that the commander can grasp the location dynamics of rescue personnel or equipment at any time. On the other hand, when the path planning and optimization module generates a rescue route, the central command platform sends the rescue route data to the mobile terminal. The rescue route data contains the coordinate information of a series of path nodes and related navigation instructions. The platform pushes this information to the mobile terminal according to the established data format and communication protocol to provide accurate action guidance for rescue personnel.

[0117] Mobile terminal operation:

[0118] Information reception and display: The mobile terminal continuously monitors the data sent by the central command platform; when receiving path guidance information, the mobile terminal presents the rescue route to the rescuers in a visual way through the built-in map application or a specially developed display interface; the route is usually drawn on the map in eye-catching colors, and is equipped with a voice navigation prompt function to provide rescuers with clear direction guidance; if task assignment information is received, the mobile terminal will pop up a task details window on the screen, displaying key information such as the specific content, objectives, and time requirements of the task, so that rescuers can clearly understand their own responsibilities and task arrangements;

[0119] Data feedback and interaction: When performing their tasks, if rescue personnel discover new situations, such as encountering sudden obstacles or finding new trapped persons, they can input relevant information through the interactive interface of the mobile terminal and feed it back to the central command platform. For example, rescue personnel click the "Feedback" button on the mobile terminal interface, enter text descriptions or take photos and videos of the scene, and then send this information to the central command platform through the data communication link. The mobile terminal can also receive real-time dispatch instructions from the central command platform, such as changing the route, adjusting the task priority, etc., and respond and execute them in a timely manner to ensure that the rescue operation is highly consistent with the command decision.

[0120] In this embodiment, in the three-dimensional modeling and map generation module, the generated three-dimensional model of complex terrain with geographic information coordinates is stored in an octree data structure to improve the storage efficiency and retrieval speed of the model; the division rule of the octree nodes is: the spatial area is recursively divided into eight sub-areas, each sub-area corresponds to a node, until the preset division stop condition is met, which is that the number of point clouds in the sub-area is less than or equal to the set threshold. The specific operation steps are as follows:

[0121] Initialize the octree: take the entire spatial region containing the complex terrain 3D model as the root node of the octree; determine the boundaries of the spatial region, such as setting its minimum and maximum coordinate values ​​in the 3D coordinate system and , in order to define the spatial range represented by the root node; at the same time, set the point cloud number threshold in the division stop condition, which needs to be determined according to the model accuracy requirements and actual storage requirements, such as setting it to 10 point clouds;

[0122] Recursively partition the space into regions:

[0123] Division operation: For the node that needs to be divided, the spatial area it represents is evenly divided into eight sub-areas; the boundary coordinates of each sub-area are calculated, for example, Direction, the interval Divided into and , , The same goes for the direction; create a corresponding octree node for each sub-area, and establish an association relationship between the parent node and the child node;

[0124] Point cloud allocation: traverse the point cloud data of the complex terrain 3D model and determine which sub-area each point cloud belongs to; according to the coordinates of the point cloud The relationship between the coordinates of the boundaries of each sub-region is determined, such as and and , then the point cloud belongs to the corresponding sub-region; the point cloud is assigned to the corresponding sub-node, and each node stores the point cloud set belonging to the region;

[0125] Determine the stop condition of division: check the number of point clouds in each newly created child node; count the number of point clouds contained in the child node. If the number is less than or equal to the set threshold, the child node will no longer be divided and become a leaf node of the octree; if the number of point clouds in the child node is greater than the threshold, continue to recursively divide the spatial area represented by the child node, and repeat the above steps of dividing the spatial area, allocating point clouds and judging the stop condition until all nodes meet the division stop condition;

[0126] Model storage and retrieval preparation: After the octree is constructed, the point cloud data of the complex terrain 3D model is stored in an octree structure; when the model data needs to be retrieved, it can be based on the spatial position information, starting from the root node, by comparing the relationship between the target position and the spatial area represented by each node, quickly locate the leaf node containing the target position, obtain the point cloud data stored in the node, achieve efficient retrieval, and improve the storage and access efficiency of complex terrain 3D models.

[0127] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fast 3D modeling system for complex terrain in a dark environment, characterized by: include: Data collection and fusion module: includes laser radar, infrared camera, depth camera and ultrasonic sensor mounted on the drone for collecting emergency rescue scenes, and uses adaptive multimodal sensor fusion algorithm to fuse the perception data collected by infrared sensor, low light sensor and laser radar to obtain fused perception data; 3D modeling and map generation module: Based on the fused perception data provided by the data acquisition and fusion module, a 3D model of complex terrain with geographic information coordinates is generated through a dynamic hierarchical point cloud reconstruction algorithm and a 3D reconstruction method based on a generative adversarial network (GAN); Path planning and optimization module: It performs feature recognition based on the complex terrain 3D model with geographic information coordinates, determines the coordinates of the rescue location and the mobile terminal, and plans the rescue route based on the recognition features, the coordinates of the rescue location and the mobile terminal and the attributes of the mobile terminal. It also uses a multi-factor integrated intelligent path planning and optimization algorithm to optimize the rescue route. System control and interface display module: It has a central command platform, which receives the three-dimensional model of complex terrain with geographic information coordinates for real-time display; it communicates data with the mobile terminal to receive the coordinates of the mobile terminal and send the rescue route. The coordinates of the mobile terminal are displayed in real time on the three-dimensional model of complex terrain with geographic information coordinates; With mobile terminal support, rescuers can receive real-time route guidance and task assignments through mobile terminals, and interact with the central platform for data; The calculation formula of the adaptive multimodal sensor fusion algorithm is: ； in, is the fused data, , and are the weights of infrared sensor, low light sensor and lidar data respectively, and ; The data collected by the infrared sensor is The data collected by the low-light sensor is Data collected for LiDAR; weights are calculated based on sensor reliability , , and environmental factors , , , the calculation formula is as follows: ； ； ； Among them, reliability , , Based on the historical performance data of the sensor, the environmental impact factor , , Based on the current dark environment, temperature, and humidity factors, the value range is 0 to 1, where 0 means no impact and 1 means complete impact.

2. The rapid 3D modeling system for complex terrain in a dark environment according to claim 1, characterized in that: In the dynamic hierarchical point cloud reconstruction algorithm, the hierarchical representation formula of the point cloud is: ,in, is a point cloud collection, is the number of levels, For the Point clouds at each level Updated by the following formula: ,in, For the The level Point cloud at the moment, For the The level Point cloud at the moment, For the The feature constraints at each level are For the The neighborhood information of each level, is the update rate, the value range is between 0 and 1; the update function The specific form is: ,in, is a point in the neighborhood, Neighborhood points The weight of Neighborhood points The characteristic value of Based on the distance between points Calculate, that is , is a small positive number used to avoid the denominator being zero.

3. The rapid 3D modeling system for complex terrain in a dark environment according to claim 1, characterized in that: In the 3D reconstruction method based on the generative adversarial network GAN, the generator and the discriminator The objective function is: ,in, For real data, is a random noise input, is the distribution of real data, is the distribution of noise, The noise is added to the generator Converted to a generating function for three-dimensional data, Judge the input for the discriminator is the probability of true data, is the regularization coefficient, is the regularization term of the generator, that is: ,in, From the noise distribution The samples sampled in is the sample size, express Norm, used to penalize the gradient of the generator to prevent overfitting; yes About input The gradient of Indicates that the random noise samples Input to the generator After that, the synthetic data output by the generator.

4. The rapid 3D modeling system for complex terrain in a dark environment according to claim 1, characterized in that: The cost function of the multi-factor fusion intelligent path planning and optimization algorithm is: ,in, is the total cost, is the path length, is the height difference of the path, is the risk factor of the path, is the smoothness of the path, is the time cost of the path; , , , and are the weight coefficients of path length, height difference, risk factor, smoothness and time cost respectively. ; Path length Calculated by the Euclidean distance formula, for a series of points on the path ： ； Height difference of the path Calculated as: ,in, Indicates the path Points Coordinate value; Risk Factors of Path According to the slope of the terrain Obstacle density and the roughness of the terrain Calculate, that is: ,in, and is the corresponding weight, the slope Obstacle density is calculated by calculating the height difference and horizontal distance between adjacent points. The roughness of the terrain is calculated based on the number of obstacles and path length in the 3D model. Calculated by the local curvature of the point cloud; the smoothness of the path Based on the curvature of the path Calculate, that is: , Indicates the path The curvature at a point; Time cost of the path According to the moving speed of rescue personnel or rescue equipment and path length calculate: .

5. The rapid 3D modeling system for complex terrain in a dark environment according to claim 1, characterized in that: Feature recognition based on a complex terrain 3D model with geographic information coordinates includes: The feature extraction of the complex terrain 3D model with geographic information coordinates is performed using the following formula: ,in, is the extracted feature set, is the number of extracted features, For the Features include terrain slope, curvature, roughness, and obstacle information; The feature extraction function is expressed as: ,in, is the feature extraction function, It is a point cloud collection of 3D models. Extract different features, for slope features : , is the number of sampling points, For the The slope of each sampling point; Then according to the feature set , use the following classification formula to identify the rescue location: ,in, are the coordinates of the rescue location, is the classification function, For the The weight of the feature, is the bias term, which is trained with data and Optimize to make the classification results more accurate.

6. The rapid 3D modeling system for complex terrain in a dark environment according to claim 1, characterized in that: In the three-dimensional modeling and map generation module, the generated three-dimensional model of complex terrain with geographic information coordinates is stored in an octree data structure to improve the storage efficiency and retrieval speed of the model; the division rule of the octree nodes is: recursively divide the spatial area into eight sub-areas, each sub-area corresponds to a node, until the preset division stop condition is met, which is that the number of point clouds in the sub-area is less than or equal to the set threshold.

Images