Multi-form intelligent device collaborative data acquisition method

Abstract

The invention provides a multi-form intelligent equipment collaborative data acquisition method. The method comprises the steps that a ground mobile robot moves in a narrow terrain and acquires three-dimensional point cloud data, and an unmanned aerial vehicle flies in a low altitude and acquires image data. The comprehensive support platform monitors the residual electric quantity of the ground robot and the storage capacity of the unmanned aerial vehicle in real time, and broadcasts the coordinates of the gathering point when the residual electric quantity is lower than a threshold value. The ground robot goes to the gathering point along the optimal path to receive wireless charging, and the unmanned aerial vehicle flies to the gathering point linearly and transmits image data. And the support platform fuses the point cloud and the image data to generate an environment model, and executes subsequent operation according to the task type. According to the invention, efficient collection and fusion of multi-dimensional data in a complex environment can be realized, and the cruising ability and task execution efficiency of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robot collaborative operations, and more specifically, to a method for collaborative data collection of multi-modal intelligent devices. Background Art

[0002] In today's collaborative data collection landscape involving intelligent devices, the rapid development of robotics, sensor, and communication technologies has led to an increasing number of intelligent devices being used for data collection tasks in complex environments. Ground mobile robots, with their excellent adaptability to terrain and high payload capacity, are often used for data collection in complex terrain. Equipped with sensors such as lidar, they can acquire high-precision three-dimensional point cloud data, providing essential data support for tasks such as topographic mapping and environmental monitoring. Drones, with their fast and flexible flight capabilities, are ideal for data collection at low altitudes. Their onboard aerial cameras can capture image data over a wide area, offering significant advantages for rapidly acquiring regional information.

[0003] However, while ground-based mobile robots can move stably and collect data in narrow terrain, their movement speed is relatively slow, and when their battery runs low, they need to return to a charging station for recharging, which interrupts data collection and impacts mission continuity. While drones can move quickly and collect image data, their storage capacity is limited. When storage space is insufficient, they need to return to a base station for data transmission, which also leads to mission interruptions. Furthermore, the type of data collected by a single robot is relatively simple, making it difficult to fully reflect the multidimensional information of complex environments. To overcome these limitations and improve the efficiency and quality of data collection, researchers have begun exploring collaborative models for multi-modal intelligent devices. By combining ground-based mobile robots and drones, the strengths of each can be fully utilized, achieving more efficient and comprehensive data collection. However, in the existing technology, systematic approaches for collaborative data collection with multi-modal intelligent devices are still insufficient.

[0004] In the process of implementing the embodiments of the present invention, there are at least the following problems or defects in the existing technology: First, there is a lack of effective power and storage capacity monitoring mechanism and corresponding collaborative scheduling strategy, which makes it impossible to perform effective collaborative operations in time when the power or storage capacity is insufficient, resulting in an increased risk of task interruption; second, the ground optimal path planning is not accurate enough, and factors such as terrain passability are not fully considered, which affects the mobility efficiency of the ground mobile robot; third, the data fusion method is not perfect enough, and different types of data cannot be efficiently and accurately fused to generate a high-quality environmental model; fourth, for different task types, there is a lack of targeted subsequent operation process design, and the value of collaborative data collection cannot be fully utilized. Summary of the Invention

[0005] The present invention provides a method for collaborative data collection of multi-modal intelligent devices, comprising:

[0006] The ground mobile robot moves in a narrow terrain area and collects 3D point cloud data through LiDAR;

[0007] The drone moves in a low-altitude flight area and collects image data through an aerial camera;

[0008] The integrated support platform resides in the open area and monitors the remaining power of the ground mobile robot in real time. d and the remaining storage capacity C of the drone u ;

[0009] When E d <E th or C u <C th When the integrated support platform broadcasts the coordinates of the assembly point (x m ,y m );

[0010] The ground mobile robot moves along the optimal path on the ground to (x m ,y m ) and receive wireless charging, while the drone moves along a straight path in the air to (x m ,y m ) and transmit image data;

[0011] The comprehensive support platform fuses 3D point cloud data and image data to generate an environmental model and performs subsequent operations based on the task type.

[0012] Furthermore, the coordinates of the broadcast assembly point (x m ,y m )include:

[0013] Get the current position coordinates of the ground mobile robot in real time (x d ,y d ), the current position coordinates of the drone (x u ,y u ) and the current position coordinates of the integrated support platform (x b ,y b );

[0014] Calculating the power weight factor of a ground mobile robot

[0015]

[0016] in is the maximum power of the ground mobile robot, E d The remaining power of the ground mobile robot; calculate the storage weight factor of the drone

[0017]

[0018] in is the maximum storage capacity of the UAV, C u The remaining storage capacity of the drone;

[0019] Set the comprehensive support platform position weight constant w b =0.2;

[0020] Calculate the coordinates of the assembly point:

[0021]

[0022] Furthermore, the generation of the optimal ground path includes:

[0023] Obtain a digital elevation map of the work area;

[0024] Extract the coordinates of the ground mobile robot's current position to the assembly point (x m ,y m ) between the candidate paths;

[0025] Calculate the terrain passability cost of each path:

[0026]

[0027] Among them, α is the slope influence coefficient, β is the obstacle density influence coefficient, is the slope value of path segment i, ρ o is the obstacle density coefficient;

[0028] Select Cost g The path with the smallest value is taken as the optimal ground path.

[0029] Furthermore, the receiving wireless charging includes:

[0030] Comprehensive support platform to detect the current power E of ground mobile robots d and drones currently store C u ;

[0031] Calculate the charging power of a ground mobile robot:

[0032]

[0033] Calculate the drone charging power:

[0034]

[0035] in, The maximum charging power of the comprehensive support platform, E is the maximum power of the ground mobile robot d E is the remaining power of the ground mobile robot C is the maximum storage capacity of the UAV u C is the remaining storage capacity of the UAV

[0036] P is transmitted to the ground mobile robot through the electromagnetic induction coil d P is transmitted to the UAV u P is transmitted to the UAV.

[0037] Further, the fusion of the three-dimensional point cloud data and the image data comprises:

[0038] Performing voxel grid down-sampling on the three-dimensional point cloud data collected by the ground mobile robot:

[0039]

[0040] wherein V grid is the voxel grid, v k is a voxel in the voxel grid, p i is the original point cloud coordinate, and r is the voxel resolution;

[0041] Extracting feature descriptors of the down-sampled point cloud;

[0042] Matching the SIFT feature points of the UAV aerial image;

[0043] Solving the optimal rigid transformation matrix T of the point cloud to the image:

[0044]

[0045] wherein T is the rigid transformation matrix of the point cloud to the image, p j is the point cloud feature point, q k is the image feature point, and π is the projection function.

[0046] Further, the generating of the environment model comprises: constructing the Poisson equation:

[0047]

[0048] wherein Δ is the Laplace operator, Φ is the three-dimensional indicator function, is the divergence operator, is the point cloud normal vector field;

[0049] Solving the three-dimensional indicator function Φ;

[0050] Extracting the isosurface M 3d ={x|Φ(x)=∈} as the environment model surface.

[0051] Furthermore, executing subsequent operations according to the task type includes:

[0052] When the task type is terrain surveying:

[0053] Extracting a set of contour lines from an environment model

[0054] Generate digital elevation maps and output them to the integrated support platform display screen;

[0055] When the task type is environmental monitoring:

[0056] Analyzing vegetation indices from aerial images

[0057]

[0058] NDVI is the normalized vegetation index, NIR is the near-infrared band reflectance, and R is the red band reflectance;

[0059] Generate ecological assessment reports and upload them through the 5G module of the integrated support platform;

[0060] When the task type is material delivery:

[0061] Calculate the optimal transport route

[0062] path * =arg min∫(w1·slope+w2·curv)ds

[0063] where path * is the optimal transportation path, w1 is the slope weight coefficient, w2 is the curvature weight coefficient, slope is the path slope, curv is the path curvature, and ds is the path element.

[0064] Furthermore, the calculating of the optimal transport path includes:

[0065] Construct terrain cost map G = (V, E);

[0066] Define vertex v i The cost function is:

[0067]

[0068] Among them, c(v i ) is the vertex v i The cost function value is α, which is the slope influence coefficient, and β is the direction deviation influence coefficient. For vertex v i The slope value at n(v i ) is the vertex v i Normal vector at n target is the normal vector of the target direction;

[0069] A* algorithm is used to search for the minimum cost path from the starting point to the end point.

[0070] Furthermore, it also includes dynamic collaborative control:

[0071] Real-time detection of terrain roughness parameter ξ by ground mobile robot d ;

[0072] When d >ξ max When ξ max is the terrain ruggedness threshold;

[0073] The integrated support platform controls the UAV to fly to the current position coordinates of the ground mobile robot;

[0074] The drone collects supplementary data through downward-looking cameras and transmits it to the integrated support platform;

[0075] The integrated support platform fuses the supplementary data with the ground mobile robot point cloud data.

[0076] Furthermore, the data transmission adopts a layered protocol:

[0077] The LoRa protocol is used to transmit point cloud data between the ground mobile robot and the integrated support platform;

[0078] The Wi-Fi 6 protocol is used to transmit image data between the drone and the integrated support platform;

[0079] The comprehensive support platform transmits environmental model data via the 5G network.

[0080] The above embodiments of the present invention have at least the following beneficial effects:

[0081] 1. Through the collaborative operation mechanism of ground mobile robots and drones, full coverage data collection in narrow terrain and low-altitude areas is achieved, solving the problem of incomplete data collection by a single device in complex environments. At the same time, the intelligent scheduling function of the integrated support platform is used to ensure that the equipment can return to the assembly point in time when the battery is low or the storage is full, thereby improving the system's work efficiency and continuity.

[0082] 2. Dynamic weights are used to calculate the coordinates of the assembly point, taking into account the power of the ground robot, the storage capacity of the drone, and the location of the support platform. This optimizes the equipment recovery path, reduces energy waste and time loss, solves the path redundancy problem caused by traditional fixed assembly points, and improves the system's response speed and resource utilization.

[0083] 3. Through the high-precision fusion of 3D point cloud data and aerial images, a more accurate environmental model is constructed, and subsequent operations such as terrain mapping, environmental monitoring, or material distribution are automatically performed in combination with the task type. This solves the problems of insufficient data fusion accuracy and poor task adaptability in traditional methods, and enhances the intelligence level and practical application value of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily apparent by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present invention are shown by way of example and not limitation, in which:

[0085] Figure 1 A flowchart of a method for collaborative data collection by multi-modal intelligent devices provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0086] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0087] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0088] It should be noted that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0089] Reference below Figure 1 , Figure 1 This is a flow chart of a method for collaborative data collection of multi-modal intelligent devices provided by an embodiment of the present invention. Figure 1 As shown, a method for collaborative data collection of multi-modal intelligent devices includes:

[0090] S1, a ground mobile robot moves in a narrow terrain area and collects 3D point cloud data through LiDAR;

[0091] S2, the drone moves in the low-altitude flight area and collects image data through the aerial camera;

[0092] S3. The integrated support platform resides in the open area and monitors the remaining power of the ground mobile robot in real time. d and the remaining storage capacity C of the drone u ;

[0093] S4. When E d <E th or C u <C th When the integrated support platform broadcasts the coordinates of the assembly point (x m ,y m );

[0094] S6, the ground mobile robot moves along the optimal path on the ground to (x m ,y m ) and receive wireless charging, while the drone moves along a straight path in the air to (x m ,y m ) and transmit image data;

[0095] S7. The integrated support platform integrates 3D point cloud data and image data to generate an environmental model and performs subsequent operations based on the task type.

[0096] It should be noted that a ground mobile robot is a robot capable of traveling and performing tasks on the ground. It is typically equipped with a mobile mechanism such as wheels or tracks, enabling stable operation in complex ground environments. LiDAR is a sensor that uses lasers to measure distance. It emits a laser beam and receives the reflected signal, generating three-dimensional point cloud data of the surrounding environment. This data contains the location information of each point in the environment and can be used to construct a three-dimensional model of the environment. In confined terrain areas, such as mountain trails, narrow urban streets, or ruins, ground mobile robots, with their small size and excellent maneuverability, can penetrate deep into these areas to collect data, providing fundamental data support for subsequent environmental analysis and mission planning.

[0097] The comprehensive support platform serves as the intelligent core of the entire system, bearing the core computing power integration and collaborative scheduling functions. The comprehensive support platform not only monitors the key operating parameters of the ground mobile robot and the unmanned aerial vehicle in real time, such as power, storage capacity, etc., but also efficiently integrates the computing power resources of each terminal through dynamic calculation of the cluster node coordinates, planning of the optimal path, allocation of the charging power, etc. The computing power resource management mechanism can dynamically adjust the task priority and resource allocation according to the real-time state of the equipment, for example, when the power is low, the computing power demand for charging path planning is prioritized, and when the storage is full, the processing efficiency of data transmission is optimized. At the same time, the comprehensive support platform generates an environment model by fusing three-dimensional point cloud and image data, and automatically matches the corresponding algorithm module to execute subsequent tasks according to the task type, realizing the full-process computing power collaboration from data acquisition to decision execution, and effectively improving the intelligent level and task execution efficiency of the entire system.

[0098] Specifically, the ground mobile robot moves in a narrow terrain area and collects three-dimensional point cloud data through a laser radar. The narrow terrain area refers to areas with a small width and limited space, such as mountain paths less than two meters wide or narrow alleys in cities. In such terrain, the ground mobile robot needs to have good mobility and flexibility to adapt to complex road conditions. Laser radar is a high-precision sensor that calculates distance by emitting laser pulses and measuring their reflection time, thereby obtaining the three-dimensional structure of the surrounding environment. This data is crucial for building a high-precision environment model, as it provides detailed terrain and obstacle information. Three-dimensional point cloud data refers to the three-dimensional coordinate information of each point in the environment measured by a laser radar. These data are usually presented in the form of a point cloud, with each point containing its x, y, z coordinate values in space. The scanning frequency of the laser radar can be adjusted according to actual needs, such as increasing the scanning frequency in complex terrain to obtain more detailed data.

[0099] Specifically, the ground mobile robot can be a "robot dog", which is a bionic quadruped mobile robot with high mobility, environmental perception, and data acquisition capabilities, suitable for narrow and complex terrain. Its bionic biomechanics quadruped structure can move flexibly in rough terrain and other areas, carrying sensors such as laser radars to collect three-dimensional point cloud data. The comprehensive support platform can be a "robot horse", which is an intelligent device with multi-modal mobility, energy supply, and data processing functions, usually in the form of a bionic quadruped or wheeled structure, capable of stable residence or movement in complex terrain. It can integrate large-capacity batteries and wireless charging modules to supply power to other devices, serve as a data hub for real-time processing, storage, and communication, and fuse multi-source data to generate an environment model. It can also monitor the status of collaborative devices through sensors and dynamically plan task processes.

[0100] Preferably, the lidar sensor equipped with the ground mobile robot can achieve millimeter-level measurement accuracy. The lidar's scanning frequency can be adjusted according to actual needs; for example, the scanning frequency can be increased in complex terrain to obtain more detailed data. During data collection, the ground mobile robot can use a preset path planning algorithm, such as the A* algorithm or the Dijkstra algorithm, to determine the optimal movement path to ensure efficient data collection in confined terrain. Furthermore, the 3D point cloud data collected by the lidar can be optimized through preprocessing steps such as data filtering and downsampling to remove noise points and reduce the data volume, thereby improving the efficiency of subsequent data processing.

[0101] In some embodiments, the broadcast rendezvous point coordinates (x m ,y m )include:

[0102] Get the current position coordinates of the ground mobile robot in real time (x d ,y d ), the current position coordinates of the drone (x u ,y u ) and the current position coordinates of the integrated support platform (x b ,y b );

[0103] Calculating the power weight factor of a ground mobile robot

[0104]

[0105] in is the maximum power of the ground mobile robot, E d The remaining power of the ground mobile robot; calculate the storage weight factor of the drone

[0106]

[0107] in is the maximum storage capacity of the UAV, C u The remaining storage capacity of the drone;

[0108] Set the comprehensive support platform position weight constant w b =0.2;

[0109] Calculate the coordinates of the assembly point:

[0110]

[0111] It should be noted that the process of broadcasting the assembly point coordinates involves acquiring the current position coordinates of the ground mobile robots, drones, and integrated support platforms in real time, and calculating the assembly point location based on these coordinates and their respective power and storage capacity status. The assembly point coordinates referred to here refer to the specific location that the ground mobile robots and drones need to reach when their power or storage capacity is insufficient for charging or data transmission. The power weighting factor and storage weighting factor are calculated based on the respective power and storage capacity status and are used to determine the assembly point location to ensure that the robots can efficiently complete charging and data transmission tasks. The integrated support platform position weight constant is a fixed parameter used to balance the weight of the integrated support platform location in the assembly point calculation. Using these parameters, an optimal assembly point location can be calculated, allowing the robots to quickly reach the assembly point for charging or data transmission when their power or storage capacity is insufficient.

[0112] Specifically, the process of broadcasting the assembly point coordinates involves the following key steps: First, the current position coordinates of the ground mobile robot, the drone, and the integrated support platform must be acquired in real time. These coordinates can be obtained using the positioning systems on the robot and platform, such as GPS or the BeiDou navigation system. Second, the ground mobile robot's power weighting factor is calculated. This factor is calculated based on the robot's maximum power and remaining power, reflecting the impact of the remaining power on the assembly point location. Specifically, the power weighting factor is calculated by subtracting the remaining power from the maximum power, then dividing it by the maximum power. The closer this value is to 1, the lower the power level, and the greater the impact on the assembly point location. Similarly, the drone's storage weighting factor is calculated based on the drone's maximum storage capacity and remaining storage capacity, reflecting the impact of remaining storage capacity on the assembly point location. The storage weighting factor is calculated similarly to the power weighting factor: subtracting the remaining storage capacity from the maximum storage capacity, then dividing it by the maximum storage capacity. Finally, the integrated support platform's position weight constant is set. This is a fixed parameter used to balance the weight of the integrated support platform's position in the assembly point calculation. Using these parameters, the coordinates of the assembly point are calculated according to a specific formula that takes into account the positions of the robot and platform as well as the power and storage capacity status to determine an optimal assembly point location.

[0113] Preferably, the process of broadcasting the rendezvous point coordinates can be further refined by first ensuring that the positioning system has a high enough accuracy to obtain accurate current position coordinates when calculating the rendezvous point coordinates. For the calculation of the power weight factor, a threshold can be set, and when the remaining power is below the threshold, the value of the power weight factor is increased to encourage the robot to reach the rendezvous point faster for charging. Similarly, for the storage weight factor, a threshold of storage capacity can also be set, and when the remaining storage capacity is below the threshold, the value of the storage weight factor is increased to encourage the robot to reach the rendezvous point faster for data transmission. In practical applications, these thresholds can be adjusted according to the specific tasks and environmental conditions of the robot.

[0114] Further, the comprehensive support platform position weight constant can be adjusted according to actual conditions to better balance the relationship between the rendezvous point position and the comprehensive support platform position. Through these refined steps and parameter adjustments, the rendezvous point coordinates can be calculated more accurately, improving the coordination efficiency and reliability of the entire system.

[0115] In some embodiments, the generation of the ground optimal path includes:

[0116] Obtaining a digital elevation map of the work area;

[0117] Extracting a set of candidate paths between the current position of the ground mobile robot and the rendezvous point coordinates (x m ,y m );

[0118] Calculating the terrain passability cost of each path:

[0119]

[0120] where α is the slope influence coefficient, β is the obstacle density influence coefficient, is the slope value of path segment i, and ρ o is the obstacle density coefficient.

[0121] Selecting the path with the minimum Cost g value as the ground optimal path.

[0122] It should be noted that the generation of the optimal ground path is based on the digital elevation map of the operating area. The optimal path is selected by extracting a set of candidate paths and calculating the terrain passability cost of each path. The optimal ground path here refers to the best moving path between the current position of the ground mobile robot and the coordinates of the assembly point. This path takes into account factors such as the complexity of the terrain and the distribution of obstacles. The digital elevation map is a kind of geographic information system (GIS) data. It contains the elevation information of the terrain and is used to describe the undulations and slopes of the terrain. The terrain passability cost is a quantitative indicator that comprehensively considers factors such as slope and obstacle density, and is used to evaluate the feasibility and safety of the path. By calculating the terrain passability cost of each candidate path, the path with the lowest cost can be selected as the optimal ground path, thereby improving the mobility efficiency and safety of the ground mobile robot.

[0123] Specifically, the process of generating the optimal ground path involves the following key steps: First, a digital elevation map of the work area is obtained. This map contains terrain elevation information and is generated using satellite remote sensing, aerial photography, or other geographic mapping technologies. Second, a set of candidate paths is extracted between the ground mobile robot's current location and the coordinates of the assembly point. These candidate paths can be extracted from the digital elevation map using path planning algorithms such as the A* algorithm or the Dijkstra algorithm. Next, the terrain passability cost of each path is calculated. This cost is composed of the slope influence coefficient and the obstacle density influence coefficient. The slope influence coefficient is a weight parameter that measures the impact of slope on path passability; the obstacle density influence coefficient is a weight parameter that measures the impact of obstacle density on path passability. The slope value represents the slope of the path segment, and the obstacle density coefficient represents the density of obstacles along the path. By calculating these parameters, the terrain passability cost of each path is determined, and the path with the lowest cost is selected as the optimal ground path.

[0124] Preferably, the generation of the optimal ground path can be further refined in the following ways: when obtaining a digital elevation map, ensure that the resolution of the map is high enough to accurately reflect the details of the terrain. When extracting a set of candidate paths, multiple path planning algorithms can be set to ensure that the extracted path set has sufficient diversity and coverage. When calculating the terrain passability cost, the values ​​of the slope influence coefficient and the obstacle density influence coefficient can be adjusted according to the actual mission requirements. For example, in mountainous tasks, the value of the slope influence coefficient can be increased to pay more attention to the impact of the slope on the passability of the path; in urban environments, the value of the obstacle density influence coefficient can be increased to pay more attention to the impact of obstacles on the passability of the path. In addition, the settings of these parameters can be optimized through field tests and simulation experiments to ensure that the generated optimal ground path is both efficient and safe.

[0125] In some embodiments, receiving wireless charging includes:

[0126] Comprehensive support platform to detect the current power E of ground mobile robots d and drones currently store C u ;

[0127] Calculate the charging power of a ground mobile robot:

[0128]

[0129] Calculate the drone charging power:

[0130]

[0131] in, The maximum charging power of the comprehensive support platform, is the maximum power of the ground mobile robot, E d is the remaining power of the ground mobile robot, is the maximum storage capacity of the UAV, C u The remaining storage capacity of the drone;

[0132] Transmit P to the ground mobile robot through electromagnetic induction coil d Power, transmitted to the drone P u power.

[0133] It's important to note that the wireless charging process involves an integrated support platform detecting the current power and storage capacity of ground mobile robots and drones, calculating the charging power based on this information, and then transmitting the corresponding power to the robot via an electromagnetic induction coil. The integrated support platform is a multifunctional device that monitors the robot's power and storage capacity in real time and provides wireless charging services. The charging power is calculated based on the robot's power and storage capacity status to ensure fast and efficient charging or data transfer when power or storage capacity is insufficient.

[0134] Specifically, the process of accepting wireless charging includes the following key steps: first, the integrated support platform detects the current power of the ground mobile robot and the current storage capacity of the unmanned aerial vehicle. These information can be transmitted in real time to the integrated support platform through sensors and communication modules on the robot. Second, calculate the charging power of the ground mobile robot, which is determined by the maximum charging power of the integrated support platform and the power state of the ground mobile robot. Specifically, the charging power is calculated according to the maximum power and the remaining power of the ground mobile robot, and the maximum storage capacity and the remaining storage capacity of the unmanned aerial vehicle. Finally, the calculated charging power is transmitted to the ground mobile robot through the electromagnetic induction coil to realize wireless charging. The electromagnetic induction coil is a device that can convert electrical energy into a magnetic field, and then transmit energy to the receiving end through the magnetic field, which is used here to realize wireless energy transmission between the robot and the integrated support platform.

[0135] Preferably, the process of accepting wireless charging can be further refined in the following way: when detecting power and storage capacity, ensure the accuracy and reliability of the sensor to obtain accurate data. When calculating the charging power, the allocation ratio of the maximum charging power can be adjusted according to the actual task requirements. For example, if the power of the ground mobile robot is more critical, the proportion of charging power allocated to the ground mobile robot can be appropriately increased.

[0136] Further, the calculation formula of the charging power can be optimized through experiments and simulations to ensure efficient charging under different power and storage capacity states. During the wireless charging process, a charging efficiency monitoring mechanism can also be introduced to adjust the charging power in real time to improve charging efficiency and reduce energy loss.

[0137] In some embodiments, the fusion of three-dimensional point cloud data and image data includes:

[0138] Perform voxel grid down-sampling on the three-dimensional point cloud data collected by the ground mobile robot:

[0139]

[0140] where V grid is a voxel grid, v k is a voxel in the voxel grid, p i is the original point cloud coordinate, and r is the voxel resolution;

[0141] Extract feature descriptors of the down-sampled point cloud;

[0142] Match the SIFT feature points of the unmanned aerial vehicle aerial image;

[0143] Solve the optimal rigid transformation matrix T from the point cloud to the image:

[0144]

[0145] Among them, T is the rigid body transformation matrix from point cloud to image, p j is the feature point of the point cloud, q k is the image feature point, and π is the projection function.

[0146] It should be noted that the process of fusing 3D point cloud data with image data involves voxel grid downsampling of the 3D point cloud data collected by a ground mobile robot, extracting feature descriptors of the downsampled point cloud, matching SIFT feature points of drone aerial images, and solving the optimal rigid body transformation matrix from the point cloud to the image. The voxel grid downsampling mentioned here is a data preprocessing method used to reduce the amount of point cloud data and improve data processing efficiency. Feature descriptors are mathematical expressions used to describe point cloud features, which facilitate subsequent matching and fusion operations. SIFT feature points are image feature points extracted by the Scale-Invariant Feature Transform algorithm. They have good scale invariance and rotation invariance and are suitable for image matching. The optimal rigid body transformation matrix is ​​a matrix used to transform point cloud data from its original coordinate system to the image coordinate system. It is solved by minimizing the distance between point cloud feature points and image feature points.

[0147] Specifically, the process of fusing 3D point cloud data with image data involves the following key steps: First, voxel grid downsampling is performed on the 3D point cloud data collected by a ground mobile robot. This process divides the point cloud data into a 3D voxel grid, where each point within the voxel is simplified to a representative point, typically the mean or median value within that voxel. Voxel resolution is a key parameter that determines the accuracy of the downsampling and the extent of data reduction. Second, feature descriptors are extracted from the downsampled point cloud. This step typically involves calculating local geometric features of each point in the point cloud, such as curvature and normal vector. These feature descriptors are used in subsequent matching operations. Next, SIFT feature points are matched to the drone aerial imagery. The SIFT algorithm extracts scale- and rotation-invariant feature points from the image. These feature points are then matched with the feature descriptors of the point cloud to determine the correspondence between the point cloud and the image. Finally, the optimal rigid body transformation matrix from the point cloud to the image is calculated. This matrix is ​​calculated by minimizing the distance between feature points in the point cloud and those in the image. This is typically achieved using the Iterative Closest Point (ICP) algorithm or other optimization methods.

[0148] Preferably, the process of fusing three-dimensional point cloud data with image data can be further refined in the following ways: when downsampling the voxel grid, the voxel resolution can be adjusted according to the actual task requirements. For example, when high-precision data is required, a smaller voxel resolution can be set; when fast processing is required, a larger voxel resolution can be set. When extracting feature descriptors, a variety of algorithms can be used, such as the FPFH Fast Point Feature Histograms algorithm, which can quickly calculate the feature descriptors of the point cloud and has a certain degree of robustness to noise. When matching SIFT feature points, the FLANN Fast Library for Approximate Nearest Neighbors algorithm can be used to accelerate the matching process and improve matching efficiency. When solving the optimal rigid body transformation matrix, the ICP algorithm can be used, combined with the RANSAC Random Sample Consensus algorithm to improve the robustness of the transformation matrix and reduce the impact of outliers. Through these refined steps and parameter adjustments, the three-dimensional point cloud data and image data can be more accurately fused, and the quality of the environment model construction can be improved.

[0149] In some embodiments, generating the environment model includes constructing a Poisson equation:

[0150]

[0151] Among them, Δ is the Laplace operator, Φ is the three-dimensional indicator function, is the divergence operator, is the point cloud normal field;

[0152] Solve the three-dimensional indicator function Φ;

[0153] Extract the isosurface M 3d ={x|Φ(x)=∈} as the environment model surface.

[0154] It should be noted that the process of generating an environmental model involves constructing the Poisson equation and solving the three-dimensional indicator function, and extracting the isosurface as the surface of the environmental model. The Poisson equation mentioned here is a partial differential equation used to describe the distribution of a function under given boundary conditions. In three-dimensional point cloud data processing, the Poisson equation is used to reconstruct the surface, and a continuous three-dimensional indicator function can be obtained by solving the equation. The three-dimensional indicator function is a mathematical function that represents the distribution of point cloud data in space and is obtained by solving the Poisson equation. An isosurface refers to a surface composed of points with equal function values ​​in three-dimensional space. By extracting the isosurface, the surface of the environmental model can be obtained, thereby realizing three-dimensional reconstruction of the environment.

[0155] Specifically, the process of generating an environment model includes the following key steps: First, construct the Poisson equation. This is done by taking the normal vector field of the point cloud data as input and using the Poisson equation to reconstruct the surface. The Poisson equation takes the form of the Laplace operator acting on a three-dimensional indicator function equal to the divergence of the normal vector field. The Laplace operator here is a mathematical operator used to calculate the sum of the second-order derivatives of a function, while the divergence operator is used to calculate the divergence of the vector field, that is, the outflow of the vector field at a certain point. The normal vector field is the set of normal vectors for each point in the point cloud data and is used to describe the local geometric structure of the point cloud. Second, solve the three-dimensional indicator function. This step solves the Poisson equation numerically to obtain a continuous function that represents the distribution of the point cloud data in space. Finally, extract the isosurface. By setting an isovalue, the surface composed of points whose function value is equal to the isovalue is extracted. This surface is the surface of the environment model.

[0156] Preferably, the process of generating the environment model can be further refined in the following ways: when constructing the Poisson equation, the point cloud data needs to be preprocessed, including removing noise points and filling holes to improve the reconstruction quality. When solving the three-dimensional indicator function, numerical methods such as the finite difference method or the finite element method can be used. These methods can effectively handle complex boundary conditions and irregular point cloud distributions. When extracting isosurfaces, the Marching Cubes algorithm can be used. This is a classic isosurface extraction algorithm that can extract isosurfaces from three-dimensional scalar fields.

[0157] Furthermore, the level of detail in the environment model can be controlled by adjusting the size of the isovalue. Smaller isovalues ​​can extract finer surface details, while larger isovalues ​​produce smoother surfaces. Through these refined steps and parameter adjustments, high-quality environment models can be generated with greater precision to meet the needs of different application scenarios.

[0158] In some embodiments, executing subsequent operations according to the task type includes:

[0159] When the task type is terrain surveying:

[0160] Extracting a set of contour lines from an environment model

[0161] Generate digital elevation maps and output them to the integrated support platform display screen;

[0162] When the task type is environmental monitoring:

[0163] Analyzing vegetation indices from aerial images

[0164]

[0165] NDVI is the normalized vegetation index, NIR is the near-infrared band reflectance, and R is the red band reflectance;

[0166] Generate ecological assessment reports and upload them through the 5G module of the integrated support platform;

[0167] When the task type is material delivery:

[0168] Calculate the optimal transport route

[0169] path * =arg min∫(w1·slope+w2·curv)ds

[0170] where path * is the optimal transportation path, w1 is the slope weight coefficient, w2 is the curvature weight coefficient, slope is the path slope, curv is the path curvature, and ds is the path element.

[0171] It's important to note that the subsequent operations, depending on the task type, involve different processing and analysis of the generated environmental model to meet the needs of each task. The task type here refers to the specific task that the robotic system needs to complete, such as terrain mapping, environmental monitoring, or material distribution. Each task type has its own unique data processing and analysis process. For example, in terrain mapping tasks, contour lines need to be extracted from the environmental model and a digital elevation map generated; in environmental monitoring tasks, vegetation indices need to be analyzed and an ecological assessment report generated; and in material distribution tasks, the optimal transportation route needs to be calculated. These subsequent operational steps ensure that the collected data can be effectively utilized to achieve the mission objectives.

[0172] Specifically, the process of executing subsequent tasks based on the mission type includes the following key steps: First, when the mission type is terrain mapping, a set of contour lines is extracted from the environmental model. Contour lines are closed curves formed by connecting points of the same elevation on a topographic map, used to represent the undulations of the terrain. Contour lines are extracted at a certain elevation interval, called the contour interval. Second, a digital elevation map is generated and output to the display screen of the integrated support platform. A digital elevation map is a map that digitally represents terrain elevation information. It can be generated from contour data and intuitively displays the three-dimensional structure of the terrain. Finally, when the mission type is environmental monitoring, the vegetation index of the aerial imagery is analyzed. Vegetation indices, such as the Normalized Difference Vegetation Index (NDVI), are used to assess vegetation growth. They are calculated by calculating the ratio of near-infrared reflectance to red reflectance. Furthermore, when the mission type is material distribution, the optimal transportation route must be calculated. This involves considering factors such as the slope and curvature of the route to determine the most efficient and safest transportation route.

[0173] Preferably, the process of performing subsequent operations according to the task type can be further refined in the following ways: In terrain mapping tasks, the density of contour lines can be controlled by adjusting the contour interval, so that the details of the terrain can be more clearly displayed in the digital elevation map. In environmental monitoring tasks, a comprehensive analysis can be conducted in combination with multiple vegetation indices to more comprehensively assess the health of vegetation. For example, in addition to NDVI, the soil adjusted vegetation index SAVI can also be calculated. In material distribution tasks, a terrain cost map can be constructed to define the cost function of each vertex, including parameters such as the slope influence coefficient and the direction deviation influence coefficient, and the minimum cost path from the start point to the end point can be searched through the A* algorithm. These refined steps and parameter adjustments can improve the efficiency and accuracy of task execution and ensure that data collection and processing can better serve actual application scenarios.

[0174] In some embodiments, calculating the optimal transportation path includes:

[0175] Construct terrain cost map G = (V, E);

[0176] Define vertex v i The cost function is:

[0177]

[0178] Among them, c(v i ) is the vertex v i The cost function value is α, which is the slope influence coefficient, and β is the direction deviation influence coefficient. For vertex v i The slope value at n(v i ) is the vertex v i Normal vector at n target is the normal vector of the target direction;

[0179] A* algorithm is used to search for the minimum cost path from the starting point to the end point.

[0180] It should be noted that the process of calculating the optimal transportation path involves constructing a terrain cost map, defining the cost function of the vertices, and finally searching for the minimum cost path from the starting point to the end point through the A* algorithm. The terrain cost map mentioned here is a map used to represent the complexity of the terrain and the cost of the path. It converts various terrain features such as slope and curvature into cost values ​​for path planning. The cost function is a mathematical expression used to evaluate the cost of each point on the path. It comprehensively considers various factors of the terrain, such as slope and direction deviation. The A* algorithm is a heuristic search algorithm used to find the optimal path from the starting point to the end point on the map. It selects the optimal path by evaluating the cost of each node.

[0181] Specifically, the process of calculating the optimal transportation path includes the following key steps: First, construct a terrain cost map, which is accomplished by converting various terrain features, such as slope and curvature, into cost values. The terrain cost map consists of vertices and edges, each vertex represents a location on the terrain, and each edge represents the path between two locations. Secondly, define the cost function of the vertex, which usually includes parameters such as the slope influence coefficient and the direction deviation influence coefficient. The slope influence coefficient is used to measure the impact of the slope of the path on the cost, and the direction deviation influence coefficient is used to measure the impact of the degree to which the path deviates from the target direction on the cost. Finally, the minimum cost path from the start point to the end point is searched through the A* algorithm. The A* algorithm selects the optimal path by evaluating the cost of each node. It combines the heuristic function and the actual cost to optimize the search process.

[0182] Preferably, the process of calculating the optimal transport path can be further refined in the following ways: when constructing the terrain cost map, a high-resolution digital elevation map can be used as input data to ensure the accuracy and detail of the terrain features. When defining the cost function, the values ​​of the slope influence coefficient and the direction deviation influence coefficient can be adjusted according to the actual mission requirements. For example, in mountainous tasks, the value of the slope influence coefficient can be increased to pay more attention to the impact of the slope on the path cost; in urban environments, the value of the direction deviation influence coefficient can be increased to pay more attention to the directionality of the path. When using the A* algorithm, a variety of heuristic functions, such as Euclidean distance, Manhattan distance, etc., can be combined to improve search efficiency. In addition, the settings of these parameters can be optimized through field tests and simulation experiments to ensure that the generated optimal transport path is both efficient and safe.

[0183] In some embodiments, dynamic collaborative control is also included:

[0184] Real-time detection of terrain roughness parameter ξ by ground mobile robot d ;

[0185] When d >ξ max When ξ max is the terrain ruggedness threshold;

[0186] The integrated support platform controls the UAV to fly to the current position coordinates of the ground mobile robot;

[0187] The drone collects supplementary data through downward-looking cameras and transmits it to the integrated support platform;

[0188] The integrated support platform fuses the supplementary data with the ground mobile robot point cloud data.

[0189] It should be noted that the dynamic collaborative control process involves the ground mobile robot detecting the terrain roughness parameters in real time and sending an assistance request to the integrated support platform when it exceeds a set threshold. The integrated support platform then controls the UAV to fly to the current position of the ground mobile robot, collects supplementary data through the downward-looking camera, transmits it to the integrated support platform, and finally fuses the supplementary data with the point cloud data of the ground mobile robot. The terrain roughness parameters mentioned here refer to indicators used to quantify the complexity of the terrain, such as slope and undulation, which are used to assess the difficulty of the robot's driving on the terrain. An assistance request refers to a signal sent by the ground mobile robot to the integrated support platform when encountering complex terrain, in order to obtain additional data support. The downward-looking camera is a camera installed on the UAV, which is used to shoot downward at the ground conditions and obtain supplementary data.

[0190] Specifically, the dynamic collaborative control process includes the following key steps: First, the ground mobile robot detects terrain roughness parameters in real time. These parameters can be obtained through sensors on the robot, such as an inertial measurement unit (IMU) or lidar. When the detected terrain roughness parameter exceeds a set threshold, the robot determines that the current terrain is too complex and requires additional data support to complete the task. Second, the ground mobile robot sends an assistance request to the integrated support platform. This request includes the robot's current position information, allowing the integrated support platform to accurately dispatch the drone. The integrated support platform then controls the drone to fly to the ground mobile robot's current position. The drone uses a downward-looking camera to collect supplementary data. This data can be high-resolution images or videos, providing more detailed terrain information. Finally, the drone transmits the collected supplementary data to the integrated support platform, which fuses this data with the ground mobile robot's point cloud data to generate a more comprehensive environmental model.

[0191] Preferably, the dynamic collaborative control process can be further refined in the following ways: When detecting terrain roughness parameters in real time, multiple thresholds can be set to distinguish different levels of terrain complexity, and different auxiliary measures can be taken based on the thresholds. For example, a lower threshold may only require a simple data update, while a higher threshold may require the UAV to collect supplementary data for a longer period of time. When controlling the UAV to fly to the ground mobile robot's current location, a preset flight path planning algorithm can be used to ensure that the UAV can quickly and safely reach the target location. When collecting supplementary data, the downward-looking camera parameters, such as resolution and exposure time, can be adjusted according to mission requirements to obtain higher-quality data. Finally, during the data fusion process, advanced fusion algorithms, such as those based on feature point matching or deep learning, can be used to improve the accuracy and reliability of the fused data. Through these refined steps and parameter adjustments, the collaborative operation between the ground mobile robot and the UAV can be more effectively achieved, improving the efficiency and success rate of mission execution.

[0192] In some embodiments, the data transmission uses a layered protocol:

[0193] The LoRa protocol is used to transmit point cloud data between the ground mobile robot and the integrated support platform;

[0194] The Wi-Fi 6 protocol is used to transmit image data between the drone and the integrated support platform;

[0195] The comprehensive support platform transmits environmental model data via the 5G network.

[0196] It should be noted that the data transmission process using a layered protocol involves the LoRa protocol for transmitting point cloud data between the ground mobile robot and the integrated support platform, the Wi-Fi 6 protocol for transmitting image data between the drone and the integrated support platform, and the integrated support platform transmitting environmental model data backhaul via the 5G network. The layered protocol here refers to the use of different communication protocols in different communication links to accommodate different data types and transmission requirements. The LoRa protocol is a low-power wide area network (LPWAN) communication protocol suitable for long-distance, low-bandwidth data transmission, especially for data with large volumes but low real-time requirements, such as point cloud data. The Wi-Fi 6 protocol is a high-speed, low-latency wireless local area network (WLAN) communication protocol suitable for transmitting large volumes of data with high real-time requirements, such as image data. The 5G network, on the other hand, is a high-speed, low-latency mobile communication network suitable for transmitting data that needs to be quickly transmitted back to the control center, such as environmental model data.

[0197] Specifically, the process of data transmission using layered protocols includes the following key steps: first, the ground mobile robot and the comprehensive support platform use LoRa protocol to transmit point cloud data. LoRa protocol realizes long-distance transmission through spread spectrum technology, and its transmission distance can reach several kilometers, which is suitable for use in open or semi-open environment. The bandwidth of LoRa protocol is low, but it can meet the transmission requirements of point cloud data, because point cloud data usually does not need real-time transmission, but can be transmitted gradually during the movement of the robot. Second, the unmanned aerial vehicle and the comprehensive support platform use Wi-Fi 6 protocol to transmit image data. Wi-Fi 6 protocol supports higher data transmission rate and lower delay, which is suitable for transmitting image data that needs fast transmission and real-time processing. Wi-Fi 6 protocol can support multiple devices to connect at the same time, ensuring the stability and efficiency of data transmission. Finally, the comprehensive support platform transmits environmental model data back to the control center through 5G network. 5G network has the characteristics of high speed and low delay, which can quickly transmit environmental model data to the control center for further analysis and decision-making.

[0198] Preferably, the process of data transmission using layered protocols can be further refined in the following ways: when transmitting point cloud data using LoRa protocol, the spread spectrum factor and bandwidth parameters of LoRa can be adjusted according to the actual environment to optimize the transmission distance and data transmission rate. For example, in urban environment, the spread spectrum factor can be appropriately reduced to increase the transmission rate; in rural or mountainous areas, the spread spectrum factor can be increased to extend the transmission distance. When transmitting image data using Wi-Fi 6 protocol, multi-band communication can be set up to improve the stability and speed of data transmission. For example, 2.4GHz and 5GHz frequency bands can be used simultaneously, and dynamic switching can be performed according to signal strength and interference. When transmitting environmental model data back to the control center through 5G network, the slicing technology of 5G network can be used to allocate a dedicated network slice for environmental model data to ensure the priority and stability of data transmission. In addition, network state monitoring mechanism can be combined to adjust data transmission strategy in real time to cope with different network environments and task requirements. Through these refined steps and parameter adjustments, the layered transmission of data can be more effectively realized, and the communication efficiency and reliability of the entire system can be improved.

[0199] The above-mentioned various embodiments of the present application have the following beneficial effects:

[0200] 1. Through the cooperative working mechanism of the ground mobile robot and the unmanned aerial vehicle, full coverage data collection in narrow terrain and low altitude area is realized, the problem of incomplete data collection by single device in complex environment is solved, and the intelligent scheduling function of the comprehensive support platform is used to ensure that the device can return to the assembly point in time when the power is insufficient or the storage is full, thereby improving the working efficiency and continuity of the system.

[0201] 2. Dynamic weights are used to calculate the coordinates of the assembly point, taking into account the power of the ground robot, the storage capacity of the drone, and the location of the support platform. This optimizes the equipment recovery path, reduces energy waste and time loss, solves the path redundancy problem caused by traditional fixed assembly points, and improves the system's response speed and resource utilization.

[0202] 3. Through the high-precision fusion of 3D point cloud data and aerial images, a more accurate environmental model is constructed, and subsequent operations such as terrain mapping, environmental monitoring, or material distribution are automatically performed in combination with the task type. This solves the problems of insufficient data fusion accuracy and poor task adaptability in traditional methods, and enhances the intelligence level and practical application value of the system.

[0203] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0204] The above descriptions are merely some preferred embodiments of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in the embodiments of the present invention.

Claims

1. A method for collaborative data collection of multi-modal intelligent devices, characterized in that: include: The ground mobile robot moves in a narrow terrain area and collects 3D point cloud data through LiDAR; The drone moves in a low-altitude flight area and collects image data through an aerial camera; The integrated support platform resides in the open area and monitors the remaining power of the ground mobile robot in real time. d and the remaining storage capacity C of the drone u ; When E d <E th or C u <C th When the integrated support platform broadcasts the coordinates of the assembly point (x m ,y m ); the E th is the preset power threshold, the C th is a preset capacity threshold; The ground mobile robot moves along the optimal path on the ground to (x m ,y m ) and receive wireless charging, while the drone moves along a straight path in the air to (x m ,y m ) and transmit image data; The comprehensive support platform fuses 3D point cloud data and image data to generate an environmental model and performs subsequent operations based on the task type.

2. The method according to claim 1, characterized in that The broadcast assembly point coordinates (x m ,y m )include: Get the current position coordinates of the ground mobile robot in real time (x d ,y d ), the current position coordinates of the drone (x u ,y u ) and the current position coordinates of the integrated support platform (x b ,y b ); Calculating the power weight factor of a ground mobile robot in, is the maximum power of the ground mobile robot, E d The remaining power of the ground mobile robot; Calculating drone storage weighting factors in, is the maximum storage capacity of the UAV, C u The remaining storage capacity of the drone; Set the comprehensive support platform position weight constant w b ; Calculate the coordinates of the assembly point:

3. The method according to claim 1, characterized in that The generation of the optimal ground path includes: Obtain a digital elevation map of the work area; Extract the coordinates of the ground mobile robot's current position to the assembly point (x m ,y m ) between the candidate paths; Calculate the terrain passability cost of each path: Among them, α is the slope influence coefficient, β is the obstacle density influence coefficient, is the slope value of path segment i, ρ o is the obstacle density coefficient; Select Cost g The path with the smallest value is taken as the optimal ground path.

4. The method according to claim 1, wherein The receiving wireless charging comprises: Comprehensive support platform to detect the current power E of ground mobile robots d and drones currently store C u ; Calculate the charging power of a ground mobile robot: Calculate the drone charging power: in, The maximum charging power of the comprehensive support platform, is the maximum power of the ground mobile robot, E d is the remaining power of the ground mobile robot, is the maximum storage capacity of the UAV, C u The remaining storage capacity of the drone; Transmit P to the ground mobile robot through electromagnetic induction coil d Power, transmitted to the drone P u power.

5. The method according to claim 1, characterized in that The fusion of three-dimensional point cloud data and image data includes: Perform voxel grid downsampling on 3D point cloud data collected by a ground mobile robot: Among them, V grid is the voxel grid, v k is a voxel in the voxel grid, p i is the original point cloud coordinate, r is the voxel resolution; Extract feature descriptors of downsampled point clouds; Match SIFT feature points of drone aerial images; Solve the optimal rigid body transformation matrix T from point cloud to image: Among them, T is the rigid body transformation matrix from point cloud to image, p j is the feature point of the point cloud, q k is the image feature point, and π is the projection function.

6. The method according to claim 5, characterized in that The generation of the environmental model includes: constructing the Poisson equation: Among them, Δ is the Laplace operator, Φ is the three-dimensional indicator function, is the divergence operator, is the point cloud normal field; Solve the three-dimensional indicator function Φ; Extract the isosurface M 3d ={x|Φ(x)=∈} as the environment model surface.

7. The method according to claim 1, characterized in that The execution of subsequent operations according to the task type includes: When the task type is terrain surveying: Extracting a set of contour lines from an environment model Generate digital elevation maps and output them to the integrated support platform display screen; When the task type is environmental monitoring: Analyzing vegetation indices from aerial images NDVI is the normalized vegetation index, NIR is the near-infrared band reflectance, and R is the red band reflectance; Generate ecological assessment reports and upload them through the 5G module of the integrated support platform; When the task type is material delivery: Calculate the optimal transport route path * =arg min∫(w1·slope+w2·curv)ds where path * is the optimal transportation path, w1 is the slope weight coefficient, w2 is the curvature weight coefficient, slope is the path slope, curv is the path curvature, and ds is the path element.

8. The method according to claim 7, characterized in that The calculation of the optimal transportation path includes: Construct terrain cost map G = (V, E); Define vertex v i The cost function is: Among them, c(v i ) is the vertex v i The cost function value is α, which is the slope influence coefficient, and β is the direction deviation influence coefficient. For vertex v i The slope value at n(v i ) is the vertex v i Normal vector at n target is the normal vector of the target direction; A* algorithm is used to search for the minimum cost path from the starting point to the end point.

9. The method according to claim 1, characterized in that Also includes dynamic collaborative control: Real-time detection of terrain roughness parameter ξ by ground mobile robot d ; When d >ξ max When ξ max is the terrain ruggedness threshold; The integrated support platform controls the UAV to fly to the current position coordinates of the ground mobile robot; The drone collects supplementary data through downward-looking cameras and transmits it to the integrated support platform; The integrated support platform fuses the supplementary data with the ground mobile robot point cloud data.

10. The method according to claim 1, characterized in that The data transmission adopts a layered protocol: The LoRa protocol is used to transmit point cloud data between the ground mobile robot and the integrated support platform; The Wi-Fi 6 protocol is used to transmit image data between the drone and the integrated support platform; The comprehensive support platform transmits environmental model data via the 5G network.

Images