Real-time modeling method and system for cooperative work of land scanning device and flight scanning device

Abstract

The invention provides a real-time modeling method and system for cooperative work of land and flight scanning devices, and relates to the technical field of scanning and modeling. The method comprises the following steps: preliminarily scanning a target area by using a land scanning device to generate three-dimensional point cloud data; a flight scanning device is used for conducting supplementary scanning on the complex area, and the coverage blind area of ground scanning is made up; fusing the land scanning data and the flight scanning data to generate a high-precision three-dimensional model; based on real-time data analysis, a flight path is dynamically adjusted to improve scanning efficiency and coverage rate. Through cooperative work of the land and the flight device, the modeling precision and efficiency are effectively improved, and the method is suitable for complex terrains and multi-scene application and has high real-time performance and reliability.

Description

Technical Field

[0001] The present invention relates to the field of scanning and modeling technology, and in particular to a real-time modeling method and system for coordinated operation of land and flight scanning devices. Background Art

[0002] Current 3D modeling technology is widely used in many industries, including construction engineering, cultural relics protection, environmental monitoring and smart city construction. In order to achieve high accuracy and comprehensive coverage of 3D models, laser scanning, photogrammetry and other technologies are usually used to obtain environmental data using different scanning devices (such as ground-based lidar, laser or optical scanning equipment carried by drones). However, traditional scanning methods still face challenges in data integrity, modeling efficiency and equipment cost in practical applications.

[0003] Traditional land scanning methods mainly rely on fixed-position laser radar or photography equipment, which can provide high accuracy and are suitable for modeling low-altitude environments and flat areas. However, the coverage of land scanning equipment is limited by its location, environmental obstacles, and height differences, and it is often difficult to achieve comprehensive scanning in complex terrain or multi-story building areas. At the same time, since land equipment usually requires manual operation or fixed installation, the scanning efficiency is low and it does not have the ability to respond to dynamically changing environments in real time.

[0004] The development of aerial scanning technology has provided a new direction for efficient environmental modeling. By using drones equipped with lidar or cameras, large areas can be quickly covered without being restricted by terrain. Nevertheless, aerial scanning has some limitations. For example, the flight equipment is limited by flight time, flight altitude and obstacles, making it difficult to obtain complete data inside buildings or in shielded areas. In addition, aerial scanning usually relies on separate route planning, which easily ignores the data requirements of local complex areas, resulting in insufficient detail and accuracy in modeling.

[0005] Changes in market demand have driven the development of scanning systems towards automation and intelligence. There is an urgent need for a real-time modeling method that allows the coordinated work of land and flight scanning devices to achieve efficient and comprehensive data collection and model building. This collaborative method should be able to integrate the advantages of land and air scanning equipment. On the basis of providing high-precision data from land scanning, the flight equipment can cover areas that are difficult to reach with ground equipment to achieve high precision, integrity and real-time performance of the three-dimensional environment. In addition, as application scenarios continue to expand, the system must also have intelligent functions such as real-time path adjustment and data fusion optimization to meet the needs of flexible modeling in complex scenarios and improve data utilization efficiency and model sophistication. Summary of the invention

[0006] In order to solve the technical problems of incomplete data collection, insufficient modeling accuracy and low efficiency in the prior art, the present invention provides a real-time modeling method and system for coordinated work of land and flight scanning devices.

[0007] The technical solution provided by the present invention is as follows:

[0008] First aspect:

[0009] The present invention provides a real-time modeling method for collaborative operation of land and flight scanning devices, comprising:

[0010] S1. Use the land scanning device to conduct a preliminary scan of the environment and generate data D land , including three-dimensional coordinate data, reflection intensity, surface normal vector and depth data, the data D land After processing, a preliminary three-dimensional model M is generated. init , providing a basis for subsequent flight scans;

[0011] S2, start the flight scanning device, and scan the three-dimensional model M init The flight scanning device optimizes the path of the scanning area through autonomous flight planning and obstacle avoidance algorithm to obtain data D fly , and combine sensor data for real-time position correction;

[0012] S3. Identify the areas that are not covered in the flight scan and generate the missing area mark A missing , the actual area is obtained by combining the three-dimensional model M init By comparing and obtaining, the scanning data and the area identification information are used to further infer the re-scanning path of the flying device;

[0013] S4, fuse the land and flight scan data, calculate and generate the final 3D environment model M final In the data fusion process, the weighting coefficient α of the land scan data is introduced land and the weighting factor α of the flight scan data fly , weighted calculation of the contribution values ​​of the two data sources;

[0014] S5. Based on the generated model and data analysis, the flight device automatically corrects the scanning path P fly , optimize the selection of flight routes to ensure the maximum scanning range and time. The optimization of flight paths is based on real-time feedback of data quality assessment, obstacle detection and scanning performance;

[0015] S6. During the scanning process, the flight scanning device and the land scanning device exchange data in real time through wireless communication, and continuously calibrate and update the scanning results. After each scan, the model M finalCorrected based on new data.

[0016] Second aspect:

[0017] The present invention provides a real-time modeling system for coordinated operation of a land scanning device and a flight scanning device, comprising:

[0018] processor;

[0019] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the real-time modeling method for the coordinated operation of the land and flight scanning devices as described in the first aspect is implemented.

[0020] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0021] (1) In the present invention, by introducing the collaborative working mechanism of land and flight scanning devices, the high precision of land scanning and the flexible coverage of flight scanning are fully utilized, so that the scanning process can dynamically identify and supplement blind areas. Especially in complex environments, after the land scanning device completes the construction of the basic model, the flight scanning device can autonomously plan the path and scan the missing areas in the preliminary model, thereby greatly reducing the areas that cannot be covered by traditional scanning methods and realizing all-round and high-completeness three-dimensional environment modeling;

[0022] (2) In the present invention, during the data fusion process, the present invention innovatively introduces a weighted fusion method for land and flight scan data, combining the fineness of land scans with the wide coverage characteristics of flight scans. By dynamically weighting the contribution of different data sources according to the complexity of the environment, the final generated three-dimensional model is more accurate and detailed, especially in scenes with rich details or changing perspectives, which can significantly improve the accuracy and authenticity of the model;

[0023] (3) In the present invention, innovations are made in the optimization of the flight scanning path. The autonomous path planning and obstacle avoidance algorithm of the flight scanning device is used to enable it to adjust the flight path in real time in a dynamic environment, avoid repeated scanning, and reasonably arrange the scanning tasks of different devices. Under the premise of ensuring coverage and modeling quality, unnecessary scanning paths are reduced, and scanning efficiency and resource utilization are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic flow chart of a real-time modeling method for collaborative operation of a land and flight scanning device provided by an embodiment of the present invention;

[0026] Figure 2 A schematic structural diagram of a real-time modeling system in which a land scanning device and a flight scanning device work in coordination is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0028] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0029] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0030] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] Reference Manual Attached Figure 1 , showing a flow chart of a real-time modeling method for collaborative work between land and flight scanning devices provided by an embodiment of the present invention.

[0033] The embodiment of the present invention provides a real-time modeling method for the coordinated operation of a land and a flying scanning device. The method can be implemented by a real-time modeling device for the coordinated operation of a land and a flying scanning device. The real-time modeling device for the coordinated operation of a land and a flying scanning device can be a terminal or a server. The processing flow of the real-time modeling method for the coordinated operation of a land and a flying scanning device can include the following steps:

[0034] S1. Use the land scanning device to conduct a preliminary scan of the environment and generate data Dland , including three-dimensional coordinate data, reflection intensity, surface normal vector and depth data, data D land After processing, a preliminary three-dimensional model M is generated. init , providing a basis for subsequent flight scans.

[0035] It is understandable that a preliminary scan using a land scanning device to generate multi-dimensional data including three-dimensional coordinate data, reflection intensity, surface normal vectors and depth data can quickly build a preliminary three-dimensional model of the environment. This preliminary model provides basic information for subsequent flight scanning, which can effectively reduce the workload and time of flight scanning. The advantage of land scanning data is the high-precision capture of ground details, which lays an important foundation for the overall accuracy of three-dimensional modeling and provides guidance for subsequent scanning.

[0036] S2, start the flight scanning device, and scan the three-dimensional model M init The flight scanning device optimizes the path of the scanning area through autonomous flight planning and obstacle avoidance algorithm to obtain data D fly , and combines sensor data for real-time position correction.

[0037] It is understandable that starting the flight scanning device to scan the missing areas in the preliminary 3D model generated by the land scan can make up for the blind spots of the ground scan and achieve full coverage of complex terrain. The flight scanning device optimizes the scanning path with the help of autonomous flight planning and obstacle avoidance algorithms, and corrects the position based on sensor data to ensure the accuracy and safety of the scan. This process significantly improves the integrity of data collection in complex areas and provides more comprehensive data support for the final model.

[0038] S3. Identify the areas that are not covered in the flight scan and generate the missing area mark A missing , indeed the area is passed and the three-dimensional model M init By comparing the obtained scan data and area identification information, the re-scan path of the flying device can be further inferred.

[0039] It is understandable that the areas that are not covered in the flight scan will be identified and the missing area mark will be generated. By comparing these missing areas with the 3D model, the completeness of the modeling can be significantly improved. Further scanning path inference of the missing areas ensures that the flight device can accurately fill in the data gaps, avoid repeated coverage, and improve scanning efficiency. This step further refines the data collection process and minimizes modeling errors.

[0040] S4, fuse the land and flight scan data, calculate and generate the final 3D environment model M finalIn the data fusion process, the weighting coefficient α of the land scan data is introduced land and the weighting factor α of the flight scan data fly , weighted calculation of the contribution values ​​of the two data sources.

[0041] It is understandable that by fusing land scan data and flight scan data, a more accurate final 3D environment model can be generated. The weighted coefficients of land and flight scan data are added in the data fusion process, and weighted calculations are performed according to the contributions of different data sources to ensure the balance of the model in terms of detail richness and globality. This fusion method makes the final 3D model more accurate and complete, and effectively improves the reliability of subsequent applications.

[0042] S5. Based on the generated model and data analysis, the flight device automatically corrects the scanning path P fly , optimize the selection of flight routes to ensure the maximization of scanning range and time. The optimization of flight paths is based on real-time feedback of data quality assessment, obstacle detection and scanning performance.

[0043] It is understandable that based on the generated 3D model and data analysis, the system can automatically correct the scanning path of the flying device and optimize the route selection. The path correction is based on data quality assessment, obstacle detection and real-time feedback mechanism to ensure the maximization of the scanning range and the reasonable allocation of working time. This step of optimization design enhances the autonomy and intelligence of the scanning device, enabling it to cope with dynamic environments more efficiently, and improves the flexibility and coverage of the scanning process.

[0044] S6. During the scanning process, the flight scanning device and the land scanning device exchange data in real time through wireless communication, and continuously calibrate and update the scanning results. After each scan, the model M final Corrected based on new data.

[0045] It is understood that during the entire scanning process, the flight and land scanning devices exchange data in real time through wireless communication, and continuously correct and update the current scanning results. This real-time communication and data correction mechanism enables the system to optimize the model based on the latest data, ensuring the real-time and accuracy of the modeling results. After each scan, the model will be adjusted according to the new data, further improving the dynamic response ability and practicality of the model.

[0046] In a possible implementation, S2 further includes:

[0047] In the selection of the scanning path of the flying device, the flying device selects the scanning path according to the existing scanning data D fly and target coverage area, calculate the potential path P fly The possibility of flight path selection is optimized by the following formula:

[0048]

[0049] Among them, P fly is the optimized flight path, D fly (p i ) is the path p i The amount of scan data, T fly (p i ) is the path p i The scanning time, f efffciency (p i ) is the path p i The efficiency function, taking into account obstacles and weather factors on the flight path, is the number of all potential paths.

[0050] In a possible implementation, S4 further includes:

[0051] In the data fusion process, the scanning data D of the flight device and the land device land and D fly Considering the contribution, weighted fusion is achieved through the following formula:

[0052]

[0053] Among them, M final (x, y, z) is the final 3D model after fusion, M land (x, y, z) is the 3D model generated by land scanning data, M fly (x, y, z) is the 3D model generated by the flight scanning data, W land is the weighting coefficient of land scan data, W fly is the weighting coefficient of the flight scanning data. After weighted calculation, the final three-dimensional model M final Provided for use in subsequent path correction and optimization steps.

[0054] In a possible implementation manner, determining the weighting coefficient specifically includes:

[0055] Calculate the spatial resolution of the data for both the land scanner and the flight scanner separately:

[0056]

[0057] Among them, R L and R F are the spatial resolutions of land scanning data and flight scanning data, respectively, L and D F are the effective detection distances of the land scanning device and the flight scanning device, N L and N Fare the point cloud densities per unit distance for land and flight scanning devices, respectively;

[0058] According to the ratio of spatial resolution, the basic weighting coefficients of land and flight scan data are preliminarily calculated:

[0059]

[0060] in, is the preliminary weighting factor for land scanning, is the preliminary weighting factor for flight scanning;

[0061] The coefficient is adjusted according to the environmental complexity of the scanning area. The environmental complexity factor C is set to represent the environmental complexity level (for example, C=1.5 is set in areas with many obstacles, and C=1 is set in open areas). In complex environments, the weighted coefficient of land scanning is increased. The adjusted coefficient is:

[0062]

[0063] Among them, W land and W fly are the adjusted weighting coefficients for land scanning and flight scanning respectively. Finally, in order to ensure that the sum of the weighting coefficients is 1, the adjusted coefficient W land and W fly Perform normalization.

[0064] In a possible implementation, to ensure that the land scan and flight scan data match correctly, the scan data needs to be checked for integrity after the scan is completed:

[0065]

[0066] Among them, A missing Indicates the identification and calculated value of the missing area, D land (x i ,y i , z i ) represents the land scan data of the i-th point, represents the estimated value of the i-th point, w i Represents the weighting coefficient, representing the importance of the point, N points Indicates the number of scanned data points;

[0067] The calculation formula of the weighting coefficient is as follows:

[0068]

[0069] Among them, w i is the weighting factor for missing region identification, N iis the number of adjacent scanning points within the specified radius, which is used to reflect the density of the scanning points. A is the area of ​​the specified range, which is used to normalize the density so that it is independent of the radius size. α i is the angle between the scanning angle and the normal, using cos(α i ) to reduce the weight of large angle scanning points.

[0070] In a possible implementation manner, the path optimization of the flight scanning device specifically includes:

[0071] During the flight, the flight device detects obstacles in the path in real time, including static and dynamic obstacles, updates the true position based on sensor data, and corrects the flight path P. fly , avoiding obstacles and optimizing the path, calculating the corrected flight path To ensure that the scanning task is completed as planned, the path correction calculation is performed using the following formula:

[0072]

[0073] in, is the corrected flight path, f avoidance (p i ) is the path p i Evaluation of obstacle avoidance effectiveness, f efficiency (p i ) is the path p i The efficiency function, N paths is the number of all potential paths.

[0074] In a possible implementation manner, the path correction further includes:

[0075] Evaluate the priority of scanning tasks and calculate the priority P of different areas task , and adjust the flight path according to the mission importance. The flight path P is adjusted according to the mission priority and regional importance through the following formula fly Choices:

[0076]

[0077] in, is the optimized flight path, f importance (p i ) is the path p i The task importance function, f efficiency (p i ) is the path p i The efficiency function, N paths is the number of all potential paths.

[0078] In a possible implementation, during the real-time scanning process, the data density is calculated according to the number of point cloud data in the unit scanning area. The data density is used to reflect the detail level of the current scanning area. When the data density is higher than a preset threshold, it means that the data of the area is detailed enough.

[0079] According to the value range of data density, the accuracy of the scanning device is adaptively adjusted. When the data density exceeds the threshold, the system reduces the scanning accuracy of the current area to save resources. When the data density is lower than the threshold, the scanning accuracy is improved to supplement the data and ensure the completeness of the model details.

[0080] After completing the data density adjustment, the local model is evaluated and optimized in real time, and a precision adaptive model is generated based on the point cloud density distribution. Supplementary sampling points are added in low-density areas to improve accuracy, and redundant data is removed in high-density areas to optimize modeling speed.

[0081] In a possible implementation, the path coverage of the land scanning device and the flight scanning device is calculated using the distribution of the scanning path and the point cloud data. The coverage is used to reflect the coverage degree of the current scanning path to the entire modeling area.

[0082] When the probability of coverage does not meet the set minimum requirement, a supplementary path is automatically generated based on the uncovered area to guide the land scanning device and the flight scanning device to perform supplementary scanning tasks. When the coverage rate meets the requirement, the supplementary scanning is skipped to save resources.

[0083] According to the real-time position and coverage of the land scanning device and the flying scanning device, the scanning path is dynamically adjusted, and the flying scanning device is given priority to handle the difficult-to-reach elevation areas, and the land scanning device handles the flat areas, to ensure the coverage uniformity and modeling quality of different areas.

[0084] To verify the effect of the embodiment of the present invention, a mixed terrain site with an actual area of ​​about 450 meters × 280 meters, including a flat area, a forest and some buildings, was selected to perform the following implementation steps:

[0085] Preliminary scanning and model building:

[0086] A vehicle-mounted land scanning device (model: RIEGLVZ-400i) was used to perform a full coverage scan of the site with the scanning resolution set to 10 mm per point, the maximum scanning distance set to 250 m, and the point cloud sampling rate set to 1.2 million points per second.

[0087] The preliminary 3D model generated after the scan is completed contains about 180 million point cloud data. The coverage analysis results show that the coverage rate of flat areas reaches 99.2%, but due to line of sight obstruction and equipment location restrictions, the coverage rate of complex areas around trees and buildings is only 78.5%. These areas are marked as missing areas and await subsequent re-scanning.

[0088] Flying Scanning:

[0089] A flying scanning device (model: DJI M300 RTK equipped with L1 LiDAR) was used, with a flight altitude of 40 meters, a total path length of 1.8 kilometers, and a resolution of 15 mm per point. The flying device completed the scan through autonomous path planning and obstacle avoidance algorithms, especially for complex terrain and tree canopies.

[0090] After the rescan, about 60 million point cloud data were added, filling in most of the missing areas. Comprehensive analysis showed that the coverage rate of complex areas after scanning increased to 96.8%.

[0091] Data fusion and model improvement:

[0092] The data from the land scanning device and the flying scanning device were fused, and different weights were assigned to the point cloud according to the resolution and data source of the two devices. The point cloud density of the fused 3D model reached 4,000 points per square meter, and the coverage rate increased to 99.8%. The final model successfully captured subtle terrain changes and tree crown details, and the error of the generated site 3D model was less than ±1.5 cm.

[0093] Flight path correction and optimization:

[0094] During the rescanning process, the system analyzes the coverage and scanning efficiency of the flight path in real time. When insufficient data is detected in a certain area, the path is automatically optimized to reduce redundant flight distance. After adjustment, it took the flight device 1.2 hours to complete the entire task, which was 20 minutes shorter than the initial path planning time. The total modeling task of the land and flight equipment took about 3 hours and 15 minutes, which was about 28% more efficient than the single-device scanning method.

[0095] Experimental results and evaluation:

[0096] In the embodiment of the present invention, efficient and high-precision three-dimensional modeling is achieved through the collaborative work of land scanning and flight scanning devices. Experimental results show that compared with the single-device modeling method, the coverage rate of the present invention is increased from 85% to 99.8%, the point cloud density is increased from 2000 points per square meter to 4000 points, and the model generation error is reduced from ±3 cm to ±1.5 cm, which significantly improves the modeling accuracy and efficiency and is suitable for multi-scenario application requirements in complex environments.

[0097] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0098] (1) In the present invention, by introducing the collaborative working mechanism of land and flight scanning devices, the high precision of land scanning and the flexible coverage of flight scanning are fully utilized, so that the scanning process can dynamically identify and supplement blind areas. Especially in complex environments, after the land scanning device completes the construction of the basic model, the flight scanning device can autonomously plan the path and scan the missing areas in the preliminary model, thereby greatly reducing the areas that cannot be covered by traditional scanning methods and realizing all-round and high-completeness three-dimensional environment modeling;

[0099] (2) In the present invention, during the data fusion process, the present invention innovatively introduces a weighted fusion method for land and flight scan data, combining the fineness of land scans with the wide coverage characteristics of flight scans. By dynamically weighting the contribution of different data sources according to the complexity of the environment, the final generated three-dimensional model is more accurate and detailed, especially in scenes with rich details or changing perspectives, which can significantly improve the accuracy and authenticity of the model;

[0100] (3) In the present invention, innovations are made in the optimization of the flight scanning path. The autonomous path planning and obstacle avoidance algorithm of the flight scanning device is used to enable it to adjust the flight path in real time in a dynamic environment, avoid repeated scanning, and reasonably arrange the scanning tasks of different devices. Under the premise of ensuring coverage and modeling quality, unnecessary scanning paths are reduced, and scanning efficiency and resource utilization are significantly improved.

[0101] Reference Manual Attached Figure 2 , showing a schematic structural diagram of a real-time modeling system in which a land and a flight scanning device work in coordination, provided by an embodiment of the present invention.

[0102] The present invention also provides a real-time modeling system for coordinated operation of a land and a flying scanning device, which is applied to the real-time modeling method for coordinated operation of a land and a flying scanning device, and comprises:

[0103] processor;

[0104] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the real-time modeling method for the coordinated operation of the land and flight scanning devices as described in the method embodiment is implemented.

[0105] The real-time modeling system for coordinated operation of land and flight scanning devices provided by the present invention can execute the real-time modeling method for coordinated operation of land and flight scanning devices and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0106] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0107] (1) In the present invention, by introducing the collaborative working mechanism of land and flight scanning devices, the high precision of land scanning and the flexible coverage of flight scanning are fully utilized, so that the scanning process can dynamically identify and supplement blind areas. Especially in complex environments, after the land scanning device completes the construction of the basic model, the flight scanning device can autonomously plan the path and scan the missing areas in the preliminary model, thereby greatly reducing the areas that cannot be covered by traditional scanning methods and realizing all-round and high-completeness three-dimensional environment modeling;

[0108] (2) In the present invention, during the data fusion process, the present invention innovatively introduces a weighted fusion method for land and flight scan data, combining the fineness of land scans with the wide coverage characteristics of flight scans. By dynamically weighting the contribution of different data sources according to the complexity of the environment, the final generated three-dimensional model is more accurate and detailed, especially in scenes with rich details or changing perspectives, which can significantly improve the accuracy and authenticity of the model;

[0109] (3) In the present invention, innovations are made in the optimization of the flight scanning path. The autonomous path planning and obstacle avoidance algorithm of the flight scanning device is used to enable it to adjust the flight path in real time in a dynamic environment, avoid repeated scanning, and reasonably arrange the scanning tasks of different devices. Under the premise of ensuring coverage and modeling quality, unnecessary scanning paths are reduced, and scanning efficiency and resource utilization are significantly improved.

[0110] The above contents are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0111] There are a few points to note:

[0112] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0113] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0114] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0115] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A real-time modeling method for collaborative operation of land and flight scanning devices, characterized in that: include: S1. Use the land scanning device to conduct a preliminary scan of the environment and generate data D land , including three-dimensional coordinate data, reflection intensity, surface normal vector and depth data, the data D land After processing, a preliminary three-dimensional model M is generated. init , providing a basis for subsequent flight scans; S2, start the flight scanning device, and scan the three-dimensional model M init The flight scanning device optimizes the path of the scanning area through autonomous flight planning and obstacle avoidance algorithm to obtain data D fly , and combine sensor data for real-time position correction; S3. Identify the areas that are not covered in the flight scan and generate the missing area mark A missing , the actual area is obtained by combining the three-dimensional model M init By comparing and obtaining, the scanning data and the area identification information are used to further infer the re-scanning path of the flying device; S4, fuse the land and flight scan data, calculate and generate the final 3D environment model M final In the data fusion process, the weighting coefficient α of the land scan data is introduced land and the weighting factor α of the flight scan data fly , weighted calculation of the contribution values ​​of the two data sources; S5. Based on the generated model and data analysis, the flight device automatically corrects the scanning path P fly , optimize the selection of flight routes to ensure the maximum scanning range and time. The optimization of flight paths is based on real-time feedback of data quality assessment, obstacle detection and scanning performance; S6. During the scanning process, the flight scanning device and the land scanning device exchange data in real time through wireless communication, and continuously calibrate and update the scanning results. After each scan, the model M final Corrected based on new data.

2. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 1, characterized in that: The S2 further comprises: In the selection of the scanning path of the flying device, the flying device selects the scanning path according to the existing scanning data D fly and target coverage area, calculate the potential path P fly The possibility of flight path selection is optimized by the following formula: Among them, P Fly is the optimized flight path, D Fly (p i ) is the path p i The amount of scan data, T fly (p i ) is the path p i The scanning time, f efffciency (p i ) is the path p i The efficiency function, taking into account obstacles and weather factors on the flight path, is the number of all potential paths.

3. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 2, characterized in that: The S4 further comprises: In the data fusion process, the scanning data D of the flying device and the land device land and D fly Considering the contribution, weighted fusion is achieved through the following formula: Among them, M final (x, y, z) is the final 3D model after fusion, M land (x, y, z) is the 3D model generated by land scanning data, M fly (x, y, z) is the 3D model generated by the flight scanning data, W land is the weighting coefficient of land scan data, W fly is the weighting coefficient of the flight scanning data. After weighted calculation, the final three-dimensional model M final Provided for use in subsequent path correction and optimization steps.

4. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 3, characterized in that: The determination of the weighting coefficient specifically includes: Calculate the spatial resolution of the data for both the land scanner and the flight scanner separately: Among them, R L and R F are the spatial resolutions of land scanning data and flight scanning data, respectively, L and D F are the effective detection distances of the land scanning device and the flight scanning device, N L and N F are the point cloud densities per unit distance for land and flight scanning devices, respectively; According to the ratio of spatial resolution, the basic weighting coefficients of land and flight scan data are preliminarily calculated: in, is the preliminary weighting factor for land scanning, is the preliminary weighting factor for flight scanning; The coefficient is adjusted according to the environmental complexity of the scanning area. The environmental complexity factor C represents the environmental complexity level. In a complex environment, the weighted coefficient of land scanning is increased. The adjusted coefficient is: Among them, W land and W fly are the adjusted weighting coefficients for land scanning and flight scanning respectively. Finally, in order to ensure that the sum of the weighting coefficients is 1, the adjusted coefficient W land and W fly Perform normalization.

5. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 4, characterized in that: include: To ensure that the data from the land scan and the flight scan match correctly, a completeness check of the scan data is required after the scan is completed: Among them, A missing Indicates the identification and calculated value of the missing area, D land (x i ,y i , z i ) represents the land scan data of the i-th point, represents the estimated value of the i-th point, w i Represents the weighting coefficient, representing the importance of the point, N points Indicates the number of scanned data points; The calculation formula of the weighting coefficient is as follows: Among them, w i is the weighting coefficient for missing region identification, N i is the number of adjacent scanning points within the specified radius, which is used to reflect the density of the scanning points. A is the area of ​​the specified range, which is used to normalize the density so that it is independent of the radius size. α i is the angle between the scanning angle and the normal, using cos(α i ) to reduce the weight of large angle scanning points.

6. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 5, characterized in that: The path optimization of the flight scanning device specifically includes: During the flight, the flight device detects obstacles in the path in real time, including static and dynamic obstacles, updates the true position based on sensor data, and corrects the flight path P. fly , avoiding obstacles and optimizing the path, calculating the corrected flight path To ensure that the scanning task is completed as planned, the path correction calculation is performed using the following formula: in, is the corrected flight path, f avoidance (p i ) is the path p i Evaluation of obstacle avoidance effectiveness, f efficiency (p i ) is the path p i The efficiency function, N paths is the number of all potential paths.

7. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 6, characterized in that: The path correction further comprises: Evaluate the priority of scanning tasks and calculate the priority P of different areas task , and adjust the flight path according to the mission importance. The flight path P is adjusted according to the mission priority and regional importance through the following formula fly Choices: in, is the optimized flight path, f importance (p i ) is the path p i The task importance function, f efficiency (p i ) is the path p i The efficiency function, N paths is the number of all potential paths.

8. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 1, characterized in that: Further including: During real-time scanning, the data density is calculated based on the number of point cloud data in the unit scanning area. The data density is used to reflect the level of detail of the current scanning area. When the data density is higher than a preset threshold, it means that the data in the area is detailed enough. According to the value range of the data density, the accuracy of the scanning device is adaptively adjusted. When the data density exceeds the threshold, the system reduces the scanning accuracy of the current area to save resources. When the data density is lower than the threshold, the scanning accuracy is increased to supplement the data and ensure the completeness of the model details. After completing the data density adjustment, the local model is evaluated and optimized in real time, and a precision adaptive model is generated according to the point cloud density distribution. Supplementary sampling points are added in low-density areas to improve accuracy, and redundant data is removed in high-density areas to optimize modeling speed.

9. A real-time modeling method for coordinated operation of land and flight scanning devices according to claim 1, characterized in that: Further including: Calculate the path coverage of the land scanning device and the flight scanning device using the distribution of the scanning path and the point cloud data. The coverage is used to reflect the coverage of the entire modeling area by the current scanning path. When the probability of the share does not meet the set minimum requirement, a supplementary path is automatically generated according to the uncovered area to guide the land scanning device and the flight scanning device to perform the supplementary scanning task. When the coverage rate meets the requirement, the supplementary scanning is skipped to save resources; According to the real-time position and coverage of the land scanning device and the flying scanning device, the scanning path is dynamically adjusted, and the flying scanning device is given priority to handle the difficult-to-reach elevation areas, and the land scanning device handles the flat areas, to ensure the coverage uniformity and modeling quality of different areas.

10. A real-time modeling system for coordinated operation of land and flight scanning devices, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the real-time modeling method for the coordinated operation of the land and flight scanning devices according to any one of claims 1 to 9 is implemented.

Images