Multi-radar fusion based on all-scene 3d environment modeling and real-time positioning method for ship's hold

Abstract

The embodiment of the application provides a multi-radar fusion cargo hold machine full-scene three-dimensional environment modeling and real-time positioning method, which is applied to the technical field of cargo hold operation, applied to a high-dust and unstructured closed cabin environment, and the method comprises the following steps: acquiring original point cloud data collected by multiple radars and evaluating the original point cloud data; dividing confidence point cloud data based on the evaluation data; registering and three-dimensionally reconstructing high-confidence point cloud data; generating an initial environment three-dimensional model; matching medium-confidence point cloud data; based on the matching result, optimizing the pose of the medium-confidence point cloud data and filtering noise; subsequently, fusing the processed medium-confidence point cloud data with the initial environment three-dimensional model and globally optimizing the processed medium-confidence point cloud data to generate a globally consistent three-dimensional environment map; and the application constructs a high-reliability three-dimensional modeling mechanism for a complex cabin environment, and realizes accurate positioning and stable operation of the cargo hold machine in a high-dust and unstructured closed cabin environment.

Description

Technical Field

[0001] This application relates to the field of cabin cleaning operation technology, and in particular to a method for full-scene 3D environment modeling and real-time positioning of cabin cleaning aircraft using multi-radar fusion. Background Technology

[0002] A cleaning machine is an automatic cleaning device used in industrial production, warehousing, or special enclosed environments. Its main function is to clean materials, dust, or impurities inside a compartment to ensure the cleanliness and safety of the equipment or production environment. With the development of automation and intelligent manufacturing, cleaning machines are widely used in modern factories, warehousing systems, and special environments (such as enclosed compartments with high dust concentrations).

[0003] When existing unmanned cabin cleaning machines are operating inside the cabin, they use radar on the cleaning machine to perceive the environment, thereby collecting point cloud data inside the cabin in real time. The collected data is then directly stitched or registered to generate a three-dimensional environment model, which is used to determine the location of the cleaning machine.

[0004] Existing technologies still have some shortcomings in practical use. In high-dust and unstructured enclosed cabin environments, due to high dust concentration, insufficient lighting, and complex cabin structures, the quality of point cloud data collected by different radars varies. Some point cloud data may have problems such as noise, missing data, or measurement errors. If all the collected point cloud data is directly used for 3D environment modeling, it will lead to a decrease in the accuracy of the 3D environment model after modeling, which will increase the deviation of the cabin cleaning machine positioning results based on the 3D environment model, and reduce the accuracy and safety of the operation process. Summary of the Invention

[0005] This application provides a multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft, which improves the accuracy of 3D environment modeling and positioning in high-dust and unstructured cabin environments.

[0006] To achieve the above objectives, this application adopts the following technical solution: This application provides a method for multi-radar fusion-based full-scene 3D environment modeling and real-time positioning of a cargo clearance aircraft. The method includes: Acquire raw point cloud data from multiple radars; The original point cloud data is subjected to quality assessment to obtain assessment data. Based on the assessment data, the original point cloud data is divided into a high-confidence point cloud data set and a medium-confidence point cloud data set. The point cloud data in the high-confidence point cloud set is registered to obtain the first pose transformation matrix set, and three-dimensional reconstruction is performed based on the first pose transformation matrix in the first pose transformation matrix set to generate an initial environment three-dimensional model. The point cloud data in the medium confidence point cloud data set is matched with the initial environment 3D model to obtain the second pose transformation matrix set; Based on the geometric constraints of the initial 3D environment model and the second pose transformation matrix in the second pose transformation matrix set, pose optimization and noise filtering are performed on the point cloud data in the medium confidence point cloud data set to obtain an optimized point cloud data set. The point cloud data in the optimized point cloud dataset is fused and globally optimized with the initial 3D environment model to generate a globally consistent 3D environment map. The three-dimensional environment map is used to determine the location of the cleaning machine.

[0007] In some possible implementations, the division of the original point cloud data into a high-confidence point cloud data set and a medium-confidence point cloud data set includes: Detect whether the evaluation data is within a preset threshold range; When the evaluation data is less than a first preset threshold, the point cloud data corresponding to the evaluation data will be divided into a high confidence point cloud data set. When the evaluation data is greater than the first preset threshold and less than the second preset threshold, the point cloud data corresponding to the evaluation data is classified into the medium confidence point cloud data for combination. When the evaluation data is greater than the second preset threshold, the point cloud data corresponding to the evaluation data is not divided.

[0008] In some possible implementations, the registration of point cloud data in the high-confidence point cloud set to obtain the first pose transformation matrix set includes: The point cloud with the most geometric features in the high-confidence point cloud dataset is determined as the reference point cloud. Align the remaining point cloud data in the high-confidence point cloud dataset with the reference point cloud to obtain the first pose transformation matrix set.

[0009] In some possible implementations, the step of performing pose optimization and noise filtering on the point cloud data in the medium-confidence point cloud dataset to obtain an optimized point cloud dataset includes: Pose optimization is performed on the point cloud data in the medium confidence point cloud dataset to obtain the first pose matrix set; Noise is filtered out from the first pose matrix in the first pose matrix set to obtain the second pose matrix set; The second pose matrix in the second pose matrix set is transformed to obtain an optimized point cloud data set.

[0010] In some possible implementations, the step of fusing and globally optimizing the point cloud data in the optimized point cloud dataset with the initial 3D environment model to generate a globally consistent 3D environment map includes: The point cloud data in the optimized point cloud dataset is fused with the initial 3D environmental model to obtain a fused model; The fusion model is globally optimized to generate a globally consistent 3D environment map.

[0011] In some possible implementations, the quality assessment of the raw point cloud data to obtain assessment data includes: The original point cloud data is evaluated based on the density, signal-to-noise ratio, and dust occlusion ratio of the point cloud to obtain evaluation data.

[0012] In some possible implementations, after generating a globally consistent 3D environment map, the method further includes: Obtain the latest collection of high-confidence point cloud data; The point cloud data in the newly collected high-confidence point cloud data set is compared with the three-dimensional environment map to obtain the difference comparison data. When the difference comparison data is greater than a preset threshold, the spatial sub-region of the three-dimensional environment map corresponding to the difference comparison data is identified. The point cloud data in the newly collected high-confidence point cloud dataset is compensated and fused with the spatial sub-region to obtain an updated 3D environment map.

[0013] In some possible implementations, when the difference comparison data is greater than a preset threshold, the location of the cleaning machine is determined based on the updated three-dimensional environment map.

[0014] In some possible implementations, the location of the cleaning machine is determined based on the three-dimensional environment map when the difference comparison data is less than or equal to a preset threshold.

[0015] In some possible implementations, multiple radars are installed around the cleaning machine, and these radars are suitable for high-dust environments.

[0016] As can be seen from the above technical solution, this application has the following beneficial effects: 1. This application achieves precise positioning and stable operation of the cleaning machine in high-dust and unstructured enclosed cabin environments. It not only significantly improves the accuracy and robustness of 3D environment modeling, but also differs from existing technologies that directly stitch together all point cloud data. This embodiment constructs a highly reliable 3D modeling mechanism for complex cabin environments by employing a technical path of point cloud data confidence leveling, high-confidence point cloud data modeling, medium-confidence point cloud data optimization, and globally consistent fusion. Therefore, under high-dust and complex structural conditions, it not only improves the accuracy of the cleaning machine's autonomous positioning and operational safety, reducing the need for manual adjustment due to inaccurate positioning in existing technologies, but also further reduces the rate of manual intervention in the automatic cleaning process. Simultaneously, it enhances the system's adaptability and stability in extreme environments, thereby achieving a dual improvement in operational efficiency and environmental modeling quality.

[0017] 2. By introducing an update mechanism, this application ensures the global consistency of the 3D environment map and enhances the map's adaptability to local environmental changes. Compared with the existing technology that relies on single modeling results for positioning, this embodiment can achieve continuous environmental perception and high-precision positioning in cabin scenarios, significantly improving the stability and operational safety of the cleaning machine in dynamic dust environments and complex structural environments, thereby further improving operational efficiency. Attached Figure Description

[0018] The invention will now be further described with reference to the accompanying drawings.

[0019] Figure 1 A flowchart illustrating the first multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cargo clearance aircraft, as provided in this application embodiment; Figure 2 A flowchart illustrating the second multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cargo clearance aircraft, as provided in this application embodiment; Figure 3 The flowchart illustrates the third multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cargo clearance aircraft, as provided in this application embodiment. Detailed Implementation

[0020] The terms "first," "second," and "third," etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0021] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0022] Research has revealed that, under current technology, in high-dust and unstructured enclosed cabin environments, the quality of point cloud data collected by different radars varies due to high dust concentration, insufficient lighting, and complex cabin structures. Some point cloud data may contain noise, missing data, or measurement errors. If all collected point cloud data is directly used for 3D environment modeling, the accuracy of the resulting 3D environment model will be reduced, leading to increased deviations in the cabin cleaning machine positioning results based on the 3D environment model, thus reducing the accuracy and safety of the operation. Example

[0023] To solve the above problems, such as Figure 1 As shown, this application provides a method for full-scene 3D environment modeling and real-time positioning of a cargo clearance aircraft using multi-radar fusion, the method comprising: Acquire raw point cloud data from multiple radars; The original point cloud data is subjected to quality assessment to obtain assessment data. Based on the assessment data, the original point cloud data is divided into a high-confidence point cloud data set and a medium-confidence point cloud data set. The point cloud data in the high-confidence point cloud set is registered to obtain the first pose transformation matrix set, and three-dimensional reconstruction is performed based on the first pose transformation matrix in the first pose transformation matrix set to generate an initial environment three-dimensional model. The point cloud data in the medium confidence point cloud data set is matched with the initial environment 3D model to obtain the second pose transformation matrix set; Based on the geometric constraints of the initial 3D environment model and the second pose transformation matrix in the second pose transformation matrix set, pose optimization and noise filtering are performed on the point cloud data in the medium confidence point cloud data set to obtain an optimized point cloud data set. The point cloud data in the optimized point cloud dataset is fused and globally optimized with the initial 3D environment model to generate a globally consistent 3D environment map. The three-dimensional environment map is used to determine the location of the cleaning machine.

[0024] In this embodiment, a closed-loop processing flow is constructed from multi-source point cloud acquisition, data reliability screening, local optimization to global map fusion by point cloud data quality assessment, confidence level processing, pose optimization and noise filtering, and global consistency modeling.

[0025] Specifically, the raw point cloud data is first acquired by multiple radars, and then analyzed using indicators such as point cloud density, signal-to-noise ratio and dust occlusion ratio in the quality assessment stage to obtain assessment data. This enables the confidence level of the point cloud data to be graded, ensuring that high-confidence point clouds are used first in the modeling process and avoiding low-quality data from directly affecting the overall modeling accuracy. After registering and initially modeling the high-confidence point cloud set, the pose optimization and noise filtering of the medium-confidence point cloud data are performed by matching it with the initial environmental 3D model to obtain an optimized point cloud set. This effectively eliminates error points caused by dust interference or structural occlusion, thus improving the reliability of the point cloud data. Based on this, the optimized point cloud data is fused and globally optimized with the initial 3D environmental model to generate a globally consistent 3D environmental map. This achieves high-precision modeling of complex cabin scenes and avoids map distortion caused by directly using point cloud data to generate 3D environmental models.

[0026] This method achieves precise positioning and stable operation of the cleaning machine in high-dust and unstructured enclosed cabin environments. It significantly improves the accuracy and robustness of 3D environment modeling. Unlike existing technologies that directly stitch together all point cloud data, this embodiment constructs a highly reliable 3D modeling mechanism for complex cabin environments by employing a technical path of point cloud data confidence grading, high-confidence point cloud data modeling, medium-confidence point cloud data optimization, and globally consistent fusion. Therefore, under high-dust and complex structural conditions, it not only improves the accuracy of the cleaning machine's autonomous positioning and operational safety, reducing the need for manual adjustments due to inaccurate positioning in existing technologies, but also further reduces the rate of human intervention in the automatic cleaning process. Simultaneously, it enhances the system's adaptability and stability in extreme environments, thus achieving a dual improvement in operational efficiency and environmental modeling quality. Example

[0027] This embodiment is based on Embodiment 1, and supplements some of the contents of Embodiment 1 in detail; like Figure 1 and Figure 2 As shown, specifically: multiple radars are installed around the cleaning machine, and the radars are suitable for high dust environments; The process of performing a quality assessment on the original point cloud data to obtain assessment data includes: The original point cloud data is evaluated based on the density, signal-to-noise ratio, and dust occlusion ratio of the point cloud to obtain evaluation data. The step of dividing the original point cloud data into a high-confidence point cloud data set and a medium-confidence point cloud data set includes: Detect whether the evaluation data is within a preset threshold range; When the evaluation data is less than a first preset threshold, the point cloud data corresponding to the evaluation data will be divided into a high confidence point cloud data set. When the evaluation data is greater than the first preset threshold and less than the second preset threshold, the point cloud data corresponding to the evaluation data is classified into the medium confidence point cloud data for combination. When the evaluation data is greater than the second preset threshold, the point cloud data corresponding to the evaluation data is not divided.

[0028] Specifically, the above is merely an example, and those skilled in the art can make specific settings according to actual circumstances. No specific limitations are made here. The original point cloud data is acquired by multiple lidars installed around the cleaning machine. In the quality assessment stage, the original point cloud data is analyzed using indicators such as point cloud density, signal-to-noise ratio, and dust obscuration ratio to obtain corresponding assessment data. Based on the assessment data, point cloud data of different quality levels are graded to ensure that high-confidence point cloud data is used first in the subsequent modeling process, thereby avoiding the direct use of low-quality data that would reduce the overall modeling accuracy. Example

[0029] This embodiment is based on Embodiment 1 and Embodiment 2, and specifically supplements some of the contents of Embodiment 1; like Figure 1 Specifically, the process of registering the point cloud data in the high-confidence point cloud set to obtain the first pose transformation matrix set includes: The point cloud with the most geometric features in the high-confidence point cloud dataset is determined as the reference point cloud. Align the remaining point cloud data in the high-confidence point cloud dataset with the reference point cloud to obtain the first pose transformation matrix set; The step of performing pose optimization and noise filtering on the point cloud data in the medium-confidence point cloud dataset to obtain an optimized point cloud dataset includes: Pose optimization is performed on the point cloud data in the medium confidence point cloud dataset to obtain the first pose matrix set; Noise is filtered out from the first pose matrix in the first pose matrix set to obtain the second pose matrix set; Transform the second pose matrix in the second pose matrix set to obtain an optimized point cloud data set; The step of fusing and globally optimizing the point cloud data in the optimized point cloud dataset with the initial 3D environment model to generate a globally consistent 3D environment map includes: The point cloud data in the optimized point cloud dataset is fused with the initial 3D environmental model to obtain a fused model; The fusion model is globally optimized to generate a globally consistent 3D environment map.

[0030] Specifically, after dividing the high-confidence point cloud data, an initial 3D environmental model is generated by registering and reconstructing the high-confidence point cloud data. Subsequently, the medium-confidence point cloud data is optimized to remove error points caused by dust interference or structural occlusion, further improving the reliability of the medium-confidence point cloud data and obtaining an optimized point cloud data set. Furthermore, the point cloud data in the optimized point cloud data set is fused with the initial 3D environmental model, and global optimization is performed after fusion. Global optimization can avoid map distortion caused by accumulated errors during multi-radar stitching, achieving high-precision modeling of complex cabin scenes and obtaining a globally consistent 3D environmental map.

[0031] Unlike existing technologies that directly stitch together all point cloud data for modeling, this embodiment first performs preliminary modeling on high-confidence point cloud data, then optimizes medium-confidence point cloud data, removes erroneous data, and improves the reliability of medium-confidence point cloud data. Finally, through fusion and global optimization, a globally consistent high-precision 3D environment map is obtained. Through the above method, this embodiment achieves accurate positioning of the cleaning machine in high-dust and unstructured enclosed cabin environments. Example

[0032] This embodiment is based on Embodiment 1, Embodiment 2 and Embodiment 3, and further extends the solution; like Figure 3 As shown, specifically: after generating a globally consistent 3D environment map, the method further includes: Obtain the latest collection of high-confidence point cloud data; The point cloud data in the newly collected high-confidence point cloud data set is compared with the three-dimensional environment map to obtain the difference comparison data. When the difference comparison data is greater than a preset threshold, the spatial sub-region of the three-dimensional environment map corresponding to the difference comparison data is identified. The point cloud data in the newly collected high-confidence point cloud dataset is compensated and fused with the spatial sub-region to obtain an updated 3D environment map.

[0033] When the difference comparison data is greater than a preset threshold, the location of the cleaning machine is determined based on the updated three-dimensional environment map.

[0034] When the difference comparison data is less than or equal to a preset threshold, the location of the cleaning machine is determined based on the three-dimensional environment map.

[0035] In this embodiment, after generating a globally consistent 3D environment map, a difference comparison and local compensation mechanism is further introduced to improve the real-time updating capability and positioning accuracy of the environment map in a dynamic cabin environment.

[0036] Specifically, the above are just examples. Those skilled in the art can make settings according to the actual situation. No specific limitations are made here. By comparing differences, it is possible to identify local differences in the cabin environment caused by dust disturbance, displacement of accumulated materials or structural changes, thereby realizing timely perception of dynamic changes in the environment. When the difference comparison data is greater than the preset threshold, the compensation fusion process of this embodiment can quickly correct the environmental model of the local area of ​​the cabin while maintaining global consistency, avoiding the problem of map decoupling from the actual scene due to environmental changes. After the updated three-dimensional environmental map is generated, the cabin cleaning machine determines the current position based on the updated three-dimensional environmental map, realizing high-precision positioning in dynamic environment. When the difference comparison data is less than or equal to a preset threshold, it indicates that the current cabin environment has changed little. The method directly determines the location of the cleaning machine based on the generated three-dimensional environment map without the need for additional compensation fusion, thereby reducing unnecessary calculations.

[0037] This embodiment introduces an update mechanism, which not only ensures the global consistency of the 3D environment map, but also enhances the map's adaptability to local environmental changes. Compared with the existing technology that relies on single modeling results for positioning, this embodiment can achieve continuous environmental perception and high-precision positioning in cabin scenarios, significantly improving the stability and operational safety of the cleaning machine in dynamic dust environments and complex structural environments, thereby further improving operational efficiency.

[0038] The technologies mentioned in the above embodiments are all mature existing technologies. This application solves the problem of poor accuracy of the three-dimensional environment model in high dust and unstructured cabin environments through methodological innovation. The method of this application improves the accuracy of the modeled three-dimensional environment map and the accuracy of positioning.

[0039] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this application. Various changes and modifications can be made to this application without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of this application as claimed. The scope of protection of this application is defined by the appended claims and their equivalents.

Claims

1. A multi-radar fusion-based full-scene 3D environment modeling and real-time positioning method for cabin cleaning aircraft, applied to high-dust and unstructured confined cabin environments, characterized by: The method includes: Acquire raw point cloud data from multiple radars; Based on the density, signal-to-noise ratio, and dust occlusion ratio of the point cloud, the original point cloud data is quality-assessed to obtain assessment data. Based on the assessment data, the original point cloud data is divided into a high-confidence point cloud data set and a medium-confidence point cloud data set. The point cloud data in the high-confidence point cloud data set is registered to obtain the first pose transformation matrix set, and three-dimensional reconstruction is performed based on the first pose transformation matrix in the first pose transformation matrix set to generate an initial environment three-dimensional model. The point cloud data in the medium confidence point cloud data set is matched with the initial environment 3D model to obtain the second pose transformation matrix set; Based on the geometric constraints of the initial 3D environment model and the second pose transformation matrix in the second pose transformation matrix set, pose optimization and noise filtering are performed on the point cloud data in the medium confidence point cloud data set to obtain an optimized point cloud data set. The point cloud data in the optimized point cloud dataset is fused and globally optimized with the initial 3D environment model to generate a globally consistent 3D environment map. The three-dimensional environment map is used to determine the location of the cleaning machine.

2. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 1, characterized in that, The step of dividing the original point cloud data into a high-confidence point cloud data set and a medium-confidence point cloud data set includes: Detect whether the evaluation data is within a preset threshold range; When the evaluation data is less than a first preset threshold, the point cloud data corresponding to the evaluation data will be divided into a high confidence point cloud data set. When the evaluation data is greater than the first preset threshold and less than the second preset threshold, the point cloud data corresponding to the evaluation data will be divided into the medium confidence point cloud data set. When the evaluation data is greater than the second preset threshold, the point cloud data corresponding to the evaluation data is not divided.

3. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 1, characterized in that, The process of registering the point cloud data in the high-confidence point cloud dataset to obtain the first pose transformation matrix set includes: The point cloud with the most geometric features in the high-confidence point cloud dataset is determined as the reference point cloud. Align the remaining point cloud data in the high-confidence point cloud dataset with the reference point cloud to obtain the first pose transformation matrix set.

4. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 1, characterized in that, The step of performing pose optimization and noise filtering on the point cloud data in the medium-confidence point cloud dataset to obtain an optimized point cloud dataset includes: Pose optimization is performed on the point cloud data in the medium confidence point cloud dataset to obtain the first pose matrix set; Noise is filtered out from the first pose matrix in the first pose matrix set to obtain the second pose matrix set; The second pose matrix in the second pose matrix set is transformed to obtain an optimized point cloud data set.

5. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 1, characterized in that, The step of fusing and globally optimizing the point cloud data in the optimized point cloud dataset with the initial 3D environment model to generate a globally consistent 3D environment map includes: The point cloud data in the optimized point cloud dataset is fused with the initial 3D environmental model to obtain a fused model; The fusion model is globally optimized to generate a globally consistent 3D environment map.

6. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 1, characterized in that, After generating a globally consistent 3D environment map, the method further includes: Obtain the latest collection of high-confidence point cloud data; The point cloud data in the newly collected high-confidence point cloud data set is compared with the three-dimensional environment map to obtain the difference comparison data. When the difference comparison data is greater than a preset threshold, the spatial sub-region of the three-dimensional environment map corresponding to the difference comparison data is identified. The point cloud data in the newly collected high-confidence point cloud dataset is compensated and fused with the spatial sub-region to obtain an updated 3D environment map.

7. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 6, characterized in that, When the difference comparison data is greater than a preset threshold, the location of the cleaning machine is determined based on the updated three-dimensional environment map.

8. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 6, characterized in that, When the difference comparison data is less than or equal to a preset threshold, the location of the cleaning machine is determined based on the three-dimensional environment map.

9. The multi-radar fusion method for full-scene 3D environment modeling and real-time positioning of a cabin cleaning aircraft according to claim 1, characterized in that, The cleaning machine is equipped with multiple radars, which are suitable for high-dust environments.

Images