A multi-sensor constraint based robust perception method for near-wall environments

Abstract

The application discloses a kind of near-wall environment robust perception methods based on multi-sensor constraint.The method first acquires laser radar point cloud, millimeter wave radar and carries out preprocessing to multiple-source data;Then, through the region of interest screening and structure consistency segmentation, the main structure point set in the complex wall is extracted to carry out plane fitting;Subsequently, a joint optimization model is constructed by fusing point-to-plane geometric residual and multi-source observation consistency residual, and stable estimation of environment parameters is achieved.The application further introduces an uncertainty evaluation mechanism, which comprehensively considers geometric error, environment degradation index and consistency error, quantifies the reliability of the perception result, and accordingly adaptively adjusts the sampling strategy.The application effectively solves the perception degradation problem caused by single environment structure and non-ideal wall interference in complex scenarios such as high-rise rescue, and significantly improves the safety and robustness of the near-wall operation of the aircraft.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing and the field of intelligent sensing and safety assurance technology for low-altitude manned aircraft, and in particular, it is a robust sensing method for near-wall environments based on multi-sensor constraints. Background Technology

[0002] In urban high-rise fire rescue scenarios, aircraft need to perform tasks such as hovering, reconnaissance, and personnel search and rescue near building facades. However, such application scenarios typically involve complex conditions such as dense smoke, drastic changes in lighting, and monotonous environmental structures, which can easily interfere with the sensing performance of sensors. For example, smoke can shorten the effective propagation distance of laser signals, unstable lighting can reduce the imaging quality of visual sensors, and large areas of flat walls can lead to the loss of environmental geometric features, thus causing sensing degradation. Under these circumstances, traditional environmental sensing methods relying on single sensors often suffer from large measurement fluctuations, weak anti-interference capabilities, and may even fail completely under extreme conditions, making it difficult to meet the high reliability requirements of close-to-wall operations.

[0003] To enhance the perception capabilities of aircraft in complex environments, existing technologies typically employ a single distance sensor (such as lidar or millimeter-wave radar) or a simple multi-sensor fusion method to measure the distance between the aircraft and a wall. For example, some solutions use lidar to acquire point clouds and fit them to a plane to estimate the distance, or use the ranging results directly output by millimeter-wave radar for auxiliary judgment. However, most of these methods only stay at the level of "data acquisition and fusion," lacking in-depth modeling of the environmental structural characteristics, systematic analysis of the consistency between multi-source perception results, and a quantitative evaluation mechanism for the reliability of perception results. When there are incomplete wall structures such as windows and balconies, as well as complex factors such as strong reflections and occlusions, the plane fitting deviation can easily increase or the sensor outputs can be inconsistent, thus affecting the accuracy of the final perception results.

[0004] Furthermore, during near-wall flight, the uniformity of the environmental geometry leads to information gaps in some directions, resulting in reduced system observability—a phenomenon known as perception degradation. Current technologies generally fail to effectively identify and address this degradation, causing the system to output seemingly "normal" perception results even with insufficient information, but with low actual reliability and significant safety risks.

[0005] In summary, existing technologies still have the following shortcomings in complex near-wall environments: first, they lack robust modeling methods for non-ideal wall structures; second, they lack a unified modeling mechanism for the consistency of multi-source sensing information; and third, they lack quantitative assessment methods for the degree of environmental degradation and the reliability of sensing results. Therefore, it is necessary to propose an environmental sensing method capable of achieving stable near-wall structure modeling in complex environments and quantitatively assessing the uncertainty of sensing results, in order to improve the sensing robustness and safety of aircraft in extreme scenarios such as high-rise fire rescue. Summary of the Invention

[0006] The purpose of this invention is to address the deficiencies or shortcomings of existing technologies by proposing a robust near-wall environment sensing method and system based on multi-sensor constraints. This method acquires multimodal observation information, constructs geometric constraints characterizing the environmental structure and dynamic constraints characterizing relative motion, and performs structural modeling and consistency analysis of the near-wall environment. Furthermore, an uncertainty quantification mechanism is introduced to evaluate and express the reliability of the sensing results, thereby achieving highly reliable sensing output of the distance between the aircraft and the wall and environmental structural parameters. This method is suitable for environmental sensing needs in environments with weak near-wall features, easy degradation, and complex interference, and can provide reliable sensing basis for subsequent decision-making or control.

[0007] The technical solution to achieve the objective of this invention is: a robust near-wall environment sensing method based on multi-sensor constraints, the method comprising the following steps:

[0008] Step 1, Multi-source information acquisition: Acquire multi-source sensing data of the environment surrounding the aircraft, including environmental spatial structure information, relative distance and speed information between the aircraft and the environment;

[0009] Step 2, Target Environment Structure Extraction: Based on the aircraft's motion state or mission requirements, the near-wall region of the target is selected from the multi-source sensing data, and structural consistency modeling is performed to extract the main wall features;

[0010] Step 3, Multi-source constraint construction: Based on the main wall features, a multi-source constraint model is constructed. The multi-source constraint model includes geometric constraints for characterizing spatial relationships and consistency constraints for characterizing relative motion relationships. The environmental structural parameters are optimized and estimated by minimizing the joint residual model.

[0011] Step 4, Uncertainty Assessment: Based on the geometric structure fitting error, the degree of environmental degradation, and the consistency of multi-source observations, an uncertainty assessment model is constructed to quantify the reliability of the sensing results;

[0012] Step 5, Robust perception output: Adaptively adjust the perception parameters based on the uncertainty assessment results, and output the perception results including distance, environmental structure parameters and uncertainty information.

[0013] Furthermore, in step 1, the multi-source sensing data includes at least three-dimensional point cloud data collected by lidar and distance and velocity observation data collected by millimeter-wave radar.

[0014] Furthermore, step 1 also includes preprocessing of multi-source sensing data, which includes at least time alignment, denoising, and outlier removal of sensing data from different modalities.

[0015] Furthermore, step 2 specifically includes:

[0016] Step 2-1: Based on the current attitude and direction of motion of the aircraft, construct a forward region of interest (ROI) and extract the candidate point cloud set within the ROI as the target near-wall region.

[0017] Step 2-2: Calculate the local normal vector of each point in the candidate point cloud set, and perform clustering and segmentation based on the normal angle constraint and spatial neighborhood constraint to obtain multiple sub-regions with structural consistency;

[0018] Steps 2-3: Select the largest connected region as the principal wall point set for plane fitting to obtain the wall normal vector and plane parameters.

[0019] Furthermore, the forward region of interest (ROI) in step 2-1 is represented as:

[0020]

[0021] Simultaneously, in step 2-1, point clouds satisfying the following formula are specifically extracted to construct a candidate point cloud set:

[0022]

[0023] In the formula, Point The angle between the aircraft and its forward direction. Indicates the first The three-dimensional coordinates of a point cloud point These are the coordinate values ​​in the x, y, and z directions, respectively. This represents the forward unit direction vector of the aircraft. Indicates the angle threshold. This represents the distance threshold.

[0024] Furthermore, the clustering conditions for cluster segmentation in step 2-2 are as follows:

[0025]

[0026] In the formula, They represent points respectively. , The local normal vector, They represent the first The 3D coordinates of the first point cloud point and the... The three-dimensional coordinates of a point cloud point Indicates the normal consistency threshold. This represents the neighborhood distance threshold.

[0027] Further, in step 3, the multi-source constraint model is expressed as:

[0028]

[0029] in,

[0030]

[0031]

[0032] In the formula, This represents the point-to-plane residual, and the geometric constraints are implemented by constructing a point-to-plane residual model. The consistency residual is represented by the consistency constraint, which is achieved by constructing a consistency residual model between the sensor's direct observation distance and the projected distance calculated based on the geometric model. respectively , The weighting coefficients satisfy ; It is the first point cloud coordinates, It is a reference point on the plane. It is the unit normal vector; It is the distance between the aircraft and the target wall calculated by the geometric model. The distance between the aircraft and the target wall is directly measured by the sensor.

[0033] Furthermore, the uncertainty assessment model described in step 4 is expressed as follows:

[0034]

[0035] In the formula, Indicators representing uncertainty This represents the geometric fit error. Indicators of environmental degradation This represents consistency error, used to indicate the difference in observations between different sensors. All are weighting coefficients, and .

[0036] Furthermore, the method for assessing the degree of environmental degradation includes:

[0037] Construct the Hessian matrix of the observation information matrix and perform eigenvalue decomposition. Define the degradation index by the distribution ratio of the largest and smallest eigenvalues. The larger the degradation index, the weaker the constraint of environmental geometric features on perception, expressed as:

[0038]

[0039] In the formula, , These are the largest and smallest eigenvalues, respectively.

[0040] The geometric structure fitting error is expressed as:

[0041]

[0042] The consistency error is expressed as:

[0043]

[0044] In the formula, It represents the number of points involved in the fitting process.

[0045] Further, in step 5, the adaptive adjustment includes: based on an uncertainty index Dynamically adjust the point cloud sampling strategy and observation weights to remove abnormal observation points with residuals greater than a preset threshold.

[0046] Preferably, the expression for the dynamically adjusted point cloud sampling strategy is:

[0047]

[0048] The output perception result in step 5 is as follows:

[0049]

[0050] In the formula, yes , It is the plane normal. Indicates the basic sampling quantity. This indicates the adjusted number of point cloud samples. This indicates an adjustable parameter, the value of which is determined experimentally based on the degree of environmental degradation. Indicates the degradation index.

[0051] Compared with the prior art, the significant advantages of this invention are:

[0052] (1) This invention addresses the problem of non-ideal planes such as windows, balconies, and ancillary structures in complex near-wall environments such as urban high-rise buildings, and proposes an environmental modeling method based on structural consistency constraints. This method performs normal consistency and spatial continuity segmentation on the point cloud, extracts only the main structure point set for planar modeling, effectively suppresses the influence of outliers and structural omissions on modeling accuracy, and thus can still obtain stable and reliable environmental structural parameters under non-complete wall conditions.

[0053] (2) This invention constructs a unified observation model that integrates geometric and dynamic constraints. Through joint optimization of point-to-plane residuals and multi-source observation consistency residuals, it achieves high-precision estimation of environmental structures and relative distances to aircraft. Compared with traditional single-sensor or simple fusion methods, this invention can make full use of the complementary relationship between multi-source information to improve the accuracy and robustness of the sensing results.

[0054] (3) This invention proposes a unified uncertainty assessment method that integrates geometric error, degradation degree, and consistency of multi-source observations, thereby enabling a quantitative expression of the credibility of the sensing results. Compared with existing methods that only output a single sensing result, this invention can simultaneously output uncertainty information, enabling the system to assess its own sensing reliability.

[0055] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0056] Figure 1 This is a flowchart of a robust near-wall environment sensing method based on multi-sensor constraints.

[0057] Figure 2 This is a flowchart illustrating the specific algorithm of the present invention in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0060] In one embodiment, combined Figure 1 and Figure 2 A robust near-wall environment sensing method based on multi-sensor constraints is provided, the method comprising the following steps:

[0061] Step 1, Multi-source information acquisition: Acquire multi-source sensing data of the environment surrounding the aircraft, including environmental spatial structure information, relative distance and speed information between the aircraft and the environment;

[0062] Step 2, Target Environment Structure Extraction: Based on the aircraft's motion state or mission requirements, the near-wall region of the target is selected from the multi-source sensing data, and structural consistency modeling is performed to extract the main wall features;

[0063] Step 3, Multi-source constraint construction: Based on the main wall features, a multi-source constraint model is constructed. The multi-source constraint model includes geometric constraints for characterizing spatial relationships and consistency constraints for characterizing relative motion relationships. The environmental structural parameters are optimized and estimated by minimizing the joint residual model.

[0064] Step 4, Uncertainty Assessment: Based on the geometric structure fitting error, the degree of environmental degradation, and the consistency of multi-source observations, an uncertainty assessment model is constructed to quantify the reliability of the sensing results;

[0065] Step 5, Robust perception output: Adaptively adjust the perception parameters based on the uncertainty assessment results, and output the perception results including distance, environmental structure parameters and uncertainty information.

[0066] Furthermore, in one embodiment, in step 1, the multi-source sensing data includes at least three-dimensional point cloud data acquired by lidar and distance and velocity observation data acquired by millimeter-wave radar.

[0067] Here, the point cloud information obtained by lidar is:

[0068]

[0069] The relative distance between the aircraft and obstacles is obtained using millimeter-wave radar. With velocity observation Environmental spatial structure information is used to characterize the geometric distribution characteristics of the surrounding environment, while relative distance and velocity information are used to characterize the dynamic relationship between the aircraft and the target environment.

[0070] Furthermore, in one embodiment, step 1 further includes preprocessing of the multi-source sensing data, which at least includes time alignment (unifying the sensing data from different modalities to the same timestamp). This includes denoising (point cloud denoising) and outlier removal, to ensure the consistency and effectiveness of subsequent processing.

[0071] Furthermore, in one embodiment, step 2 specifically includes:

[0072] Step 2-1: Based on the current attitude and direction of motion of the aircraft, construct a forward region of interest (ROI) and extract the candidate point cloud set within the ROI as the target near-wall region.

[0073] Step 2-2: Calculate the local normal vector of each point in the candidate point cloud set, and perform clustering and segmentation based on the normal angle constraint and spatial neighborhood constraint to obtain multiple sub-regions with structural consistency;

[0074] Steps 2-3: Select the largest connected region as the principal wall point set for plane fitting to obtain the wall normal vector and plane parameters.

[0075] Here, based on the aircraft's motion direction or mission requirements, the target near-wall region is selected from environmental data and structural modeling is performed. Regions of interest are filtered from the point cloud data to extract candidate point cloud sets along the aircraft's forward or operational direction. Considering that the facades of urban high-rise buildings typically consist of walls, windows, balconies, and ancillary structures, preferably, this invention models walls, windows, balconies, etc., as part of the wall structure in a unified manner.

[0076] Specifically, by calculating the local normal vector of each point in the point cloud and constructing structural consistency constraints based on spatial neighborhood relationships, the candidate point cloud is clustered and segmented to obtain multiple sub-regions with structural consistency. Based on this, according to the normal distribution and spatial location relationship, the sub-regions are divided into wall structures and convex structures (such as balconies). Simultaneously, structurally missing areas such as windows are identified through local density changes in the point cloud. The largest connected region is selected as the principal wall point set, and plane fitting is performed on it to obtain the wall normal vector and plane parameters. The fitting residual is calculated to characterize the accuracy of the structural modeling.

[0077] Preferably, in some embodiments, the forward region of interest (ROI) in step 2-1 is represented as:

[0078]

[0079] Meanwhile, in step 2-1, point clouds that satisfy the following formula are specifically extracted to construct a candidate point cloud set. :

[0080]

[0081] In the formula, Point The angle between the aircraft and its forward direction. Indicates the first The three-dimensional coordinates of a point cloud point These are the coordinate values ​​in the x, y, and z directions, respectively. This represents the forward unit direction vector of the aircraft. Indicates the angle threshold. This represents the distance threshold.

[0082] Preferably, in some embodiments, the clustering conditions for cluster segmentation in step 2-2 are:

[0083]

[0084] In the formula, They represent points respectively. , The local normal vector, They represent the first The 3D coordinates of the first point cloud point and the... The three-dimensional coordinates of a point cloud point Indicates the normal consistency threshold. This represents the neighborhood distance threshold.

[0085] Furthermore, in one embodiment, in step 3, the multi-source constraint model is represented as:

[0086]

[0087] in,

[0088]

[0089]

[0090] In the formula, This represents the point-to-plane residual, and the geometric constraints are implemented by constructing a point-to-plane residual model. The consistency residual is represented by the consistency constraint, which is achieved by constructing a consistency residual model between the sensor's direct observation distance and the projected distance calculated based on the geometric model. respectively , The weighting coefficients satisfy ; It is the first point cloud coordinates, It is a reference point on the plane. It is the unit normal vector; It is the distance between the aircraft and the target wall calculated by the geometric model. The distance between the aircraft and the target wall is directly measured by the sensor.

[0091] Here, by minimizing the residuals, we can achieve optimized estimation of the environmental structure and related parameters, thereby improving the accuracy and stability of the sensing results.

[0092] Furthermore, in one embodiment, the uncertainty assessment model described in step 4 is expressed as:

[0093]

[0094] In the formula, Indicators representing uncertainty This represents the geometric fit error. Indicators of environmental degradation This represents consistency error, used to indicate the difference in observations between different sensors. All are weighting coefficients, and Here, the uncertainty index is used to characterize the reliability of the current perception results, to determine whether the current environment is in a degraded state, and to provide a credibility reference for subsequent decision-making.

[0095] Here, based on the completion of constraint modeling and residual calculation, an uncertainty assessment model is further constructed. The uncertainty indices consider geometric structure fitting error, environmental degradation degree, and multi-source observation consistency. Specifically, geometric structure fitting error can be characterized by residual statistics, environmental degradation degree can be assessed through the eigenvalue distribution of the observation information matrix, and multi-source observation consistency is measured by the differences between different observation results.

[0096] Preferably, in some embodiments, the method for assessing the degree of environmental degradation includes:

[0097] Construct the Hessian matrix of the observation information matrix and perform eigenvalue decomposition. Define the degradation index by the distribution ratio of the largest and smallest eigenvalues. The larger the degradation index, the weaker the constraint of environmental geometric features on perception, expressed as:

[0098]

[0099] In the formula, , These are the largest and smallest eigenvalues, respectively.

[0100] The geometric structure fitting error is expressed as:

[0101]

[0102] The consistency error is expressed as:

[0103]

[0104] In the formula, It represents the number of points involved in the fitting process.

[0105] The definition process of the degradation index is explained in detail here:

[0106] Constructing the Hessian matrix:

[0107]

[0108] Perform eigenvalue decomposition:

[0109]

[0110] Define degradation metrics:

[0111]

[0112] Furthermore, in one embodiment, step 5, the adaptive adjustment includes: based on an uncertainty index. Dynamically adjust the point cloud sampling strategy and observation weights to remove abnormal observation points with residuals greater than a preset threshold.

[0113] Here, when uncertainty is high or environmental degradation is significant, the point cloud sampling strategy and observation weights are dynamically adjusted to enhance the utilization of effective structural information and suppress the impact of anomalous observations. Simultaneously, outliers with large residuals are removed to improve overall modeling stability.

[0114] The system ultimately outputs the distance between the aircraft and the wall, the wall normal vector, and the corresponding uncertainty index. It also outputs the location distribution information of convex structures such as balconies and the spatial characteristics of areas with structural gaps such as windows. By outputting perception results containing uncertainty information, a reliable basis can be provided for subsequent flight control or decision-making, thereby improving the system's robustness and adaptability in complex near-wall environments.

[0115] Preferably, in some embodiments, the expression for the dynamically adjusted point cloud sampling strategy is:

[0116]

[0117] The output perception result in step 5 is as follows:

[0118]

[0119] In the formula, yes , It is the plane normal. Indicates the basic sampling quantity. This indicates the adjusted number of point cloud samples. This indicates an adjustable parameter, the value of which is determined experimentally based on the degree of environmental degradation. Indicates the degradation index.

[0120] Preferably, in some embodiments, the perception results also include the location distribution information of the balcony protruding structure and the spatial characteristics of the window structure missing area identified by the structure extraction module.

[0121] As can be seen from the above, the method proposed in this invention has the following characteristics: (1) This invention proposes a modeling method based on structural consistency constraints for non-ideal planes such as windows, balconies and ancillary structures in urban high-rise buildings. By segmenting the normal consistency and spatial continuity, only the main structure point set is extracted for modeling, which effectively suppresses the influence of outliers and structural omissions on accuracy and ensures that stable parameters can still be obtained under non-complete wall conditions. (2) By constructing a unified observation model that integrates geometric constraints and dynamic constraints, the point-to-plane residual and the multi-source observation consistency residual are jointly optimized. Compared with single sensor or simple fusion methods, this method can make full use of the complementary relationship between multi-source information to achieve high-precision estimation of environmental structure and relative distance. (3) This invention proposes a quantitative uncertainty assessment mechanism that comprehensively considers geometric fitting error, environmental degradation degree and observation consistency. The system can output uncertainty index synchronously and dynamically adjust the sampling strategy and observation weight according to the index, which improves the safety and adaptability in complex or degraded environments. (4) In addition to basic distance information, this invention can also output wall normal vectors, the location distribution of convex structures such as balconies, and the spatial characteristics of areas with missing structures such as windows. This multi-dimensional information provides more detailed and reliable basic data for subsequent flight control and decision-making.

[0122] In one embodiment, a robust near-wall environment sensing system based on multi-sensor constraints is provided, the system comprising:

[0123] The first module is used to acquire multi-source information: acquire multi-source sensing data of the environment around the aircraft, including environmental spatial structure information, relative distance and speed information between the aircraft and the environment;

[0124] The second module is used to extract the target environment structure: based on the motion state of the aircraft or mission requirements, the target near-wall region is screened from the multi-source sensing data, and structural consistency modeling is performed to extract the main wall features;

[0125] The third module is used to construct multi-source constraints: a multi-source constraint model is constructed based on the main wall features. The multi-source constraint model includes geometric constraints for characterizing spatial relationships and consistency constraints for characterizing relative motion relationships. The environmental structural parameters are optimized and estimated by minimizing the joint residual model.

[0126] The fourth module is used to implement uncertainty assessment: an uncertainty assessment model is constructed based on geometric structure fitting error, environmental degradation degree and multi-source observation consistency to quantify the reliability of the perception results;

[0127] The fifth module is used to achieve robust perception output: based on the uncertainty assessment results, the perception parameters are adaptively adjusted, and the perception results containing distance, environmental structure parameters and uncertainty information are output.

[0128] Specific limitations regarding the robust near-wall environment sensing system based on multi-sensor constraints can be found in the limitations of the robust near-wall environment sensing method based on multi-sensor constraints mentioned above, and will not be repeated here. Each module in the aforementioned robust near-wall environment sensing system based on multi-sensor constraints can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0129] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:

[0130] Step 1, Multi-source information acquisition: Acquire multi-source sensing data of the environment surrounding the aircraft, including environmental spatial structure information, relative distance and speed information between the aircraft and the environment;

[0131] Step 2, Target Environment Structure Extraction: Based on the aircraft's motion state or mission requirements, the near-wall region of the target is selected from the multi-source sensing data, and structural consistency modeling is performed to extract the main wall features;

[0132] Step 3, Multi-source constraint construction: Based on the main wall features, a multi-source constraint model is constructed. The multi-source constraint model includes geometric constraints for characterizing spatial relationships and consistency constraints for characterizing relative motion relationships. The environmental structural parameters are optimized and estimated by minimizing the joint residual model.

[0133] Step 4, Uncertainty Assessment: Based on the geometric structure fitting error, the degree of environmental degradation, and the consistency of multi-source observations, an uncertainty assessment model is constructed to quantify the reliability of the sensing results;

[0134] Step 5, Robust perception output: Adaptively adjust the perception parameters based on the uncertainty assessment results, and output the perception results including distance, environmental structure parameters and uncertainty information.

[0135] For specific constraints on each step, please refer to the constraints on robust sensing methods for near-wall environments based on multi-sensor constraints mentioned above, which will not be repeated here.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:

[0137] Step 1, Multi-source information acquisition: Acquire multi-source sensing data of the environment surrounding the aircraft, including environmental spatial structure information, relative distance and speed information between the aircraft and the environment;

[0138] Step 2, Target Environment Structure Extraction: Based on the aircraft's motion state or mission requirements, the near-wall region of the target is selected from the multi-source sensing data, and structural consistency modeling is performed to extract the main wall features;

[0139] Step 3, Multi-source constraint construction: Based on the main wall features, a multi-source constraint model is constructed. The multi-source constraint model includes geometric constraints for characterizing spatial relationships and consistency constraints for characterizing relative motion relationships. The environmental structural parameters are optimized and estimated by minimizing the joint residual model.

[0140] Step 4, Uncertainty Assessment: Based on the geometric structure fitting error, the degree of environmental degradation, and the consistency of multi-source observations, an uncertainty assessment model is constructed to quantify the reliability of the sensing results;

[0141] Step 5, Robust perception output: Adaptively adjust the perception parameters based on the uncertainty assessment results, and output the perception results including distance, environmental structure parameters and uncertainty information.

[0142] For specific constraints on each step, please refer to the constraints on robust sensing methods for near-wall environments based on multi-sensor constraints mentioned above, which will not be repeated here.

[0143] In summary, this invention effectively solves the problem of perception degradation caused by simple environmental structure and non-ideal wall interference in complex scenarios such as high-rise rescue, significantly improving the safety and robustness of aircraft operating near walls. It is suitable for environmental perception needs in environments with weak near-wall features, easy degradation, and complex interference, and can provide a reliable perception basis for subsequent decision-making or control.

[0144] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A robust near-wall environment sensing method based on multi-sensor constraints, characterized in that, The method includes the following steps: Step 1, Multi-source information acquisition: Acquire multi-source sensing data of the environment surrounding the aircraft, including environmental spatial structure information, relative distance and speed information between the aircraft and the environment; Step 2, Target Environment Structure Extraction: Based on the aircraft's motion state or mission requirements, the near-wall region of the target is selected from the multi-source sensing data, and structural consistency modeling is performed to extract the main wall features; Step 3, Multi-source constraint construction: Based on the main wall features, a multi-source constraint model is constructed. The multi-source constraint model includes geometric constraints for characterizing spatial relationships and consistency constraints for characterizing relative motion relationships. The environmental structural parameters are optimized and estimated by minimizing the joint residual model. Step 4, Uncertainty Assessment: Based on the geometric structure fitting error, the degree of environmental degradation, and the consistency of multi-source observations, an uncertainty assessment model is constructed to quantify the reliability of the sensing results; Step 5, Robust perception output: Adaptively adjust the perception parameters based on the uncertainty assessment results, and output the perception results including distance, environmental structure parameters and uncertainty information; In step 3, the multi-source constraint model is expressed as: ； in, ； ； In the formula, This represents the point-to-plane residual, and the geometric constraints are implemented by constructing a point-to-plane residual model. The consistency residual is represented by the consistency constraint, which is achieved by constructing a consistency residual model between the sensor's direct observation distance and the projected distance calculated based on the geometric model. respectively , The weighting coefficients satisfy ; It is the first point cloud coordinates, It is a reference point on the plane. It is the unit normal vector; It is the distance between the aircraft and the target wall calculated by the geometric model. The distance between the aircraft and the target wall is directly measured by the sensor; The uncertainty assessment model described in step 4 is expressed as follows: ； In the formula, Indicators representing uncertainty This represents the geometric fit error. Indicators of environmental degradation This represents consistency error, used to indicate the difference in observations between different sensors. All are weighting coefficients, and ; The methods for assessing the degree of environmental degradation include: Construct the Hessian matrix of the observation information matrix and perform eigenvalue decomposition. Define the degradation index by the distribution ratio of the largest and smallest eigenvalues. The larger the degradation index, the weaker the constraint of environmental geometric features on perception, expressed as: ； In the formula, , These are the largest and smallest eigenvalues, respectively. The geometric structure fitting error is expressed as: ； The consistency error is expressed as: ； In the formula, It represents the number of points involved in the fitting process.

2. The robust near-wall environment sensing method based on multi-sensor constraints according to claim 1, characterized in that, In step 1, the multi-source sensing data includes at least three-dimensional point cloud data collected by lidar and distance and velocity observation data collected by millimeter-wave radar.

3. The robust near-wall environment sensing method based on multi-sensor constraints according to claim 1, characterized in that, Step 1 also includes preprocessing of multi-source sensing data, which includes at least time alignment, noise reduction and outlier removal of sensing data of different modalities.

4. The robust near-wall environment sensing method based on multi-sensor constraints according to claim 1, characterized in that, Step 2 specifically includes: Step 2-1: Based on the current attitude and direction of motion of the aircraft, construct a forward region of interest (ROI) and extract the candidate point cloud set within the ROI as the target near-wall region. Step 2-2: Calculate the local normal vector of each point in the candidate point cloud set, and perform clustering and segmentation based on the normal angle constraint and spatial neighborhood constraint to obtain multiple sub-regions with structural consistency; Steps 2-3: Select the largest connected region as the principal wall point set for plane fitting to obtain the wall normal vector and plane parameters.

5. The robust near-wall environment sensing method based on multi-sensor constraints according to claim 4, characterized in that, In step 2-1, the forward region of interest (ROI) is represented as: ； Simultaneously, in step 2-1, point clouds satisfying the following formula are specifically extracted to construct a candidate point cloud set: ； In the formula, Point The angle between the aircraft and its forward direction. Indicates the first The three-dimensional coordinates of a point cloud point These are the coordinate values ​​in the x, y, and z directions, respectively. This represents the forward unit direction vector of the aircraft. Indicates the angle threshold. This represents the distance threshold.

6. The robust near-wall environment sensing method based on multi-sensor constraints according to claim 4, characterized in that, The clustering conditions for cluster segmentation in step 2-2 are: ； In the formula, They represent points respectively. , The local normal vector, They represent the first The 3D coordinates of the first point cloud point and the... The three-dimensional coordinates of a point cloud point Indicates the normal consistency threshold. This represents the neighborhood distance threshold.

7. The robust near-wall environment sensing method based on multi-sensor constraints according to claim 1, characterized in that, In step 5, the adaptive adjustment includes: based on the uncertainty index Dynamically adjust the point cloud sampling strategy and observation weights to remove abnormal observation points with residuals greater than a preset threshold.

Images