A positioning optimization method and system based on a visual auxiliary positioning algorithm

Abstract

The application relates to a positioning optimization method and system based on a visual auxiliary positioning algorithm, and the method comprises the following steps: acquiring dynamic instruction data of a patrol unmanned plane, and synchronously collecting body micro-vibration data and environment sensing image data; based on the dynamic instruction data and the body micro-vibration data, a predicted disturbance field is established through a physical-data hybrid driven disturbance prediction model; based on the environment sensing image data and the predicted disturbance field, an observed disturbance field is extracted; the predicted disturbance field and the observed disturbance field are compared in structure, and a prediction-observation consistency index is calculated; the prediction-observation consistency index is used as a real-time measurement of the reliability of visual data, the state information confidence of the visual auxiliary positioning algorithm is dynamically adjusted according to the real-time measurement, and finally, an optimized positioning result is obtained. The positioning reliability of the patrol unmanned plane in a high safety risk scene is effectively guaranteed against visual deception attacks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of measurement not specialized for specific variables, and in particular to a positioning optimization method and system based on a visual auxiliary positioning algorithm. BACKGROUND

[0002] In the field of measurement not specialized for specific variables, an increasingly important branch is the indirect measurement and evaluation of the operating state of complex systems, especially in intelligent equipment such as autonomous drones, the performance of the visual auxiliary positioning algorithm directly determines the safety and efficiency of its operation. However, traditional positioning algorithm optimization mainly focuses on improving the accuracy of position coordinates.

[0003] In the prior art, a BIM and unmanned aerial vehicle projection auxiliary automobile box girder control system and method with publication number CN118882720A and the name of a BIM and unmanned aerial vehicle projection auxiliary automobile box girder control system and method solve the problem that the actual installation position of the box girder and the positioning position of the design drawing will be different, and secondary adjustment is needed when traditional workers install the box girder according to construction drawings, naked eye observation and past experience.

[0004] The existing visual auxiliary positioning technology, represented by visual simultaneous localization and mapping (SLAM) and visual inertial odometry (VIO), has the following deep technical defects when applied to autonomous inspection tasks of high-security level critical infrastructure:

[0005] 1. "Unconditional trust" in visual information sources: The core defect of existing technology lies in the inherent assumption of its algorithm architecture, that is, the image information obtained by the visual sensor is physically real. These algorithms are good at processing noise caused by sensor characteristics or environmental natural factors, but they lack an internal mechanism to falsify the visual information source itself. Therefore, when facing carefully designed malicious visual deception attacks, such as playing pre-recorded or real-time rendered false scene videos through high-resolution screens, existing algorithms will process them as real inputs, resulting in complete hijacking of the positioning result without the system itself being aware of it, which constitutes a fatal security risk;

[0006] 2. Limitations of multi-modal fusion: Although the scheme of fusing inertial measurement units (IMUs) and other non-visual sensors can provide some robustness when visual information is temporarily lost or degraded, this fusion is based on the assumption that the data of each sensor is self-consistent in the physical world, and cannot resist a more advanced "data poisoning" attack, that is, the attacker continuously and slowly injects false visual streams that do not completely match the movement of the drone, which incorrectly guides the VIO algorithm to calibrate the IMU bias. This incorrect calibration will gradually accumulate, eventually causing the entire fusion positioning system to produce a huge, undetectable positioning drift in the internal logic without the system itself being aware of it;

[0007] 3. Lack of physical world interaction verification: All existing optimization methods, whether based on filtering or graph optimization, essentially perform information processing at the mathematical and statistical level, lacking a closed-loop verification link, i.e., the UAV's own behavior cannot be verified to produce a physically consistent and observable response in the external world it perceives. This deficiency makes it impossible to distinguish between a highly realistic virtual environment and a real physical environment, as pure digital simulation can perfectly simulate visual and inertial data, but cannot simulate real physical interactions.

[0008] The above three deep technical defects of the existing visual aided positioning technology result in inaccurate positioning results of autonomous UAVs and other intelligent equipment, and lack of ability to resist visual deception attacks. SUMMARY

[0009] To solve the above technical problems, the present application provides a positioning optimization method and system based on a visual aided positioning algorithm.

[0010] The technical solution adopted by the present application is:

[0011] In a first aspect, a positioning optimization method based on a visual aided positioning algorithm is provided, applied to a patrol UAV, comprising:

[0012] Obtaining the dynamic instruction data of the patrol UAV, and synchronously collecting the body micro-vibration data and the environmental perception image data;

[0013] Based on the dynamic instruction data and the body micro-vibration data, a disturbance prediction model driven by physical-data hybrid is used to establish a predicted disturbance field for representing the environmental effect of the patrol UAV;

[0014] Based on the environmental perception image data and the predicted disturbance field, an observed disturbance field is extracted, which is used to represent the physical disturbance caused by the patrol UAV in the surrounding medium;

[0015] The predicted disturbance field and the observed disturbance field are structurally compared, and a prediction-observation consistency index is calculated;

[0016] The prediction-observation consistency index is used as a real-time measure of the reliability of visual data, and the state information confidence of the visual aided positioning algorithm is dynamically adjusted according to the real-time measure, and finally an optimized positioning result is obtained.

[0017] Further, obtaining the dynamic instruction data of the patrol UAV, and synchronously collecting the body micro-vibration data and the environmental perception image data, comprises:

[0018] Obtain the dynamic instruction data of the inspection unmanned aerial vehicle, construct a dynamic state feature vector, the dynamic state feature vector including a motor speed instruction and a body vibration response feature, the motor speed instruction being synchronized with a hardware timestamp, and the body vibration response feature being related to the body micro-vibration data;

[0019] Synchronously collect the body micro-vibration data, the visible light image frame and the thermal imaging image frame, which have the same hardware timestamp as the dynamic state feature vector;

[0020] Form the environment perception image data according to the visible light image frame and the thermal imaging image frame;

[0021] Synthetically consider the contribution of the motor speed instruction and the body vibration response feature to the final result, combine them by using a weighted average method, obtain a dynamic state feature value, and the dynamic state feature value is a numerical representation of the dynamic state feature vector;

[0022] Limit the output value range of the dynamic state feature value in the interval [0, 1]; when the dynamic state feature value output tends to be closer to 0, the dynamic state of the inspection unmanned aerial vehicle tends to be more stable and stationary; when the dynamic state feature value output tends to be closer to 1, the dynamic event experienced by the inspection unmanned aerial vehicle is higher in intensity.

[0023] Further, based on the dynamic instruction data and the body micro-vibration data, a disturbance prediction model driven by physical-data hybrid is used to establish a predicted disturbance field for representing the environmental effect of the inspection unmanned aerial vehicle, including:

[0024] Based on the motor speed instruction contained in the dynamic state feature value, the disturbance prediction model driven by physical-data hybrid is used to calculate a basic physical disturbance component representing the airflow disturbance in an ideal fluid state;

[0025] Based on the body vibration response feature contained in the dynamic state feature value, the disturbance prediction model is used to calculate a dynamic correction disturbance component representing the disturbance deviation caused by non-ideal factors;

[0026] The basic physical disturbance component and the dynamic correction disturbance component are vector superimposed to generate a predicted disturbance field for representing the environmental effect of the inspection unmanned aerial vehicle; the predicted disturbance field includes a predicted visual disturbance field representing the expected visual medium displacement and a predicted thermal disturbance field representing the expected thermal medium displacement.

[0027] Further, based on the environment perception image data and the predicted disturbance field, an observed disturbance field is extracted, including:

[0028] Based on the current environment perception image data at the current synchronization time point and the historical environment perception image data at the previous synchronization time point, an initial observed optical flow field containing all pixel motion information is calculated;

[0029] Based on the predicted disturbance field, the initial observation optical flow field is purified by suppressing the optical flow vectors inconsistent with the predicted disturbance field in the motion mode to obtain a filtering processing result caused by the physical disturbance actually caused by the inspection unmanned aerial vehicle;

[0030] The filtering processing result is taken as an observation disturbance field, which includes an observation visual disturbance field representing the real visual medium motion and an observation thermal disturbance field representing the real thermal medium motion.

[0031] Further, the predicted disturbance field and the observation disturbance field are structurally compared, and a prediction-observation consistency index is calculated, including:

[0032] The predicted disturbance field and the observation disturbance field are compared in multiple dimensions from the visible light channel and the thermal imaging channel to generate a visual consistency component and a thermal consistency component, respectively;

[0033] The thermal information entropy is calculated based on the thermal imaging image frame of the current environmental perception image data;

[0034] The thermal channel weight and the visual channel weight are obtained according to the thermal information entropy;

[0035] The prediction-observation consistency index is calculated according to the thermal channel weight, the visual channel weight, the visual consistency component, and the thermal consistency component.

[0036] Further, the value range of the prediction-observation consistency index is [0, 1];

[0037] When the prediction-observation consistency index output tends to be closer to 0, it indicates that the overall consistency degree of the predicted disturbance field and the observation disturbance field is lower;

[0038] When the prediction-observation consistency index output tends to be closer to 1, it indicates that the overall consistency degree of the predicted disturbance field and the observation disturbance field is higher.

[0039] Further, the prediction-observation consistency index is taken as a real-time measurement of visual data reliability, and the state information confidence of the visual aided positioning algorithm is dynamically adjusted according to the real-time measurement to obtain an optimized positioning result, including:

[0040] The prediction-observation consistency index is converted into a visual measurement confidence as a real-time measurement of visual data reliability by a preset nonlinear mapping function;

[0041] In the state estimation fusion process of the visual aided positioning algorithm, the visual measurement confidence is applied as a dynamic weight factor;

[0042] The state information confidence introduced by the visual measurement update is modulated by the dynamic weight factor to obtain the optimized positioning result.

[0043] Further, the preset nonlinear mapping function is a logistic function with adjustable parameters.

[0044] The state information confidence introduced by the visual measurement update is modulated by a dynamic weight factor, including:

[0045] The noise covariance matrix related to the visual measurement is adjusted by the dynamic weight factor;

[0046] When the dynamic weight factor increases, the noise covariance matrix is increased, so that the weight of the visual measurement in the state fusion process of the state information confidence calculation is reduced;

[0047] When the dynamic weight factor decreases, the noise covariance matrix is reduced, so that the weight of the visual measurement in the state fusion process of the state information confidence calculation is increased.

[0048] In a second aspect, a positioning optimization system based on a visual aided positioning algorithm is provided, applied to a patrol unmanned aerial vehicle, and the system comprises:

[0049] A data acquisition module is configured to acquire dynamic instruction data of the patrol unmanned aerial vehicle, and synchronously acquire body micro-vibration data and environment perception image data;

[0050] A predicted disturbance field establishment module is configured to establish a predicted disturbance field for representing the environmental effect of the patrol unmanned aerial vehicle based on the dynamic instruction data and the body micro-vibration data, by using a physical-data hybrid driven disturbance prediction model;

[0051] An observed disturbance field establishment module is configured to extract an observed disturbance field based on the environment perception image data and the predicted disturbance field, and the observed disturbance field is used to represent the physical disturbance caused by the patrol unmanned aerial vehicle in the surrounding medium;

[0052] A predicted-observed consistency index calculation module is configured to perform a structured comparison between the predicted disturbance field and the observed disturbance field, and calculate a predicted-observed consistency index;

[0053] An optimized positioning module is configured to use the predicted-observed consistency index as a real-time measurement of the reliability of the visual data, dynamically adjust the state information confidence of the visual aided positioning algorithm according to the real-time measurement, and finally obtain an optimized positioning result.

[0054] The present application has the following beneficial effects:

[0055] I. By using a physical-data hybrid driven disturbance prediction model, the physical disturbance field generated by the downwash airflow of the patrol unmanned aerial vehicle in response to the surrounding environmental medium is actively predicted based on the body micro-vibration data of the patrol unmanned aerial vehicle, and the predicted disturbance field is a theoretical expectation of the physical reality.

[0056] II. Extract the observed disturbance field from the environmental perception image data, accurately observe and purify the observed disturbance field caused by the actual unmanned aerial vehicle from the real image full of interference, and the observed disturbance field is the direct evidence from the physical reality;

[0057] III. By comparing the predicted disturbance field and the observed disturbance field in multiple dimensions, a prediction-observation consistency index is obtained, which directly quantifies the self-consistency of the world perceived by the inspection unmanned aerial vehicle and its physical interaction;

[0058] Based on the above points I, II and III, when the inspection unmanned aerial vehicle is in a false environment that cannot produce the expected physical interaction, the observed disturbance field will be greatly mismatched with the predicted disturbance field, resulting in a sharp decrease in the prediction-observation consistency index. The prediction-observation consistency index is converted into a visual measurement confidence, and the state information contribution of the visual data in the positioning algorithm is adjusted in real time and dynamically. When the visual measurement confidence is low, the weight of the visual measurement in the state fusion process of the state information confidence calculation is automatically reduced, thereby effectively resisting visual deception attacks and ensuring the positioning reliability in high-security risk scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 The flowchart of the positioning optimization method based on the visual auxiliary positioning algorithm of the present application;

[0060] Figure 2 The flowchart of the present application for establishing a predicted disturbance field;

[0061] Figure 3 The flowchart of the present application for calculating an observed disturbance field;

[0062] Figure 4 The flowchart of the present application for calculating a prediction-observation consistency index;

[0063] Figure 5 The flowchart of the present application for dynamic positioning optimization based on the prediction-observation consistency index;

[0064] Figure 6 The structural diagram of the positioning optimization system based on the visual auxiliary positioning algorithm of the present application. DETAILED DESCRIPTION

[0065] The present application will be further described below in conjunction with the drawings. The following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0066] As shown in Figure 1 , the embodiment of the present application provides a positioning optimization method based on a visual auxiliary positioning algorithm, applied to an inspection unmanned aerial vehicle, the positioning optimization method comprising:

[0067] 101, obtain the dynamic instruction data of the inspection UAV, and synchronously collect the body micro-vibration data and the environment perception image data;

[0068] In this embodiment, the dynamic instruction data of the inspection UAV is obtained, and a dynamic state feature vector V dyn is constructed dyn , which includes a motor speed instruction C rpm and a body vibration response feature R vib ; the motor speed instruction C rpm is synchronized with a hardware timestamp, and the body vibration response feature R vib is related to the body micro-vibration data;

[0069] The body micro-vibration data, a visible light image frame I vis and a thermal imaging image frame I th synchronized with the dynamic state feature vector have the same hardware timestamp; the body micro-vibration data is obtained by a three-axis inertial measurement unit with a sampling frequency not lower than 1 kHz; the motor speed instruction C rpm is a quantized control signal output by a flight controller; all data are time-stamped by a unified high-precision clock source to ensure that the time alignment error is within 1 ms; the environment perception image data is collected by a visible light sensor and a thermal imaging sensor; the environment perception image data is formed according to the visible light image frame and the thermal imaging image frame;

[0070] The specific calculation process of the dynamic state feature vector V dyn is as follows:

[0071] S1, at each synchronization time point t, two original data are collected, in sequence, a motor speed instruction C rpm and an original high-frequency acceleration signal: the motor speed instruction C rpm is a normalized speed that a flight controller expects each motor to reach; the parameter is directly read from a motor output port of the flight controller, and its value is a quantized value defined by an electronic speed controller ESC protocol, which is converted into a floating-point number in the interval [0, 1] by consulting the protocol specification;

[0072] The protocol of the electronic speed controller in this embodiment is DShot or PWM;

[0073] The original high-frequency acceleration signal is an original acceleration reading measured by an inertial measurement unit IMU on three orthogonal axes without low-pass filtering; the data is obtained by continuously sampling the accelerometer of the on-board IMU at a frequency not lower than 1 kHz, and is a time series containing X, Y and Z axis acceleration values;

[0074] S2, the body vibration response feature R vib: Based on the original high-frequency acceleration signal, the body vibration response feature R is obtained through the following calculation process vib :

[0075] S21, calculate the spectral entropy: apply the fast Fourier transform (FFT) to the original high-frequency acceleration signal in the latest time window to obtain its power spectrum; then, normalize the power spectrum, and calculate the spectral entropy value H at the current time according to the Shannon entropy calculation formula spec ; in this embodiment, the duration of the “latest time window” is initially set to 100 milliseconds;

[0076] S22, calculate the reference entropy: maintain a sliding average value based on all spectral entropy values H spec in a long time period in the past, which is defined as the reference spectral entropy and denoted as H base ; in this embodiment, the duration of the “long time period in the past” is initially set to 1 second;

[0077] S23, calculate the entropy increment: subtract the spectral entropy value H spec at the current time from the reference spectral entropy H base to obtain the vibration spectral entropy increment E inc ; E inc characterizes the degree of mutation of the vibration state of the inspection unmanned aerial vehicle relative to its recent stable state;

[0078] S24, normalization processing: apply the minimum-maximum range scaling method to the calculated vibration spectral entropy increment E inc to map it to the interval [0, 1] to obtain the normalized vibration response . The normalized vibration response is the final quantitative value of the body vibration response feature R vib ;

[0079] S3, construct the dynamic state feature vector: combine the contributions of the motor speed command C rpm and the body vibration response feature R vib to the final result using the weighted average method to obtain a single dynamic state feature value F dyn . The dynamic state feature value F dyn is a numerical representation of the dynamic state feature vector V dyn ;

[0080] S31, the weight determination method for the dynamic state feature vector V dyn is as follows: calculate the change rate of the motor speed command C rpm in a short time window to obtain the command change rate factor F rate ; the instruction contribution weight W c is determined by a function of the command change rate factor Frate is determined for the input exponential function, when the instruction changes sharply, the instruction contributes weight W c is close to 1; the response contributes weight W r The value of W c is 1-W

[0081] The calculation method of the kinetic state characteristic value F dyn is expressed as: its value is equal to the instruction contribution weight W c multiplied by the motor speed instruction C rpm , plus the response contribution weight W r multiplied by the normalized vibration response ; in this embodiment, it is ensured that when the UAV is maneuvering, the system pays more attention to the control instruction itself; and when hovering or flying at a constant speed, the system pays more attention to the vibration changes caused by the environment or internal abnormalities;

[0082] The output kinetic state characteristic value F dyn is in a positive proportional relationship with the input parameters, and the physical dimensions are consistent, and they are all dimensionless characteristic values;

[0083] S33, the calculation logic of the vibration spectrum entropy increment E inc : the calculation core comes from Shannon entropy in information theory, and is applied to the signal power spectrum obtained by Fourier analysis; the vibration spectrum entropy increment E inc is obtained by calculating the difference between the current spectrum entropy H spec and the reference spectrum entropy H base , which measures the change amount of the state rather than the absolute amount; a sustained high-intensity but stable vibration, its vibration spectrum entropy increment E inc will tend to zero, because the system has adapted to this state; only when the vibration mode changes abruptly, the vibration spectrum entropy increment E inc will significantly deviate from zero; this makes this parameter extremely sensitive to external disturbances or internal faults;

[0084] S34, the calculation logic of the kinetic state characteristic value F dyn : the calculation method is a typical weighted average method, which is a basic technology in sensor fusion and decision-making theory; the dynamic adjustment of the weight is the key of this scheme; the instruction contribution weight W c and the response contribution weight W r are determined by the instruction change rate factor F rate through a monotonically increasing exponential function, whose trend is: the smaller the instruction change rate factor F rate , the closer the function value to 0; the larger the instruction change rate factor F rate , the closer the function value to 1.

[0085] The calculation of the dynamic state characteristic value is described below through two specific scenarios:

[0086] Scenario one: the inspection UAV is stably hovering, and suddenly encounters a gust of wind;

[0087] The motor speed instruction C rpm is stabilized at 0.4, and the instruction change rate factor F rate is close to 0; according to the exponential function trend, the instruction contribution weight W c is determined to be a low value, and the value in this embodiment is 0.1; the response contribution weight W r is 0.9; the gust of wind causes a sudden change in vibration, and the normalized vibration response is calculated to be a high value, which is 0.8; the value of the dynamic state characteristic value F dyn is 0.1*0.4+0.9*0.8=0.80; although the control instruction does not change, the high value (0.80) of the dynamic state characteristic value F dyn accurately reflects the severe dynamic change caused by the external environment;

[0088] Scenario two: the UAV performs an emergency acceleration climb in a windless environment;

[0089] The motor speed instruction C rpm is sharply increased from 0.4 to 0.9, and the instruction change rate factor F rate is a high value; according to the exponential function trend, the instruction contribution weight W c is 0.9, and the response contribution weight W r is 0.1; the emergency acceleration itself also causes vibration, but its mode is relatively controllable, and the normalized vibration response is calculated to be 0.5; the value of the dynamic state characteristic value F dyn is 0.9*0.9+0.1*0.5=0.86; the high value (0.86) of the dynamic state characteristic value F dyn accurately reflects the severe dynamic change driven by the internal instruction, and the main contribution comes from the instruction itself.

[0090] In the calculation of the value of the dynamic state characteristic value F dyn , the input motor speed instruction C rpm and the normalized vibration response are both normalized, and the weights W c and W r are both positive numbers, so the dynamic state characteristic value F dyn maintains a strict positive proportional relationship with both input parameters, ensuring the monotonicity and predictability of the calculation logic;

[0091] Through normalization processing and weighted average design, the dynamic state characteristic value F dynThe output value range of the output value is limited in the interval [0, 1]; when the dynamic state characteristic value F dyn , the dynamic state of the unmanned aerial vehicle tends to be stable and stationary; the motor speed instruction C rpm , the motor tends to stop or the lowest idle speed; the value of the normalized vibration response , the vibration spectrum entropy increment E inc also tends to 0, that is, the current spectrum entropy value H spec is more and more consistent with the reference spectrum entropy H base , and this state corresponds to the patrol unmanned aerial vehicle being more and more likely to be stationary on the ground or to perform undisturbed and ideal stable hovering in the air.

[0092] When the output of the dynamic state characteristic value F dyn tends to 1, the dynamic event experienced by the unmanned aerial vehicle is more intense; to make the dynamic state characteristic value F dyn tends to 1, the embodiment sets the interval tending to 1 as an interval greater than 0.9; at least one of the following conditions needs to be met:

[0093] Condition 1, the value of the motor speed instruction C rpm tends to 1, corresponding to the maximum throttle instruction, and the instruction change rate factor F rate is high, so that the weight W c tends to 1;

[0094] Condition 2, tends to 1, corresponding to the most severe mutation of the vibration mode, and the instruction change rate factor F rate is small, so that the weight W r tends to 1; this state exactly corresponds to the most extreme maneuvering action performed by the unmanned aerial vehicle or the most intense external physical impact.

[0095] 102, based on the dynamic instruction data and the body micro-vibration data, a disturbance prediction model driven by physical-data hybrid is used to establish a predicted disturbance field for characterizing the environmental effect of the patrol unmanned aerial vehicle;

[0096] In the embodiment, a disturbance prediction model driven by physical-data hybrid is used, which calculates a basic physical disturbance component C base characterizing the airflow disturbance in the ideal fluid state based on the motor speed instruction C rpm contained in the dynamic state characteristic value F dyn ; and calculates a dynamic correction disturbance component C corr characterizing the disturbance deviation caused by non-ideal factors based on the body vibration response feature R vib contained in the dynamic state characteristic value F dyn .; the final basic physical disturbance component C base and the dynamic correction disturbance component C corr are superimposed to generate the predicted disturbance field P pred ; the basic physical disturbance component C base is calculated based on the blade element momentum theory model that equivalently treats the rotor system of the inspection UAV as an actuator disk; the dynamic correction disturbance component C corr is calculated through a pre-trained lightweight neural network with the body vibration response feature R vib as input or a multi-dimensional lookup table.

[0097] The specific process of establishing the predicted disturbance field is shown in Figure 2 , including:

[0098] 201, obtain the dynamic state feature value F dyn at the current synchronization time point t and decompose it into the motor speed instruction C rpm and the body vibration response feature R vib ;

[0099] 202, based on the motor speed instruction C rpm , calculate the basic physical disturbance component C base through a physical model;

[0100] Specifically, the motor speed instruction C rpm is converted through a quadratic function, which represents the square relationship between the motor speed and the rotor thrust, thereby obtaining a total thrust estimation value; the coefficients of the quadratic function are obtained by offline thrust test calibration of the UAV; subsequently, the total thrust estimation value is substituted into the actuator disk model; the actuator disk model grids the specific region of interest ROI below the UAV and calculates an expected downwash air flow velocity vector for each grid element generated by the total thrust; the velocity vector set of all grid elements collectively constitutes the basic physical disturbance component C base ; the basic physical disturbance component C base is a two-dimensional vector field with the physical dimension of meters per second, representing the air flow velocity distribution in the ideal state;

[0101] 203, based on the body vibration response feature R vib , calculate the dynamic correction disturbance component C corr through a data-driven model;

[0102] The scalar value of the body vibration response feature R vib is provided as input to a pre-trained correction model; the correction model is a small multi-layer perceptron MLP or a high-dimensional lookup table; the correction model outputs a dynamic correction disturbance component C baseThe same dimension two-dimensional vector field, that is, the dynamic correction disturbance component C corr Each vector in the vector field represents an additional correction to the airflow velocity of the corresponding grid cell caused by non-ideal factors; the training process of the correction model is to collect real dynamic data and corresponding disturbance field data under various complex environments, and to optimize the model parameters with the goal of minimizing the prediction error. In this embodiment, the non-ideal factors include but are not limited to turbulence;

[0103] 204, vector superposition is performed on the basic physical disturbance component C base and the dynamic correction disturbance component C corr to generate a predicted disturbance field P pred .

[0104] Specifically, for each grid cell in the region of interest ROI, the velocity vector in the basic physical disturbance component C base is added to the corresponding velocity vector in the dynamic correction disturbance component C corr ; the result set after vector addition of all grid cells constitutes the final predicted disturbance field P pred ; the predicted disturbance field P pred is a two-dimensional vector field with a physical dimension of meters per second; this step is performed independently for the visual and thermal modalities to generate a predicted visual disturbance field representing the expected displacement of the visual medium and a predicted thermal disturbance field representing the expected displacement of the thermal medium; since it is a vector addition, the enhancement of the input component will directly lead to the enhancement of the output field intensity, and the two are strictly proportional.

[0105] A pure physical model is difficult to accurately describe complex realities such as turbulence and ground effect; a pure data model lacks physical interpretability and has limited generalization ability; this scheme combines the two, using a physical model based on the blade momentum theory to provide a stable and interpretable basic physical disturbance component, and then using a data-driven model to learn and compensate for non-ideal factors that the physical model cannot cover to form a dynamic correction disturbance component; this structure makes the prediction results both physically reasonable and adaptable to complex realities, with much higher prediction accuracy and robustness than a single model;

[0106] The calculation logic of the basic physical disturbance component C base : the core of the calculation is derived from the actuator disk theory in fluid mechanics and the blade momentum theory in aerodynamics; a physical relationship between the thrust of the rotor and the motor speed and airflow velocity is established; the calculation of the basic physical disturbance component C base is a monotonically increasing function;

[0107] The calculation logic of the dynamic correction disturbance component C corrComputational logic: Core rooted in supervised learning and function approximation theory in machine learning; learn input body vibration response feature R vib through training a neural network to learn the complex non-linear relationship between input and output; this embodiment sets "output" as "actual disturbance vs. residual predicted by physical model"; body vibration response feature R vib increases, meaning the extent of UAV vibration state deviating from stable baseline increases; in the trained model, this corresponds to a stronger dynamic correction disturbance component C corr to compensate for the disturbance not captured by the physical model due to factors such as turbulence; therefore, the strength of dynamic correction disturbance component C corr is also positively correlated with body vibration response feature R vib ;

[0108] Predicted disturbance field P pred Computational logic: Its calculation method is vector addition in mathematics. Predicted disturbance field P pred is the direct vector sum of basic physical disturbance component C base and dynamic correction disturbance component C corr ; since the strength of basic physical disturbance component C base and dynamic correction disturbance component C corr are both positively correlated with their respective driving inputs (C rpm and R vib ), the field strength of final predicted disturbance field P pred is also positively correlated with the increase of motor speed command C rpm and body vibration response feature R vib .

[0109] To verify the technical effect of this step, six representative flight conditions are set for testing, and the core output of interest is the average field strength of predicted disturbance field P pred to quantify the overall disturbance intensity of prediction, as shown in Table 1 below.

[0110] ;

[0111] According to Table 1, compare condition 2 (ideal hovering) and condition 3 (hovering encountering gust), input motor speed command C rpm is unchanged (0.4), so the field strength of basic physical disturbance component C base remains at 1.6 m / s; however, due to the gust, body vibration response feature R vib increases from 0.05 to 0.8, and data-driven dynamic correction disturbance component C corr field strength also increases from 0.1 m / s to 1.5 m / s, ultimately making predicted disturbance field P predThe field strength increased significantly from 1.7 m / s² to 3.1 m / s². This demonstrates that it can effectively distinguish and quantify the effects caused by control commands and external disturbances.

[0112] Comparing operating condition 2 (ideal hovering) and operating condition 6 (near ground hovering), the motor speed command C rpm Slightly increased to counteract ground effect, but the body vibration response characteristic R vib Significantly increased due to airflow-ground interaction (from 0.05 to 0.65); Dynamically corrected disturbance component C corr The significant increase (from 0.1 to 1.25) accurately captures the ground effects that are difficult for this physical model to describe, making the final predicted perturbation field P pred It can more realistically reflect the complex airflow near the ground;

[0113] From operational condition 1 (ground stationary) to operational condition 5 (rapid acceleration maneuver), as the flight intensity increases, the motor speed command C... rpm and the vibration response characteristics of the body R vib The overall trend is upward, and the final predicted perturbation field P pred The average field strength also increased monotonically from 0 m / s to 9 m / s, verifying the correctness and predictability of the calculation logic.

[0114] It should be noted that, in order to accommodate the limited precision of digital processing and improve computational efficiency, the calculated predicted perturbation field P is... pred The magnitude of each vector is compared with a preset zero velocity threshold of 0.01 m / s. If the vector magnitude is less than the zero velocity threshold, the value of that vector is forcibly set to zero in subsequent comparison calculations. This processing method is a common technique used by those skilled in the art when implementing numerical algorithms. It aims to effectively process parameters that theoretically approach the target value infinitely in engineering, thereby ignoring minimum values ​​that have no practical impact on the final result.

[0115] 103. Based on environmental perception image data and predicted disturbance field, the observed disturbance field is extracted. The observed disturbance field is used to characterize the physical disturbance caused by the inspection drone in the surrounding medium.

[0116] In this embodiment, based on the current environmental perception image data at the current synchronization time point and the historical environmental perception image data at the previous synchronization time point, the initial observed optical flow field O containing all pixel motion information is calculated. init Based on the predicted perturbation field P pred For the initial observed optical flow field O init By suppressing and predicting the perturbation field P pred By analyzing optical flow vectors that are inconsistent in their motion patterns, and refining the physical disturbances actually caused by the inspection drone, a filtered result is obtained. This filtered result is then used as the observed disturbance field O.obs The observed disturbance field includes an observed visual disturbance field representing the motion of the real visual medium and an observed thermal disturbance field representing the motion of the real thermal medium;

[0117] calculating an initial observed optical flow field O init Specifically, a Farnebäck dense optical flow estimation algorithm is applied to the visible light image frame sequence and the thermal imaging image frame sequence respectively; the filtering processing is specifically calculating an initial observed optical flow field O init and the predicted disturbance field P pred The direction similarity of the corresponding vector at each pixel position is calculated, and the initial optical flow vector is amplitude weighted based on the similarity.

[0118] The specific calculation process of the observed disturbance field is shown in Figure 3 , which includes:

[0119] 301, obtaining the predicted disturbance field P pred and the environmental perception image data;

[0120] The environmental perception image data includes visible light image frames and thermal imaging image frames at the current synchronization time point t and the previous synchronization time point t-1; and the visible light camera and the thermal imaging camera on board are used to synchronously collect the visible light image frames and the thermal imaging image frames at a preset frame rate.

[0121] 302, calculating an initial observed optical flow field through the visible light channel and the thermal imaging channel;

[0122] The visible light image frames at the current synchronization time point t and the previous synchronization time point t-1 are taken as input, and a dense optical flow estimation algorithm is applied in the same region of interest ROI as the predicted disturbance field P pred ;

[0123] The dense optical flow estimation algorithm calculates a two-dimensional displacement vector for each pixel in the ROI, which represents the visual motion of the pixel from the previous synchronization time point t-1 to the current synchronization time point t; the displacement vector set of all pixels collectively constitutes the initial observed optical flow field O init of the visible light channel. The initial observed optical flow field O init captures all the motions in the field of view, including the motion caused by the downwash of the inspection UAV and other irrelevant motions in the background.

[0124] 303, separating the real disturbance caused by the inspection UAV from the initial observed optical flow field to calculate a disturbance similarity weight W sim ;

[0125] The calculation method is as follows:

[0126] Vector direction comparison: for each pixel position in the ROI, the corresponding vector in the initial observed optical flow field Oinit the displacement vector in the initial observed optical flow field O pred and the prediction vector in the predicted disturbance field P

[0127] Similarity calculation: the directional consistency is quantified by calculating the dot product between the displacement vector and the unit vector of the prediction vector (i.e. cosine similarity); the value range of the quantified result of directional consistency is [-1, 1], where 1 means the directions are exactly the same, -1 means the directions are exactly opposite, and 0 means the directions are perpendicular;

[0128] Weight normalization: the quantified result of directional consistency of the dot product is mapped to the interval [0, 1] through a linear transformation function to obtain the disturbance similarity weight W sim of the pixel position; this weight is a scalar value, and a weight of 1 means that the observed motion is completely consistent with the predicted disturbance pattern, and a weight of 0 means that it is completely inconsistent.

[0129] 304, the initial observed optical flow field O sim is weighted filtered using the disturbance similarity weight W init to obtain the observed disturbance field O obs .

[0130] For each pixel position within the ROI, the displacement vector in the initial observed optical flow field O init is multiplied by the disturbance similarity weight W sim scalar value corresponding to the position; the new displacement vector set of all pixel positions after weighted calculation constitutes the final observed disturbance field O obs .

[0131] It is ensured that only the motion components consistent with the prediction of the unmanned aerial vehicle dynamics are retained or enhanced, while the background motion unrelated to the prediction (e.g. driving of a vehicle far away, blowing of a distant tree by the wind) is effectively suppressed because its direction is inconsistent with the prediction, and its corresponding weight tends to 0; since the disturbance similarity weight W sim is non-negative, this weighting process ensures that the vector components of the input initial observed optical flow field O init and the corresponding components of the output observed disturbance field O obs are in a positive proportional relationship.

[0132] The output at each synchronization time point t is a data structure, the observed disturbance field O obs contains the observed visual disturbance field representing the real visual medium motion and the observed thermal disturbance field representing the real thermal medium motion after purification.

[0133] Traditional dense optical flow algorithm will indiscriminately calculate the movement of all objects in the field of view, which leads to a large amount of "noise" in the output irrelevant to the disturbance of the unmanned aerial vehicle itself, such as moving vehicles in the background, swaying trees in the distance, etc.; these "noise" will seriously interfere with the judgment of the subsequent steps;

[0134] The innovation of step S103 of the present application is that the original optical flow result is not directly used, but the predicted disturbance field P pred As prior knowledge, the original optical flow is filtered once "target-oriented"; only the observed motion consistent with the prediction in the motion pattern will be retained and enhanced; this method transforms a general visual measurement tool (optical flow method) into a high signal-to-noise ratio measurement tool dedicated to the target of the present application, which is much purer and more relevant than the traditional method.

[0135] For the calculation logic of the initial observed optical flow field O init The core comes from the optical flow estimation algorithm in the field of computer vision, such as Farnebäck algorithm, which estimates the motion by analyzing the spatio-temporal changes of pixel intensity in image sequences;

[0136] For the calculation logic of the disturbance similarity weight W sim The core comes from the vector dot product in linear algebra and the cosine similarity derived therefrom; cosine similarity is a classic method for measuring the similarity of the directions of two vectors; the disturbance similarity weight W sim is the core of the innovation; through a monotonically increasing linear transformation function, the value range of cosine similarity [-1, 1] is mapped to the value range of disturbance similarity weight [0, 1]; the following scenarios are set:

[0137] Scenario (one), the observed motion is completely consistent with the prediction;

[0138] At the preselected pixel points, the initial observed optical flow field O init represented by the observed motion vector is completely the same as the predicted disturbance field P pred represented by the predicted vector; the cosine similarity is 1.0; through the transformation function (1.0 plus 1 divided by 2), the calculated disturbance similarity weight W sim is 1.0; indicating that the observed motion is highly related to the prediction and should be completely retained;

[0139] Scenario (two), the observed motion is completely perpendicular to the prediction;

[0140] At the pixel points, the initial observed optical flow field O init represented by the observed motion vector is completely perpendicular to the predicted disturbance field P predThe orientation is perpendicular; the cosine similarity is 0; the perturbation similarity weight W is calculated using a transformation function (adding 1 to 0.0 and then dividing by 2). sim A value of 0.5 indicates a certain deviation between the observed motion and the prediction, thus reducing its reliability.

[0141] Scenario (3): The observed motion is completely opposite to the prediction;

[0142] At the pre-selected pixel, the observed motion vector represents the initial observed optical flow field O. init The predicted perturbation field P with the predicted vector pred The directions are completely opposite; the cosine similarity is -1.0; the perturbation similarity weight W is calculated using a transformation function (adding 1 to -1.0 and then dividing by 2). sim A value of 0.0 indicates that the observed motion is completely unrelated to the prediction and should be completely filtered out.

[0143] For the observed perturbation field O obs The computational logic is as follows: the calculation method is the scalar multiplication of a scalar and a vector in mathematics; the observed perturbation field O... obs The initial observed optical flow field O init After perturbation of similarity weight W sim The weighted result; due to the perturbation of the similarity weight W sim The value range is [0, 1]. This operation essentially scales the original optical flow vector based on the correlation of the motion direction; the perturbation similarity weight W sim The larger the value, the greater the initial observed optical flow field O. init The direction of motion and the predicted disturbance field P pred The more consistent the observations, the better the final observed perturbation field O. obs The more completely the amplitude is preserved, the better; it is a filtering process that ensures a positive proportional relationship.

[0144] To verify the technical effectiveness of this step, six flight conditions with different interferences were set up for testing. The core parameter of interest was the vector magnitude at a specific point in the optical flow field to demonstrate the filtering effect, as shown in Table 2 below.

[0145] ;

[0146] Target signal fidelity: According to Table 2, in case 1 (target point - pure disturbance), when the observation and prediction are completely consistent, the disturbance similarity weight W sim The final output is the observed perturbation field O, which is set to 1. obs The modulus (2.5) and the initial observed optical flow field O init They are completely identical, proving that the true target signal can be preserved without loss.

[0147] Effective suppression of noise: In Table 2, working condition 4 (background point-irrelevant motion) and working condition 5 (background point-reverse motion) simulate the irrelevant motion existing in the background; due to the inconsistency or even opposition of the motion direction to the predicted downward airflow direction (downward), the calculated disturbance similarity weight W sim is 0.5 and 0, respectively; this causes the initial motion length of 3.0 to be significantly weakened to 1.5 or even completely eliminated to 0; it is proved that the background noise irrelevant to the UAV dynamics can be effectively filtered out;

[0148] Decomposition ability for mixed motion: In Table 2, working condition 2 (target point-partial disturbance) and working condition 3 (target point-strong crosswind) simulate the case where the real disturbance is mixed with disturbances such as crosswind; through the direction similarity calculation, the weight of these mixed motions is reasonably reduced, so that the final output observation disturbance field O obs has a length (2.13 and 1.25) lower than the initial value (2.5), which corresponds to the physical extraction of the component consistent with the predicted direction in the initial motion, achieving signal purification.

[0149] It should be noted that, in order to adapt to the limited precision of digital processing and improve the calculation efficiency, the calculated initial observation optical flow field O init The length of each vector in the field is compared with a preset small motion threshold (0.1 pixels / frame in this example); if the vector length is less than the small motion threshold, the weight of the vector in the subsequent disturbance similarity weight W sim is forcibly set to 0 in the calculation. This processing method is a conventional technical means for those skilled in the art when implementing numerical algorithms, aiming to effectively process the parameter theoretically approaching the target value in engineering, so as to ignore the extremely small value caused by image noise and other factors, which has no actual effect on the final result.

[0150] 104, the predicted disturbance field and the observation disturbance field are structurally compared, and the prediction-observation consistency index is calculated;

[0151] In this embodiment, for the visible light channel and the thermal imaging channel, respectively, the prediction disturbance field and the observation disturbance field are compared in multiple dimensions to generate a visual consistency component and a thermal consistency component; then the thermal channel weight and the visual channel weight are determined; finally, the visual consistency component and the thermal consistency component are weighted and fused to obtain the prediction-observation consistency index; the value range of the prediction-observation consistency index is [0, 1]; when the prediction-observation consistency index output tends to 0, it indicates that the overall consistency degree of the prediction disturbance field and the observation disturbance field is lower; when the prediction-observation consistency index output tends to 1, it indicates that the overall consistency degree of the prediction disturbance field and the observation disturbance field is higher.

[0152] For example, Figure 4As shown, the specific calculation process of the prediction-observation consistency index includes:

[0153] 401, prepare the predicted disturbance field P pred , the observed disturbance field O obs , and the thermal imaging image frame I th ;

[0154] For both visible light and thermal imaging channels, the consistency component is calculated independently; this embodiment uses subscript index "vis" to represent the visible light channel, and subscript index "th" to represent the thermal imaging channel.

[0155] 402, calculate the visual amplitude similarity and visual structure similarity of the predicted disturbance field and the observed disturbance field through the visible light channel;

[0156] Visual amplitude similarity S mag，vis is used to quantify the degree of agreement between the predicted disturbance field and the observed disturbance field in disturbance intensity; the vector module length of each pixel position in the predicted visual field and the observed visual field is extracted respectively to form two module length matrices; then, the normalized mean absolute error between the two matrices is calculated, and the error value is subtracted by 1; its value range is [0, 1], and 1 indicates complete amplitude consistency;

[0157] Visual structure similarity S str，vis is used to quantify the degree of agreement between the predicted disturbance field and the observed disturbance field in disturbance mode, texture and spatial distribution; the predicted visual field and the observed visual field are taken as input, and the structural similarity index SSIM algorithm is applied for calculation; its value range is [0, 1], and 1 indicates complete structural consistency;

[0158] 403, calculate the visual amplitude similarity and the visual structure similarity by weighted average method to obtain the visual consistency component;

[0159] Visual consistency component C vis combines the visual amplitude similarity S mag，vis and the visual structure similarity S str，vis by weighted average method; the visual consistency component C vis is equal to the amplitude similarity multiplied by its preset weight, plus the structure similarity multiplied by its preset weight; the preset weight here is determined according to the specific needs of the application of the present technology; if more attention is paid to the accuracy of the disturbance mode, the weight of the structure similarity is set to be higher than that of the amplitude similarity; the sum of the two weights is 1; this step ensures that there is a proportional relationship between the input amplitude similarity, structure similarity and the output visual consistency component; this embodiment sets the weight of the structure similarity to 0.7 and the weight of the amplitude similarity to 0.3.

[0160] 404, calculating the thermal amplitude similarity and the thermal structure similarity between the predicted disturbance field and the observed disturbance field through the thermal imaging channel;

[0161] 405, calculating the thermal amplitude similarity and the thermal structure similarity through the weighted average method to obtain the thermal consistency component;

[0162] The thermal consistency component C th is calculated in the same way as the visual consistency component C vis in steps 402 and 403.

[0163] 406, calculating the thermal information entropy according to the thermal imaging image frame;

[0164] The thermal information entropy H th is calculated as follows: the parameter is used to quantify the information amount of the current thermal imaging image; the higher the entropy value, the richer the details and the stronger the contrast of the thermal image, and the higher the reliability of the verification basis; the calculation method is to extract the pixels of the region of interest ROI in the thermal imaging image frame I th , count the gray histogram, and calculate the Shannon entropy based on the histogram. The thermal information entropy H th is obtained by calculating the entropy in information theory on the current collected original thermal imaging image.

[0165] 407, mapping the thermal information entropy in different ranges to a unified weight interval to generate a thermal channel weight;

[0166] The calculation method is to map the thermal information entropy H th in different ranges to a unified weight interval by applying a Sigmoid function; the Sigmoid function takes the thermal information entropy H th as input and outputs a thermal channel weight W th in the interval [0, 1]; the Sigmoid function has two adjustable parameters, center point and gain, which are used to set the range in which the entropy value starts to significantly affect the weight and the degree of weight change.

[0167] 408, determining the visual channel weight;

[0168] The calculation method is that the visual channel weight W vis and the thermal channel weight W th are weight complementary, and the calculation formula is: W vis =1-W th ;

[0169] 409, calculating the predicted-observed consistency index according to the thermal channel weight, the visual channel weight, the visual consistency component, and the thermal consistency component.

[0170] The predicted-observed consistency index M consist is equal to the visual consistency component Cvis multiplied by the visual channel weight W vis plus the thermal consistency component C th multiplied by the thermal channel weight W th ; since all input values are in the range [0, 1], the final prediction-observation consistency index is also in the range [0, 1], and is positively proportional to each input component. The physical dimension of prediction-observation consistency is dimensionless, and its value directly reflects the overall consistency between prediction and observation.

[0171] The dynamic calculation logic of thermal channel weight and visual channel weight: the core comes from Shannon entropy in information theory and Sigmoid function in machine learning; entropy is used to quantify information, and Sigmoid function is used to smoothly map a variable in real number domain to the interval [0, 1]. The value range of thermal channel weight W th is [0, 1];

[0172] When thermal information entropy H th increases due to clear thermal imaging pictures and rich details, the output thermal channel weight W th of Sigmoid function tends to 1; this makes the visual channel weight W vis tend to 0;

[0173] When thermal information entropy H th decreases due to blurred thermal imaging pictures and lack of thermal features, the output thermal channel weight W th of Sigmoid function tends to 0; this makes the visual channel weight W vis tend to 1.

[0174] The value range of prediction-observation consistency index M consist is [0, 1]; when the output of prediction-observation consistency index M consist tends to 0, it indicates that the overall consistency between prediction and observation is lower; the values of visual consistency component C vis and thermal consistency component C th themselves are low (i.e. there is a huge difference between prediction and observation in both visible light and thermal field), or the consistency component of a channel is high but its channel weight tends to 0 due to unreliable information; for example, even if the thermal consistency component C th is 0.9, but if the thermal imaging picture is blurred, resulting in low thermal information entropy H th , the thermal channel weight W th tends to 0, and its contribution to the final result also tends to 0; it is proved that this algorithm can avoid being misled by high scores of unreliable channels.

[0175] When the prediction-observation consistency index M consistThe closer the output is to 1, the higher the consistency between the prediction and the observation; this requires that the consistency component of at least one channel is close to 1 and the weight of the channel itself is also high. For example, in a scene with obvious water vapor, the thermodynamic information entropy H th is high, resulting in the thermodynamic channel weight W th tending to 1; at this time, as long as the thermodynamic consistency component C th tends to 1, even if the visual consistency component C vis is low, the final result of the prediction-observation consistency index M consist will also tend to 1; it is demonstrated that the algorithm can grasp the main contradiction and let the most reliable evidence dominate the final judgment.

[0176] To verify the adaptive fusion effect of the step, six typical scenes with different environmental characteristics and consistency performances are set, and the test data are shown in Table 3.

[0177] ;

[0178] Environmental adaptability: in Table 3, the scene 1 (daytime-no heat source) and the scene 3 (daytime-strong heat source) are compared; in the scene 1, the thermodynamic information entropy H th is very low, and the automatically calculated thermodynamic channel weight W th is 0.12, and the prediction-observation consistency index M consist is 0.884, which is mainly contributed by the visual consistency component C vis of 0.95; in the scene 3, the thermodynamic information entropy H th is very high, which in turn gives the thermodynamic channel a high thermodynamic channel weight W th of 0.88, and at this time, the prediction-observation consistency index M consist is 0.852, which is mainly contributed by the high thermodynamic consistency component C th of 0.90, even if the visual consistency component C vis is low at this time; this perfectly embodies the core idea of the algorithm of the application that “reliable information sources are trusted”;

[0179] Scientificity of information fusion: in the scene 2 (daytime-weak heat source), both channel information is relatively reliable, and an equal weight (0.50 each) is given, and the prediction-observation consistency index M consist is 0.875, which is a fair fusion of the consistency components of the two;

[0180] Judgment under adverse conditions: the scene 4 (night-no heat source) and the scene 5 (night-strong heat source) are compared; in the night, the visible light information quality decreases, resulting in the visual consistency component C vis being generally low; in the scene 4, both channels are unreliable, and the prediction-observation consistency index M consist0.305 is also correspondingly low, correctly reflecting the high uncertainty; but in scenario 5, although the visible channel fails, the visual consistency component C vis 0.20, but with the high-weight thermal channel (thermal channel weight W th 0.91) and high-score thermal consistency (thermal consistency component C th 0.92), still obtains a high-score prediction-observation consistency index M consist 0.855, achieving robust judgment in the case of partial sensor failure;

[0181] Identification of full mismatch: in scenario 6 (full mismatch), the consistency components of the two channels are extremely low regardless of the environment, and the prediction-observation consistency index M consist is also low, which proves that the algorithm will not fabricate a high score by weight allocation, and its foundation is still the respect for true consistency.

[0182] It should be noted that in order to adapt to the limited precision of digital processing and improve the calculation efficiency, the calculated thermal channel weight is compared with a preset weight truncation threshold 0.01; if the thermal channel weight is less than the weight truncation threshold, the value of the thermal channel weight is forcibly set to 0 in the subsequent weighted fusion, and the corresponding visual channel weight is forcibly set to 1. This processing method is a common technical means for those skilled in the art when implementing numerical algorithms, which aims to effectively process the parameters that theoretically tend to approach the target value in engineering, so as to ignore the extremely small values that have no actual impact on the final result and simplify the calculation.

[0183] 105, taking the prediction-observation consistency index as a real-time measure of the reliability of visual data, dynamically adjusting the state information confidence of the visual aided positioning algorithm according to the real-time measure, and finally obtaining an optimized positioning result.

[0184] Specifically, the specific process of dynamic positioning optimization based on the prediction-observation consistency index is as shown in Figure 5 , which includes:

[0185] 501, obtaining a noise covariance matrix related to visual measurement;

[0186] The noise covariance matrix R vis，base related to visual measurement is determined in the system initialization or offline calibration stage, representing the noise level of visual measurement under ideal working conditions, and is a preset fixed matrix.

[0187] 502, obtaining a prediction-observation consistency index;

[0188] 503, by using a preset nonlinear mapping function, the prediction-observation consistency index is used as a real-time measure of the reliability of visual data and converted into visual measurement confidence.

[0189] The preset nonlinear mapping function is a logistic function (Sigmoid function) with adjustable parameters, and the prediction-observation consistency index M is used. consist As a real-time measure of the reliability of visual data, it is converted into visual measurement confidence level C. vis，meas ;

[0190] The logistic function contains two core preset parameters:

[0191] Transformation center point x0: This parameter determines the prediction-observation consistency index M. consist For what value is the confidence level C of the visual measurement? vis，meas The value is 0.5; specifically, it is set according to the system's safety margin. In this embodiment, it is set to 0.6, meaning that only when the prediction-observation consistency index M... consist The system only begins to "trust" visual data when the value is above 0.6;

[0192] kurtosis factor k: This parameter determines the confidence level C of the visual measurement. vis，meas The degree of change from 0 to 1; a larger kurtosis factor k value means a sharp "trust-distrust" switch, suitable for scenarios with high security requirements; a smaller kurtosis factor k value provides a smoother transition;

[0193] The conversion center point x0 and the steepness factor k are both preset according to the specific application scenario and risk assessment.

[0194] 504. In the state estimation fusion process of the visual-assisted localization algorithm, the confidence level of visual measurement is used as a dynamic weighting factor.

[0195] The confidence level C of visual measurement vis，meas This is converted into a dynamic weighting factor that can directly affect the noise covariance matrix, denoted as α. uncert ;α uncert It is a scalar used to amplify or reduce the nominal uncertainty of visual measurements; its core logic is: the confidence level C of a visual measurement. vis，meas The lower the value, the greater the introduced uncertainty; dynamic weighting factor α uncert Equal to the confidence level of visual measurement C vis，meas The reciprocal of the sum of a preset positive constant; the preset positive constant is used to measure the confidence level C in visual measurements. vis，meas Theoretically, when the value approaches 0, it avoids the error of dividing by zero in the calculation and ensures the numerical stability of the algorithm; dynamic weighting factor α uncert With visual measurement confidence level C vis，measInversely proportional relationship; the preset normal number is initially set to 1e -6 .

[0196] 505, adjusting the noise covariance matrix related to visual measurement by a dynamic weight factor;

[0197] The real-time adjusted visual measurement noise covariance matrix is equal to the product of the dynamic weight factor a uncert and the initial noise covariance matrix R vis，base , and is denoted as R vis，adj ; the real-time adjusted noise covariance matrix R vis，adj directly participates in the state update step based on the extended Kalman filter EKF, and is used to calculate the Kalman gain;

[0198] When the prediction-observation consistency index M consist is low, the visual measurement confidence C vis，meas is reduced, resulting in an increase in the dynamic weight factor a uncert , and the real-time adjusted noise covariance matrix R vis，adj of the visual measurement is also increased accordingly; an increased noise covariance matrix R vis，adj will directly result in a decrease in the calculated Kalman gain, so that the weight of the state information confidence in the state fusion process when updating the system state is reduced, and the visual effect is effectively suppressed. This process ensures that all inputs and the final positioning effect are logically consistent and proportional or inversely proportional, and the physical quantities are consistent.

[0199] Specifically, the confidence conversion scenario under high safety requirements is:

[0200] The conversion center point x0=0.7 and the steepness factor k=20 are set; this means that the system has high requirements for visual quality, and only when the "prediction-observation consistency index M consist " is significantly higher than 0.7, a high confidence is given, and the transition process of the trust degree is very rapid; in this embodiment, the "prediction-observation consistency index M consist " is set to decrease from 0.75 to 0.65 at a certain time;

[0201] When M consist =0.75, the visual measurement confidence C vis，meas is calculated to be 0.73; when M consist =0.65, the visual measurement confidence C vis，meas is calculated to be 0.27; from the results, it can be seen that M consist only decreases by 0.1, but the visual measurement confidence C vis，meas suddenly decreases from 0.73 to 0.27, which embodies the "rapid switching" feature; and ensures the proportional relationship between the input and the output.

[0202] For dynamic weight factor α uncert Calculation logic: derived from inverse relationship in basic mathematics, combined with the idea of "gain is inversely proportional to confidence" in control theory; the value range of dynamic weight factor α uncert is (1, +∞);

[0203] When visual measurement confidence C vis，meas output tends to 0, the trend of dynamic weight factor α uncert is to tend to positive infinity; visual measurement confidence C vis，meas tends to 0, which means that the trust degree of visual measurement drops to the freezing point; according to the inverse relationship, dynamic weight factor α uncert will become large; which will lead to R vis，adj also becomes large, which in Kalman filtering is equivalent to declaring that visual measurement is completely unreliable, that is, the state noise is infinite, and its weight in state update will tend to 0; it is proved that this algorithm can effectively isolate unreliable visual data.

[0204] When visual measurement confidence C vis，meas output tends to 1, the trend of dynamic weight factor α uncert is to tend to 1; visual measurement confidence C vis，meas tends to 1, which means that the trust degree of visual measurement reaches the peak; at this time, dynamic weight factor α uncert tends to 1, R vis，adj will tend to the initial noise covariance matrix R vis，base ; which means that the current visual measurement quality is good, and it should be used according to its best performance; it is proved that this algorithm can fully play the performance of VIO in high-quality data; it should be noted that this embodiment sets the interval close to 0 to the range of positive numbers less than 0.6, and the interval close to 1 to the range greater than 0.92, and the upper and lower limits of the two intervals are limited by the value range of visual measurement confidence C vis，meas .

[0205] To verify the effect of dynamic intervention of this step on VIO system, six scenes representing different visual information quality are set, and the adjustment effect on measurement noise is observed; the diagonal elements of the initial noise covariance matrix R vis，base in position uncertainty are set to 0.01. As shown in Table 4 below.

[0206] ;

[0207] Smooth intervention curve: according to the content in Table 4, from scene A (ideal environment) to scene E (close to failure), with the smooth decrease of prediction-observation consistency index M consist , visual measurement confidence C vis，measAlso smooth down, and then lead to dynamic weight factor α uncert and the adjusted noise covariance matrix R vis，adj The noise item in the matrix is smoothly and nonlinearly increased; Avoids the occurrence of system state mutation, ensures the continuity and smoothness of the positioning trajectory;

[0208] Significant adjustment range: in scenario A (ideal environment), the adjusted noise item (0.0102) is basically consistent with the reference value (0.01), and the system fully trusts the visual data; While in scenario E (visual near failure), the noise item is enlarged by 50 times, and the visual weight is effectively suppressed to a low level; Proved to have a large enough dynamic adjustment range, which can cover all working conditions from optimal to worst;

[0209] Effective isolation of failure: in scenario F (complete failure), when the prediction-observation consistency index M consist The dynamic weight factor α uncert will increase sharply, making the visual weight tend to zero; Let the system rely on IMU for track calculation, so as to realize the complete isolation of catastrophic visual data and ensure the final safety of the system.

[0210] It should be noted that in order to adapt to the limited precision of digital processing and improve the calculation efficiency, the calculated visual measurement confidence is compared with a preset confidence cutoff threshold 0.01; If the visual measurement confidence is less than the confidence cutoff threshold, the value of the visual measurement confidence will be forcibly set to 0 in the subsequent step of calculating the uncertainty scaling factor (a small normal number is used when calculating the reciprocal to protect it). This processing method is a common technical means for those skilled in the art when implementing numerical algorithms, which aims to effectively process the parameter that theoretically approaches the target value infinitely in engineering, so as to ignore the small value that has no actual impact on the final result, and ensure the complete isolation of failure data in function. Explanation of the concept represented by "state information contribution degree": In traditional, non-adaptive visual aided positioning algorithms (such as standard VIO), the system's trust degree of each sensor is set once in the design and calibration stage, and remains unchanged throughout the running process. This "trust degree" is specifically manifested in the algorithm level as the noise covariance matrix of the sensor. The numerical value of this matrix reflects the engineer's prior judgment of the measurement accuracy of the sensor under "general" or "ideal" working conditions.

[0211] Static trust model cannot cope with the dynamic changes of sensor performance in actual operation. For example, when the UAV enters an area with dim light or full motion blur, the actual measurement quality of the visual sensor will drop sharply, but the traditional algorithm still processes the visual data that has actually become unreliable according to the "high trust" (i.e. a smaller noise covariance) set at the factory; which will lead to wrong decisions.

[0212] The "state information contribution degree" is a dynamic, real-time and quantitative indicator, which represents the degree to which the measurement information provided by the visual sensor is allowed to be used to correct and update the final state estimation of the system at the current moment, including but not limited to position, attitude and speed. The "state information contribution degree" is not a directly set parameter, but a description of the final effect. The present application does not directly set a variable named "contribution degree", but through a series of steps (M consist →C vis，meas →α uncert →Adjust R vis，adj →Impact Kalman gain), the effective control of the "state information contribution degree" is finally realized. The "state information contribution degree" represents the dynamic change and real-time "speaking right" or "influence" of the visual sensor in positioning decision-making.

[0213] The beneficial effects achieved by the embodiments of the present application are:

[0214] I. By using a physical-data hybrid driven disturbance prediction model, the physical disturbance field generated by the washing airflow of the inspection UAV in response to the surrounding environmental medium is actively predicted based on the body micro-vibration data of the inspection UAV, and the predicted disturbance field is a theoretical expectation of the physical reality.

[0215] II. The observed disturbance field is extracted from the environmental perception image data, and the observed disturbance field caused by the actual UAV is accurately observed and purified from the real image full of interference. The observed disturbance field is a direct evidence from the physical reality.

[0216] III. By performing multi-dimensional structured comparison on the predicted disturbance field and the observed disturbance field, the prediction-observation consistency index is obtained, which directly quantifies the self-consistency degree of the world perceived by the inspection UAV and its physical interaction.

[0217] In summary, when the inspection unmanned aerial vehicle is in a false environment that cannot produce the expected physical interaction, the observation disturbance field will have a large mismatch with the predicted disturbance field, causing the prediction-observation consistency index to drop sharply. By converting the prediction-observation consistency index into a visual measurement confidence, the state information contribution of visual data in the positioning algorithm is adjusted in real time and dynamically. When the visual measurement confidence is low, the weight of visual measurement in the state fusion process of state information confidence calculation is automatically reduced, thereby effectively resisting visual deception attacks and ensuring the positioning reliability in high security risk scenarios.

[0218] In combination with the positioning optimization method based on the visual auxiliary positioning algorithm described in the above embodiments, the positioning optimization system based on the visual auxiliary positioning algorithm is described below by way of embodiments.

[0219] As shown in Figure 6 , the embodiment of the present application provides a positioning optimization system based on a visual auxiliary positioning algorithm, comprising:

[0220] The data acquisition module 601 is configured to acquire the dynamic instruction data of the inspection unmanned aerial vehicle, and synchronously collect the body micro-vibration data and the environment perception image data.

[0221] The predicted disturbance field establishment module 602 is configured to establish a predicted disturbance field for characterizing the environmental effect of the inspection unmanned aerial vehicle based on the dynamic instruction data and the body micro-vibration data, and through a physical-data hybrid driven disturbance prediction model.

[0222] The observation disturbance field establishment module 603 is configured to extract an observation disturbance field based on the environment perception image data and the predicted disturbance field, and the observation disturbance field is used to characterize the physical disturbance caused by the inspection unmanned aerial vehicle in the surrounding medium.

[0223] The prediction-observation consistency index calculation module 604 is configured to perform a structured comparison between the predicted disturbance field and the observation disturbance field, and calculate a prediction-observation consistency index.

[0224] The optimized positioning module 605 is configured to take the prediction-observation consistency index as a real-time measurement of the reliability of visual data, dynamically adjust the state information confidence of the visual auxiliary positioning algorithm according to the real-time measurement, and finally obtain an optimized positioning result.

[0225] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0226] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0227] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0228] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0229] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for positioning optimization based on a vision-aided positioning algorithm, characterized in that, The method is applied to a patrol unmanned aerial vehicle and comprises the following steps: Obtaining dynamic instruction data of the patrol unmanned aerial vehicle, constructing a dynamic state feature vector, the dynamic state feature vector comprising a motor speed instruction and a body vibration response feature, the motor speed instruction being synchronized with a hardware timestamp, and the body vibration response feature being related to body micro-vibration data; Synchronously collecting the body micro-vibration data, a visible light image frame and a thermal imaging image frame having the same hardware timestamp as the dynamic state feature vector; Forming environment perception image data according to the visible light image frame and the thermal imaging image frame; Combining the motor speed instruction and the body vibration response feature contribution to a final result, using a weighted average method for combination to obtain a dynamic state feature value, the dynamic state feature value being a numerical representation of the dynamic state feature vector; Limiting an output value range of the dynamic state feature value to an interval of [0, 1]; when the dynamic state feature value output tends to be 0, the dynamic state of the patrol unmanned aerial vehicle tends to be stable and stationary; when the dynamic state feature value output tends to be 1, the dynamic event intensity experienced by the patrol unmanned aerial vehicle is higher; Using a physical-data hybrid driven disturbance prediction model, calculating a basic physical disturbance component representing airflow disturbance in an ideal fluid state based on the motor speed instruction contained in the dynamic state feature value; Using the disturbance prediction model, calculating a dynamic correction disturbance component representing disturbance deviation caused by non-ideal factors based on the body vibration response feature contained in the dynamic state feature value; Vector superimposing the basic physical disturbance component and the dynamic correction disturbance component to generate a predicted disturbance field used for representing environmental effects of the patrol unmanned aerial vehicle; the predicted disturbance field comprises a predicted visual disturbance field representing expected visual medium displacement and a predicted thermal disturbance field representing expected thermal medium displacement; Based on current environment perception image data at a current synchronization time point and historical environment perception image data at a previous synchronization time point, calculating an initial observation optical flow field containing all pixel motion information; Based on the predicted disturbance field, purifying physical disturbance actually caused by the patrol unmanned aerial vehicle by suppressing optical flow vectors inconsistent with the predicted disturbance field in a motion mode to obtain a filtering processing result; Taking the filtering processing result as an observation disturbance field, the observation disturbance field comprising an observation visual disturbance field representing real visual medium motion and an observation thermal disturbance field representing real thermal medium motion; Structurally comparing the predicted disturbance field and the observation disturbance field to calculate a prediction-observation consistency index; Taking the prediction-observation consistency index as a real-time measurement of visual data reliability, dynamically adjusting state information confidence of a visual auxiliary positioning algorithm according to the real-time measurement to finally obtain an optimized positioning result.

2. The method of claim 1, wherein, The step of structurally comparing the predicted disturbance field and the observation disturbance field to calculate a prediction-observation consistency index comprises the following steps: Multi-dimensional feature comparison is performed on the predicted disturbance field and the observed disturbance field from a visible light channel and a thermal imaging channel to generate a visual consistency component and a thermal consistency component, respectively; Thermal information entropy is calculated based on a thermal imaging frame of the current environment perception image data; A thermal channel weight and a visual channel weight are obtained according to the thermal information entropy; A prediction-observation consistency index is calculated based on the thermal channel weight, the visual channel weight, the visual consistency component and the thermal consistency component.

3. The method of claim 1, wherein, The value range of the prediction-observation consistency index is [0, 1]; When the prediction-observation consistency index output tends to be closer to 0, it indicates that the overall consistency degree of the predicted disturbance field and the observed disturbance field is lower; When the prediction-observation consistency index output tends to be closer to 1, it indicates that the overall consistency degree of the predicted disturbance field and the observed disturbance field is higher.

4. The method of claim 3, wherein, The prediction-observation consistency index is used as a real-time measurement of visual data reliability, and the state information confidence of a visual aided positioning algorithm is dynamically adjusted according to the real-time measurement to obtain an optimized positioning result, including: The prediction-observation consistency index is converted into a visual measurement confidence by a preset nonlinear mapping function as a real-time measurement of visual data reliability; The visual measurement confidence is applied as a dynamic weight factor in a state estimation fusion process of the visual aided positioning algorithm; The state information confidence introduced by visual measurement update is modulated by the dynamic weight factor to obtain the optimized positioning result.

5. The method of claim 4, wherein, The preset nonlinear mapping function is a logistic function. The state information confidence introduced by visual measurement update is modulated by the dynamic weight factor, including: The noise covariance matrix related to visual measurement is adjusted by the dynamic weight factor; When the dynamic weight factor increases, the noise covariance matrix is increased, so that the weight of visual measurement in the state fusion process of state information confidence calculation is reduced; When the dynamic weight factor decreases, the noise covariance matrix is reduced, so that the weight of visual measurement in the state fusion process of state information confidence calculation is increased.

6. A positioning optimization system based on a vision-aided positioning algorithm, characterized in that, The system is applied to a patrol unmanned aerial vehicle, and the system includes: The data acquisition module is configured to acquire dynamic instruction data of the inspection UAV, construct a dynamic state feature vector, and the dynamic state feature vector includes a motor speed instruction and a body vibration response feature, the motor speed instruction is synchronized with a hardware timestamp, and the body vibration response feature is related to body micro-vibration data; the body micro-vibration data, a visible light image frame and a thermal imaging image frame having the same hardware timestamp as the dynamic state feature vector are synchronously collected; environment perception image data is formed according to the visible light image frame and the thermal imaging image frame; the motor speed instruction and the body vibration response feature are combined by using a weighted average method according to the contribution of the motor speed instruction and the body vibration response feature to a final result, and a dynamic state feature value is obtained, the dynamic state feature value is a numerical representation of the dynamic state feature vector; the output value range of the dynamic state feature value is limited in the [0, 1] interval; when the dynamic state feature value output tends to be 0, the dynamic state of the inspection UAV tends to be stable and stationary; when the dynamic state feature value output tends to be 1, the dynamic event experienced by the inspection UAV is higher in intensity; The predicted disturbance field establishment module is configured to calculate a basic physical disturbance component representing airflow disturbance in an ideal fluid state based on the motor speed instruction contained in the dynamic state feature value by using a physical-data hybrid driven disturbance prediction model; calculate a dynamic correction disturbance component representing disturbance deviation caused by non-ideal factors based on the body vibration response feature contained in the dynamic state feature value by using the disturbance prediction model; and generate a predicted disturbance field for representing the environmental effect of the inspection UAV by vector superposition of the basic physical disturbance component and the dynamic correction disturbance component; the predicted disturbance field includes a predicted visual disturbance field representing expected visual medium displacement and a predicted thermal disturbance field representing expected thermal medium displacement; The observed disturbance field establishment module is configured to calculate an initial observed optical flow field containing all pixel motion information based on current environment perception image data at a current synchronization time point and historical environment perception image data at a previous synchronization time point; purify physical disturbance actually caused by the inspection UAV by suppressing optical flow vectors inconsistent with the predicted disturbance field in a motion mode based on the predicted disturbance field, to obtain a filtering processing result; and take the filtering processing result as the observed disturbance field, and the observed disturbance field includes an observed visual disturbance field representing real visual medium motion and an observed thermal disturbance field representing real thermal medium motion; The predicted-observed consistency index calculation module is configured to perform structured comparison between the predicted disturbance field and the observed disturbance field, and calculate a predicted-observed consistency index; The optimized positioning module is configured to take the predicted-observed consistency index as a real-time measurement of visual data reliability, dynamically adjust the state information confidence of a visual auxiliary positioning algorithm according to the real-time measurement, and finally obtain an optimized positioning result.

Images