Real-time data processing method based on machine learning algorithm

By adopting a multimodal sensor fusion system based on machine learning algorithms in drones, the problems of low detection rate and reaction delay of existing drone obstacle avoidance systems are solved, and more efficient dynamic obstacle avoidance and safety performance are achieved.

CN120217259AActive Publication Date: 2025-06-27XIAN JESSE ELECTRONICS TECH DEV CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202510591879.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-09
Publication Date
2025-06-27
Estimated Expiration
2045-05-09

AI Technical Summary

Technical Problem

The existing UAV obstacle avoidance system has problems such as low detection rate and delayed response during dynamic detection, resulting in collision accidents.

Method used

Real-time data processing method based on machine learning algorithms is adopted to obtain multimodal comprehensive data through a multimodal sensor fusion system (including 4D millimeter wave radar, vision detection camera and ultrasonic sensor), calculate radar change index, camera change index, ultrasonic change index and drone anomaly coefficient, and perform dynamic obstacle avoidance judgment and sensor abnormality identification.

Benefits of technology

It improves the detection accuracy and response efficiency of the drone to the surrounding environment, reduces the occurrence of collision accidents, and accurately judges sensor abnormalities, improves the safety performance and operating efficiency of the drone.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120217259A_ABST
    Figure CN120217259A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses a machine learning algorithm-based data real-time processing method, which comprises the following steps of: 1, establishing a multi-modal sensor fusion system, and acquiring multi-modal comprehensive data based on the multi-modal sensor fusion system, step 2, calculating a radar change index, a camera change index, an ultrasonic change index, an influence index and an unmanned aerial vehicle abnormal coefficient based on the multi-modal comprehensive data; step 3, performing unmanned aerial vehicle dynamic obstacle avoidance judgment based on an unmanned aerial vehicle abnormal coefficient calculation result. The method comprises the following steps of: 1, determining whether the unmanned aerial vehicle needs to perform dynamic obstacle avoidance, 2, determining the sensor abnormality when the unmanned aerial vehicle needs to perform dynamic obstacle avoidance, and 3, performing feedback based on a sensor abnormality determination result, thereby improving the detection rate of dynamic obstacle avoidance, improving the reaction efficiency, effectively reducing the occurrence of collision accidents of the unmanned aerial vehicle, and improving the safety performance of the unmanned aerial vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a real-time data processing method based on a machine learning algorithm. Background Art

[0002] As an outstanding representative of modern science and technology, drones have been widely used in military, civilian and other fields. Drones, also known as unmanned aerial vehicles (UAVs), are unmanned aircraft controlled by radio remote control equipment and self-contained program control devices. It can be operated by remote control or automatic navigation systems, or completely or intermittently autonomously by onboard computers. The development of drones has gone through an expansion process from military applications to commercial, agricultural, consumer and other fields. With the continuous advancement of technology, the integration of drones in intelligence and artificial intelligence technology has become increasingly in-depth, becoming the core trend of future drone development.

[0003] The existing obstacle avoidance system has many shortcomings in the dynamic detection process. For example, the detection rate for thin wires and small birds is low, and there is a delay in response to fast-moving objects, such as fast-flying or suddenly falling branches, which can easily lead to the drone not having time to avoid and causing a collision accident. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a real-time data processing method based on a machine learning algorithm. It has the advantages of forming a multi-dimensional detection method through a multimodal sensor fusion system combined with a machine learning algorithm, so that the UAV has a more complete and accurate understanding of the surrounding environment, improving the detection rate of dynamic obstacle avoidance, improving reaction efficiency, and effectively reducing the occurrence of UAV collision accidents. At the same time, it can accurately determine which sensor has an abnormality, avoid flight accidents caused by sensor failure, improve the safety performance of the UAV, and improve the operating efficiency of the UAV and the detection accuracy of the data.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a real-time data processing method based on a machine learning algorithm, comprising the following steps:

[0008] Step 1: Establish a multimodal sensor fusion system, and obtain multimodal comprehensive data based on the multimodal sensor fusion system;

[0009] Step 2: Calculate the radar change index, camera change index, ultrasonic change index, impact index and drone anomaly coefficient based on multimodal comprehensive data;

[0010] Step 3: Perform dynamic obstacle avoidance judgment for the UAV based on the calculation result of the UAV anomaly coefficient;

[0011] Step 4: Identify sensor anomalies in the case where it is determined that the UAV needs to perform dynamic obstacle avoidance;

[0012] Step 5: Provide feedback based on the sensor anomaly identification result.

[0013] Preferably, the multi-modal comprehensive data includes a millimeter-wave radar data set, a vision detection camera data set, an ultrasonic data set, and an influencing factor data set. The millimeter-wave radar data set is obtained through a 4D millimeter-wave radar, the vision detection camera data set is obtained through a vision detection camera, and the ultrasonic data set is obtained through an ultrasonic sensor.

[0014] Preferably, the numbering expression of the millimeter-wave radar data set LDsj is:

[0015] LDsj = l1, L2, L3, ···, L n

[0016] The numbering expression of the vision detection camera data set XJsj is:

[0017] XJsj = X1, X2, X3, ···, X n

[0018] The numbering expression of the ultrasonic data set Sbsj is:

[0019] SBsj = S1, S2, S3, ···, S n

[0020] In the expression, LDsj, XJsj, and SBsj represent the millimeter-wave radar data set, the vision detection camera data set, and the ultrasonic data set in sequence. L1, X1, and S1 represent the first millimeter-wave radar data obtained through the 4D millimeter-wave radar, the first vision detection camera data obtained through the vision detection camera, and the first ultrasonic data obtained through the ultrasonic sensor respectively. L n , X n , S n represent the nth millimeter-wave radar data obtained through the 4D millimeter-wave radar, the nth vision detection camera data obtained through the vision detection camera, and the nth ultrasonic data obtained through the ultrasonic sensor respectively. n represents the total number of data in the millimeter-wave radar data set, the vision detection camera data set, and the ultrasonic data set;

[0021] The numbering expression of the influencing factor data set YXsj is:

[0022] YXsj = Hj, Fx

[0023] In the expression, YXsj represents the dataset of influencing factors, and Hj represents the environmental influencing factors in the dataset of influencing factors. The environmental influencing factors include: light intensity, electromagnetic interference index, temperature data, and humidity data;

[0024] Fx represents the flight influencing factors in the dataset of environmental influencing factors. The flight influencing factors include: speed, altitude, distance, and flight time;

[0025] The acquisition time and interval time of the data in the millimeter-wave radar dataset, visual detection camera dataset, ultrasonic dataset, and dataset of influencing factors are the same.

[0026] Preferably, the calculation formula for the radar change index LDbh is:

[0027]

[0028] In the calculation formula, LDbh represents the radar change index, L i represents the current millimeter-wave radar data, L i-1 represents the penultimate millimeter-wave radar data, L i-2 represents the third-to-last millimeter-wave radar data, L i-3 represents the fourth-to-last millimeter-wave radar data.

[0029] Preferably, the calculation formula for the camera change index XJbh is:

[0030]

[0031] In the calculation formula, XJbh represents the camera change index, X i represents the current visual detection camera data, X i-1 represents the penultimate visual detection camera data, X i-2 represents the third-to-last visual detection camera data, X i-3 represents the fourth-to-last visual detection camera data.

[0032] Preferably, the calculation formula for the ultrasonic change index CSbh is:

[0033]

[0034] In the calculation formula, CSbh represents the ultrasonic change index, S i represents the current ultrasonic data, S i-1 represents the penultimate ultrasonic data, S i-2 represents the third-to-last ultrasonic data, S i-3 represents the fourth-to-last ultrasonic data.

[0035] Preferably, the calculation formula of the influence index YXzs is as follows:

[0036] YXzs = α1·HJyx + α2·FXyx

[0037] In the calculation formula, YXzs represents the influence index, HJyx represents the environmental influence index, α1 represents the weight of the environmental influence index, FXyx represents the flight influence index, α2 represents the weight of the flight influence index, and α1 + α2 = 1;

[0038] The calculation formula of the environmental influence index HJyx is as follows:

[0039]

[0040] In the calculation formula, Gq i represents the current light intensity, Dc i represents the electromagnetic interference index, Wd i represents the temperature data, Sd i represents the humidity data; Gq o represents the standard value of the current light intensity, Dc o represents the standard value of the electromagnetic interference index, Wd o represents the standard value of the temperature data, Sd o represents the standard value of the humidity data; β1 represents the weight of the current light intensity, β2 represents the weight of the electromagnetic interference index, β3 represents the weight of the temperature data, β4 represents the weight of the humidity data, and β1 + β2 + β3 + β4 = 1;

[0041] The calculation formula of the flight influence index FXyx is as follows:

[0042]

[0043] In the calculation formula, Dd i represents the current speed, Gd i represents the altitude, Jl i represents the distance, Sj i represents the flight time; Dd o represents the standard value of the current speed, Gd o represents the standard value of the altitude, Jl o represents the standard value of the distance, Sj o represents the standard value of the flight time; γ1 represents the weight of the current speed, γ2 represents the weight of the altitude, γ3 represents the weight of the distance, γ4 represents the weight of the flight time, and γ1 + γ2 + γ3 + γ4 = 1.

[0044] Preferably, the calculation formula of the UAV anomaly coefficient YCxs is as follows:

[0045]

[0046] In the calculation formula, YCxs represents the UAV anomaly coefficient, and I, P, and Q represent the standard instruction response times of the 4D millimeter-wave radar, vision detection camera, and ultrasonic sensor in sequence. I i , P i , Q i represent the current instruction response times of the 4D millimeter-wave radar, vision detection camera, and ultrasonic sensor in sequence.

[0047] Preferably, compare the calculated value of the UAV anomaly coefficient YCxs with the UAV anomaly coefficient threshold. When the calculated value of the UAV anomaly coefficient YCxs is greater than the UAV anomaly coefficient threshold, it is determined that UAV dynamic obstacle avoidance is required currently.

[0048] Preferably, in step four, when it is determined that UAV dynamic obstacle avoidance is required, identify sensor anomalies. The specific method is as follows:

[0049] Compare the radar change index, camera change index, and ultrasonic change index with the radar change threshold, camera change threshold, and ultrasonic change threshold;

[0050] When the radar change index and camera change index exceed the radar change threshold and camera change threshold, and the ultrasonic change index does not exceed the ultrasonic change threshold, send an ultrasonic sensor anomaly signal;

[0051] When the radar change index and ultrasonic change index exceed the radar change threshold and ultrasonic change threshold, and the camera change index does not exceed the camera change threshold, send a vision detection camera anomaly signal;

[0052] When the camera change index and ultrasonic change index exceed the camera change threshold and ultrasonic change threshold, and the radar change index does not exceed the radar change threshold, send a 4D millimeter-wave radar anomaly signal.

[0053] Compared with the prior art, the present invention provides a data real-time processing method based on a machine learning algorithm, which has the following beneficial effects:

[0054] 1. The present invention forms a multi-modal sensor fusion system through a 4D millimeter-wave radar, a vision detection camera, and an ultrasonic sensor. Combining machine learning algorithms, it can make full use of their respective advantages, verify and complement each other, and form a multi-dimensional detection method. When comprehensively evaluating environmental factors and flight factors, not only environmental conditions are considered, but flight data is also analyzed in detail, including the flight speed of the unmanned aerial vehicle (UAV), whose perception and reaction capabilities to obstacles are different at different speeds; flight altitude, the air density, airflow conditions, etc. at different altitudes will affect obstacle avoidance decisions; the distance from obstacles, which is a key factor in determining whether obstacle avoidance is required and how to select an obstacle avoidance path; and flight time, long-term flight may cause factors such as changes in equipment performance. In this way, the UAV can have a more complete and accurate understanding of the surrounding environment, improve the detection rate of dynamic obstacle avoidance, enhance the reaction efficiency, effectively reduce the occurrence of UAV collision accidents, and at the same time avoid the problem that the judgment result is affected by the failure of a single sensor, comprehensively and accurately reflect the obstacle situation of the UAV, make the judgment result more accurate, and reduce errors.

[0055] 2. By comparing the radar change index, the camera change index, and the ultrasonic change index with the corresponding thresholds, the present invention can accurately determine which sensor is abnormal, avoid flight accidents caused by sensor failures, improve the safety performance of the UAV, and improve the operation efficiency of the UAV and the detection accuracy of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flowchart of the method steps of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0058] Please refer to Figure 1 , a method for real-time data processing based on a machine learning algorithm, comprising the following steps:

[0059] Step 1: Establish a multi-modal sensor fusion system, and obtain multi-modal comprehensive data based on the multi-modal sensor fusion system;

[0060] The multi-modal comprehensive data includes a millimeter-wave radar data set, a vision detection camera data set, an ultrasonic data set, and an influencing factor data set. The millimeter-wave radar data set is obtained through a 4D millimeter-wave radar, the vision detection camera data set is obtained through a vision detection camera, and the ultrasonic data set is obtained through an ultrasonic sensor;

[0061] The numbering expression of the millimeter-wave radar dataset LDsj is as follows:

[0062] LDsj = L1, L2, L3, ···, L n

[0063] The numbering expression of the vision detection camera dataset XJsj is as follows:

[0064] XJsj = X1, X2, X3, ···, X n

[0065] The numbering expression of the ultrasonic dataset SBsj is as follows:

[0066] SBsj = S1, S2, S3, ···, S n

[0067] In the expression, LDsj, XJsj, and SBsj represent the millimeter-wave radar dataset, the vision detection camera dataset, and the ultrasonic dataset respectively. L1, X1, and S1 represent the first millimeter-wave radar data obtained by the 4D millimeter-wave radar, the first vision detection camera data obtained by the vision detection camera, and the first ultrasonic data obtained by the ultrasonic sensor. L n , X n , S n represent the nth millimeter-wave radar data obtained by the 4D millimeter-wave radar, the nth vision detection camera data obtained by the vision detection camera, and the nth ultrasonic data obtained by the ultrasonic sensor respectively. n represents the total number of data in the millimeter-wave radar dataset, the vision detection camera dataset, and the ultrasonic dataset;

[0068] The numbering expression of the influencing factor dataset YXsj is as follows:

[0069] YXsj = Hj, Fx

[0070] In the expression, YXsj represents the influencing factor dataset, and Hj represents the environmental influencing factors in the influencing factor dataset. The environmental influencing factors include: light intensity, electromagnetic interference index, temperature data, and humidity data;

[0071] Fx represents the flight influencing factors in the environmental influencing factor dataset. The flight influencing factors include: speed, altitude, distance, and flight time;

[0072] The acquisition time and interval time of the data in the millimeter-wave radar dataset, the vision detection camera dataset, the ultrasonic dataset, and the influencing factor dataset are the same;

[0073] A multi-modal sensor fusion system is formed by a 4D millimeter-wave radar, a vision detection camera, and an ultrasonic sensor, which can make full use of their respective advantages, verify and complement each other, form a multi-dimensional detection method, enable the UAV to have a more complete and accurate understanding of the surrounding environment, improve the detection rate of dynamic obstacle avoidance, enhance the reaction efficiency, effectively reduce the occurrence of UAV collision accidents, and at the same time avoid the problem that a single sensor failure affects the judgment result;

[0074] Step 2: Calculate the radar change index, camera change index, ultrasonic change index, influence index, and UAV anomaly coefficient based on the multi-modal comprehensive data;

[0075] The calculation formula for the radar change index LDbh is:

[0076]

[0077] In the calculation formula, LDbh represents the radar change index, L i represents the current millimeter-wave radar data, L i-1 represents the penultimate millimeter-wave radar data, L i-2 represents the third-to-last millimeter-wave radar data, L i-3 represents the fourth-to-last millimeter-wave radar data;

[0078] represents the ratio of the total value of the last two millimeter-wave radar data to the total value of the third-to-last and fourth-to-last millimeter-wave radar data;

[0079] The calculation of the radar change index is to add the current radar data and the last radar data as the numerator, and add the penultimate and third-to-last radar data as the denominator, which amplifies the difference between adjacent data. When an obstacle approaches or moves away, the changes in the numerator and denominator will cause the ratio to change significantly, making it easier to detect the change in radar data and improving the detection accuracy of close-range obstacles;

[0080] The calculation formula for the camera change index XJbh is:

[0081]

[0082] In the calculation formula, XJbh represents the camera change index, X i represents the current vision detection camera data, X i-1 represents the penultimate vision detection camera data, X i-2 represents the third-to-last vision detection camera data, X i-3 represents the fourth-to-last vision detection camera data;

[0083] Represents the ratio of the total value of the data of the last two visual detection cameras to the total value of the data of the third and fourth visual detection cameras from the bottom;

[0084] The camera change index reflects the abnormal conditions during the monitoring process of the visual detection cameras. By combining multiple data for determination, the error is reduced and the accuracy of the calculated value of the camera change index is improved;

[0085] The calculation formula for the ultrasonic change index CSbh is:

[0086]

[0087] In the calculation formula, CSbh represents the ultrasonic change index, S i represents the current ultrasonic data, S i-1 represents the second ultrasonic data from the bottom, S i-2 represents the third ultrasonic data from the bottom, S i-3 represents the fourth ultrasonic data from the bottom;

[0088] Represents the ratio of the total value of the last two ultrasonic data to the total value of the third and fourth ultrasonic data from the bottom;

[0089] By adding the current ultrasonic data to the last ultrasonic data as the numerator, and adding the second and third ultrasonic data from the bottom as the denominator, it plays a role in smoothing the data to a certain extent, reducing the impact of the fluctuation of a single data point on the result, making the ultrasonic change index more stable and reliable, and being able to effectively capture the change trend of the ultrasonic data;

[0090] The calculation formula for the influence index YXzs is:

[0091] YXzs = α1·HJyx + α2·FXyx

[0092] In the calculation formula, YXzs represents the influence index, HJyx represents the environmental influence index, α1 represents the weight of the environmental influence index, FXyx represents the flight influence index, α2 represents the weight of the flight influence index, and α1 + α2 = 1;

[0093] The calculation formula for the environmental influence index HJyx is:

[0094]

[0095] In the calculation formula, Gq i 、Dc i 、Wd i 、Sd i represent the current light intensity, electromagnetic interference index, temperature data, humidity data respectively, Gq o, Dc o , Wd o , Sd o respectively represent the standard values of the current light intensity, electromagnetic interference index, temperature data, and humidity data, and β1, β2, β3, and β4 respectively represent the weights of the current light intensity, electromagnetic interference index, temperature data, and humidity data, and β1 + β2 + β3 + β4 = 1;

[0096] The calculation formula for the flight influence index FXyx is:

[0097]

[0098] In the calculation formula, Dd i , Gd i , Jl i , Sj i respectively represent the current speed, altitude, distance, and flight time, and Dd o , Gd o , Jl o , Sj o respectively represent the standard values of the current speed, altitude, distance, and flight time, and γ1, γ2, γ3, and γ4 respectively represent the weights of the current speed, altitude, distance, and flight time;

[0099] γ1 + γ2 + γ3 + γ4 = 1;

[0100] By comprehensively evaluating the influence of environmental conditions such as light intensity, electromagnetic interference index, temperature, and humidity on the obstacle avoidance system, and by comprehensively analyzing the influence of flight factors such as speed, altitude, distance, and flight time on the obstacle avoidance system, and then combining environmental and flight factors, the obstacle avoidance system is made not to rely on a single factor, avoiding misjudgment of the influence on the obstacle avoidance system caused by the one-sidedness of a certain factor, and comprehensively reflecting the influence of multiple factors on the determination result;

[0101] The calculation formula for the UAV anomaly coefficient YCxs is:

[0102]

[0103] In the calculation formula, YCxs represents the UAV anomaly coefficient, and I, P, and Q respectively represent the standard command response times of the 4D millimeter-wave radar, vision detection camera, and ultrasonic sensor, and I i , P i , Q i respectively represent the current command response times of the 4D millimeter-wave radar, vision detection camera, and ultrasonic sensor;

[0104] Represents the relative change rate of the current command response time of the 4D millimeter-wave radar with respect to its standard command response time;

[0105] Reflects the relative deviation between the current command response time of the vision detection camera and the standard command response time;

[0106] Embodies the relative difference between the current command response time of the ultrasonic sensor and the standard command response time;

[0107] By synthesizing the differences in the respective response times, it reduces the influence existing when the device itself acquires data, improves the accuracy of calculating the UAV anomaly coefficient, combines radar, camera, and ultrasonic data, comprehensively reflects the obstacle situation of the UAV, makes the judgment result more accurate, and reduces errors;

[0108] Step Three: Perform dynamic obstacle avoidance judgment for the UAV based on the calculated result of the UAV anomaly coefficient;

[0109] Compare the calculated value of the UAV anomaly coefficient YCxs with the UAV anomaly coefficient threshold. When the calculated value of the UAV anomaly coefficient YCxs is greater than the UAV anomaly coefficient threshold, it is determined that dynamic obstacle avoidance of the UAV is required currently;

[0110] Step Four: Identify sensor anomalies in the case where it is determined that dynamic obstacle avoidance of the UAV is required. The specific method is as follows:

[0111] Compare the radar change index, camera change index, and ultrasonic change index with the radar change threshold, camera change threshold, and ultrasonic change threshold;

[0112] When the radar change index and camera change index exceed the radar change threshold and camera change threshold, and the ultrasonic change index does not exceed the ultrasonic change threshold, send an ultrasonic sensor anomaly signal;

[0113] When the radar change index and ultrasonic change index exceed the radar change threshold and ultrasonic change threshold, and the camera change index does not exceed the camera change threshold, send a vision detection camera anomaly signal;

[0114] When the camera change index and ultrasonic change index exceed the camera change threshold and ultrasonic change threshold, and the radar change index does not exceed the radar change threshold, send a 4D millimeter-wave radar anomaly signal;

[0115] By comparing the radar change index, camera change index, and ultrasonic change index with the corresponding thresholds, it can accurately determine which sensor is abnormal, avoid flight accidents caused by sensor failures, improve the safety performance of the UAV, improve the operation efficiency of the UAV, and the detection accuracy of data;

[0116] Step Five: Provide feedback based on the sensor anomaly identification result.

[0117] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time data processing method based on a machine learning algorithm, characterized in that: The following steps are involved: Step 1: Establish a multimodal sensor fusion system, and obtain multimodal comprehensive data based on the multimodal sensor fusion system; Step 2: Calculate the radar change index, camera change index, ultrasonic change index, impact index and drone anomaly coefficient based on multimodal comprehensive data; Step 3: Make dynamic obstacle avoidance judgment for the UAV based on the calculation result of the UAV abnormality coefficient; Step 4: Identify sensor anomalies when it is determined that the drone needs to perform dynamic obstacle avoidance; Step 5: Provide feedback based on the sensor abnormality identification results.

2. The method for real-time data processing based on a machine learning algorithm according to claim 1, characterized in that: The multimodal comprehensive data includes a millimeter-wave radar data set, a visual detection camera data set, an ultrasonic data set, and an influencing factor data set. The millimeter-wave radar data set is acquired through a 4D millimeter-wave radar, the visual detection camera data set is acquired through a visual detection camera, and the ultrasonic data set is acquired through an ultrasonic sensor.

3. The method for real-time data processing based on a machine learning algorithm according to claim 2, characterized in that: The numbering expression of the millimeter wave radar data set LDsj is: <h2 style=";text-align:left;direction:ltr">LDsj = L1, L2, L3, L<h2 style=";text-align:left;direction:ltr"> n The numbering expression of the visual inspection camera dataset XJsj is: XJsj=X1、X2、X3、···、X n The numbering expression of the ultrasonic data set SBsj is: <h2 style=";text-align:left;direction:ltr">SBsj = S1, S2, S3, S<h2 style=";text-align:left;direction:ltr"> n In the expression, LDsj, XJsj, and SBsj represent the millimeter-wave radar data set, the visual detection camera data set, and the ultrasonic data set respectively. L1, X1, and S1 represent the first millimeter-wave radar data obtained by the 4D millimeter-wave radar, the first visual detection camera data obtained by the visual detection camera, and the first ultrasonic data obtained by the ultrasonic sensor respectively. n , X n , S n represents the nth millimeter-wave radar data acquired by the 4D millimeter-wave radar, the nth visual detection camera data acquired by the visual detection camera, and the nth ultrasonic data acquired by the ultrasonic sensor, respectively. n represents the total number of data in the millimeter-wave radar dataset, the visual detection camera dataset, and the ultrasonic dataset; The numbering expression of the influencing factor data set YXsj is: YXsj=Hj、Fx In the expression, YXsj represents the influencing factor data set, and Hj represents the environmental influencing factors in the influencing factor data set. The environmental influencing factors include: light intensity, electromagnetic interference index, temperature data, and humidity data; Fx represents the flight influencing factors in the environmental influencing factor data set, and the flight influencing factors include: speed, altitude, distance, and flight time; The data collection time and interval time of the millimeter wave radar data set, the visual detection camera data set, the ultrasonic data set, and the influencing factor data set are consistent.

4. The method for real-time data processing based on a machine learning algorithm according to claim 3, characterized in that: The calculation formula of the radar change index LDbh is: In the calculation formula, LDbh represents the radar change index, L i Represents the current millimeter-wave radar data, L i-1 Represents the second to last millimeter-wave radar data, L i-2 Represents the third to last millimeter-wave radar data, L i-3 Represents the fourth to last millimeter-wave radar data.

5. The method for real-time data processing based on a machine learning algorithm according to claim 4, characterized in that: The calculation formula of the camera change index XJbh is: In the calculation formula, XJbh represents the camera change index, X i Represents the current visual inspection camera data, X i-1 Represents the second-to-last visual inspection camera data, X i-2 Represents the third to last visual inspection camera data, X i-3 Represents the fourth to last visual inspection camera data.

6. The method for real-time data processing based on a machine learning algorithm according to claim 5, characterized in that: The calculation formula of the ultrasonic change index CSbh is: In the calculation formula, CSbh represents the ultrasonic change index, S i Represents the current ultrasonic data, S i-1 Represents the second to last ultrasonic data, S i-2 Represents the third to last ultrasonic data, S i-3 Represents the fourth to last ultrasonic data.

7. The method for real-time data processing based on a machine learning algorithm according to claim 6, characterized in that: The calculation formula of the impact index YXzs is: YXzs=α1·HJyx+α2·FXyx In the calculation formula, YXzs represents the impact index, HJyx represents the environmental impact index, α1 represents the weight of the environmental impact index, FXyx represents the flight impact index, α2 represents the weight of the flight impact index, and α1+α2=1; The calculation formula of the environmental impact index HJyx is: In the calculation formula, Gq i Represents the current light intensity, Dc i Represents the electromagnetic interference index, Wd i Represents temperature data, Sd i Represents humidity data; Gq o Represents the standard value of the current light intensity, Dc o Represents the standard value of the electromagnetic interference index, Wd o Represents the standard value of temperature data, Sd o represents the standard value of humidity data; β1 represents the weight of the current light intensity, β2 represents the weight of the electromagnetic interference index, β3 represents the weight of the temperature data, β4 represents the weight of the humidity data, and β1+β2+β3+β4=1; The calculation formula of the flight impact index FXyx is: In the calculation formula, Dd i Represents the current speed, Gd i Represents height, Jl i Represents distance, Sj i represents the flight time; D o Represents the standard value of the current speed, Gd o Represents the standard value of height, Jl o Represents the standard value of distance, Sj o represents the standard value of the flight time; γ1 represents the weight of the current speed, γ2 represents the weight of the altitude, γ3 represents the weight of the distance, γ4 represents the weight of the flight time, and γ1+γ2+γ3+γ4=1.

8. The method for real-time data processing based on a machine learning algorithm according to claim 7, characterized in that: The calculation formula of the UAV abnormality coefficient YCxs is: In the calculation formula, YCxs represents the drone anomaly coefficient, I, P, and Q represent the standard command response time of the 4D millimeter wave radar, visual detection camera, and ultrasonic sensor respectively, and I i , P i , Q i Represents the current command response time of 4D millimeter wave radar, visual detection camera, and ultrasonic sensor respectively.

9. The method for real-time data processing based on a machine learning algorithm according to claim 8, characterized in that: The calculated value of the drone abnormality coefficient YCxs is compared with the drone abnormality coefficient threshold. When the calculated value of the drone abnormality coefficient YCxs is greater than the drone abnormality coefficient threshold, it is determined that dynamic obstacle avoidance of the drone is currently required.

10. The method for real-time data processing based on a machine learning algorithm according to claim 9, characterized in that: When it is determined that the UAV needs to perform dynamic obstacle avoidance, the sensor anomaly is identified. The specific method is as follows: Comparing the radar change index, the camera change index, and the ultrasonic change index with the radar change threshold, the camera change threshold, and the ultrasonic change threshold; When the radar change index and the camera change index exceed the radar change threshold and the camera change threshold, and the ultrasonic change index does not exceed the ultrasonic change threshold, an ultrasonic sensor abnormal signal is sent; When the radar change index and the ultrasonic change index exceed the radar change threshold and the ultrasonic change threshold, and the camera change index does not exceed the camera change threshold, a visual detection camera abnormality signal is sent; When the camera change index and the ultrasonic change index exceed the camera change threshold and the ultrasonic change threshold, and the radar change index does not exceed the radar change threshold, a 4D millimeter wave radar abnormal signal is sent.

Citation Information

Patent Citations

  • Disaster rescue rotor unmanned plane obstacle avoiding system based on OODA ring multi-sensor information fusion

    CN107783545A

  • Single rotor wing plant protection unmanned aerial vehicle obstacle avoidance system based on multi-sensor information fusion technology

    CN107783549A

  • Unmanned aerial vehicle flight state monitoring system and method

    CN118816937A

  • Identifying a fault in a sensor-based air data system

    DE102023107326A1