Weather identification method and device, aircraft and storage medium

Abstract

The invention relates to the technical field of aircraft data processing, and discloses a weather identification method and device, an aircraft and a storage medium. The method comprises the following steps: acquiring environment images acquired at at least one acquisition view angle of an aircraft, and determining an image weather result according to all the environment images; when the image weather result is rainy weather or foggy weather, determining the number of point cloud noisy points according to radar point cloud data of a plurality of point cloud acquisition points in a preset acquisition range, and determining a radar weather result according to the number of point cloud noisy points; and determining a flight weather identification result of the aircraft according to the image weather result and the radar weather result. According to the invention, data analysis of different processes is carried out on the environment image data and the radar point cloud data, and the flight weather identification result is obtained through comparison and fusion of analysis results, so that multi-dimensional cross validation is realized, the environment weather state can be accurately identified in real time, the reliability and fault tolerance of weather identification are improved, and the method is suitable for popularization and application. And the flight safety of the aircraft is ensured.

Description

Technical Field

[0001] This invention relates to the field of aircraft data processing technology, and in particular to a weather identification method, device, aircraft, and storage medium. Background Technology

[0002] Aircraft flying in low-altitude airspace are susceptible to weather changes, especially in rainy or foggy weather. Water droplets in the air reduce visibility, affecting the aircraft's field of vision and obstacle detection capabilities. Current technologies rely on images acquired by visual sensors for weather identification. However, image clarity is significantly affected by inclement weather such as rain and fog, and image transmission communication links are also limited. This results in insufficient reliability of weather identification methods based on a single visual sensor, reducing the accuracy of identification under adverse weather conditions. Summary of the Invention

[0003] Therefore, it is necessary to provide a weather identification method, device, aircraft, and storage medium to address the aforementioned technical problems and solve the issues of insufficient reliability and low accuracy in existing weather identification systems.

[0004] A weather identification method, comprising: Acquire environmental images from at least one acquisition viewpoint of the aircraft, and determine image weather results based on all said environmental images; When the image weather result is rainy or foggy, the number of point cloud noise points is determined based on radar point cloud data from several point cloud collection points within a preset collection range, and the radar weather result is determined based on the number of point cloud noise points. Based on the image weather results and the radar weather results, the flight weather identification result of the aircraft is determined.

[0005] A weather recognition device, comprising: An image recognition module is used to acquire environmental images collected from at least one acquisition viewpoint of the aircraft, and to determine the image weather result based on all the environmental images; The radar identification module is used to determine the number of point cloud noise points based on radar point cloud data from several point cloud collection points within a preset collection range when the image weather result is rainy or foggy, and to determine the radar weather result based on the number of point cloud noise points. The comparison and analysis module is used to determine the flight weather identification result of the aircraft based on the image weather result and the radar weather result.

[0006] An aircraft includes a processor and a memory, wherein, Memory, used to store computer programs; The processor is used to execute the program stored in the memory to implement the weather recognition method described above.

[0007] A computer-readable storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the weather recognition method described above.

[0008] In the aforementioned weather recognition method, device, aircraft, and storage medium, the weather recognition method acquires environmental images from at least one acquisition perspective of the aircraft, determines the image weather result based on all environmental images, and when the image weather result indicates rainy or foggy weather, determines the number of point cloud noise points based on radar point cloud data from several point cloud acquisition points within a preset acquisition range, and determines the radar weather result based on the number of point cloud noise points; finally, the flight weather recognition result of the aircraft is determined based on the image weather result and the radar weather result. This invention combines image sensors and radar sensors to jointly acquire environmental data for weather recognition, avoiding the problem of insufficient reliability in weather recognition using a single sensor. Simultaneously, this invention uses a redundancy mechanism to perform different processes of data analysis on environmental image data and radar point cloud data, and obtains the flight weather recognition result through comparison and fusion of the analysis results, achieving multi-dimensional cross-validation. This enables real-time and accurate identification of environmental weather conditions, improves the reliability and fault tolerance of weather recognition, and ensures the flight safety of the aircraft. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of a weather recognition method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a weather recognition device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of an aircraft in one embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] The weather recognition method provided in this embodiment can be applied to the electronic control unit (ECU) of an aircraft. The ECU communicates with a data acquisition component, which is a terminal for real-time data acquisition, including but not limited to image acquisition devices (such as cameras) and radar sensors. The image acquisition device collects environmental images of the surrounding environment and sends them to the ECU, while the radar sensor collects radar point cloud data of the surrounding environment and sends it to the ECU. The ECU analyzes the environmental images to obtain image weather results and analyzes the radar point cloud data to obtain radar weather results. It then combines the image weather results and the radar weather results to determine the flight weather recognition result.

[0013] In one embodiment, such as Figure 1 As shown, a weather recognition method is provided, taking the application in the electronic control unit of an aircraft as an example, including the following steps S10-S30.

[0014] S10. Acquire environmental images from at least one acquisition viewpoint of the aircraft, and determine the image weather result based on all the environmental images.

[0015] Understandingly, the acquisition viewpoint refers to the specific angle or direction used by the image acquisition device when capturing images. Different acquisition viewpoints yield environmental images corresponding to different field-of-view environments. The aircraft can be equipped with image acquisition devices in different directions as needed. For example, two cameras can be placed in front, behind, left, and right of the aircraft as image acquisition devices, and eight cameras can cover a 360-degree horizontal field of view. The electronic control unit combines environmental images from all acquisition viewpoints acquired simultaneously for weather recognition, covering weather conditions in all directions and improving the accuracy and robustness of image-based weather results. Image-based weather results refer to the specific classification and judgment results made on the current weather conditions from the perspective of image analysis.

[0016] In one embodiment, step S10, namely determining the image weather result based on all the environmental images, includes: S101. Using a preset weather classification model, perform image recognition processing on the environmental images collected from each of the collection perspectives to obtain the initial weather results for each of the collection perspectives. S102. Smooth the initial weather results and historical initial weather results of each acquisition perspective to obtain the smoothed weather results of each acquisition perspective; the historical initial weather results refer to the initial weather results of the acquisition perspective at multiple historical moments within a preset historical time period. S103. Based on the weather smoothing results of each acquisition perspective, determine the view weather results for each acquisition perspective. S104. Determine the image weather results based on the view weather results of all the acquired viewpoints.

[0017] Understandably, the preset weather classification model is a pre-trained neural network model used to analyze and predict the probability of different weather types from input environmental images. Besides predicting the probability of different weather types, the preset weather classification model can also analyze the probability of image quality types. Specifically, the input to the preset weather classification model is an environmental image captured from a single viewpoint. The initial output weather result can include only the probability of rain and fog, or it can be a combination of probability values ​​for rain, fog, night, image blur, and image occlusion, such as {P(rain), P(fog), P(night), P(blur), P(occlusion)}. Before the electronic control unit analyzes and processes the environmental image using the preset weather classification model, it needs to undergo a training process. Specifically, firstly, historical environmental images and their corresponding actual weather type labels and image quality type labels are collected as sample data. Next, after predicting the sample data using the constructed initial weather classification model, the model parameters are adjusted based on the deviation between the output predicted sample label and the corresponding actual sample label, until the deviation can be controlled within a preset threshold, indicating that the training of the preset weather classification model is complete. The initial weather classification model can use ResNet-50 as the backbone network, or it can be constructed using a CNN architecture (such as VGG, DenseNet, EfficientNet, etc.), a Visual Transformer (ViT) architecture, and a lightweight model (such as MobileNet). The initial weather result refers to the output result when the environmental image is input into the preset weather classification model.

[0018] The image acquisition unit acquires environmental images at a preset image acquisition frequency and sends them to the electronic control unit. The preset image acquisition frequency can be a default value or can be adjusted as needed; for example, the default value is 10Hz (acquiring an environmental image every 0.1 seconds). For each acquisition viewpoint, each acquisition moment corresponds to one frame of environmental image, and each frame of environmental image corresponds to an initial weather result and a weather smoothing result. The historical initial weather result refers to the initial weather result of multiple historical moments within a preset historical time period for the current acquisition viewpoint. The preset historical time period is a pre-defined time window range that includes multiple historical acquisition moments; for example, the default value of the preset historical time period is 1 second. Smoothing is a signal smoothing process that calculates the average value of multiple consecutive data points. The weather smoothing result refers to the result of smoothing using the initial weather results from multiple acquisition moments. Specifically, for each acquisition viewpoint, when the preset image acquisition frequency is 10Hz and the preset historical time period is 1 second, the average value of the initial weather result corresponding to the current frame of environmental image and the historical initial weather results corresponding to 10 frames of environmental images within the past second is used as the weather smoothing result for that acquisition viewpoint. For example, the probability of rainy weather in the weather smoothing results is the average of the probability of rainy weather at the current collection time and the probability of rainy weather at all historical collection times within the past second.

[0019] The electronic control unit (ECU) performs a comprehensive analysis of all probability values ​​in the weather smoothing results to obtain the viewpoint weather result. The viewpoint weather result refers to the result used to characterize the specific weather type and image quality type corresponding to a particular acquisition viewpoint. The ECU compares and analyzes the viewpoint weather results from all acquisition viewpoints, comparing whether the viewpoint weather results from different acquisition viewpoints are consistent, thereby determining the image weather result.

[0020] This embodiment effectively identifies the characteristics of environmental images from different acquisition perspectives by using a pre-set weather classification model, capturing key weather-related information in the environmental images and obtaining more accurate initial weather results. By comprehensively considering current and historical data, this embodiment smooths the initial weather results from each acquisition perspective and historical initial weather results, avoiding abrupt changes in weather identification due to individual abnormal data and improving robustness. Furthermore, this embodiment evaluates weather conditions from multiple angles, reducing misjudgments caused by the limitations of a single perspective and ensuring the comprehensiveness and accuracy of weather identification.

[0021] In one embodiment, step S103, namely determining the view weather result for each acquisition view based on the weather smoothing result of each acquisition view, includes: S1031. For each of the aforementioned acquisition perspectives, obtain the prediction type probability corresponding to each preset weather type in the weather smoothing result; S1032. Determine the Boolean value of the prediction type corresponding to each preset weather type based on the prediction type probability and the preset type probability threshold corresponding to each preset weather type; S1033. Determine the view weather result of the acquisition view based on the prediction type Boolean value corresponding to all the preset weather types.

[0022] Understandably, preset weather types are pre-defined weather and image quality types, including but not limited to rainy weather, foggy weather, nighttime, image blur, and image occlusion. Predicted type probability is the probability that an environmental image, after image recognition, matches the features corresponding to each preset weather type. For each acquisition viewpoint, the weather smoothing result is a combination of smoothed probability values ​​for rainy weather, foggy weather, nighttime weather, image blur, and image occlusion, such as { (rain), (Foggy day) (Night) (Vague), (Covering)}, where, This represents the average of current data and historical data. The preset type probability threshold is a pre-defined critical value used to determine whether the image recognition result of an environmental image belongs to a specific weather category. It can be set to a default value or adjusted as needed. Each preset weather type corresponds to its own preset type probability threshold and its own predicted type Boolean value. The predicted type Boolean value is a binary output generated based on the comparison between the predicted type probability and the corresponding threshold, including 1 and 0, where 1 represents "yes" and 0 represents "no". For example, the preset type probability threshold for rainy weather is 0.5. When the predicted type probability for rainy weather in the weather smoothing result is "yes", the threshold value is 0.5. When the value for "(Rainy Day)" is 0.7", the Boolean value for the prediction type corresponding to rainy weather is 1. For each acquisition viewpoint, the electronic control unit determines the viewpoint weather result by combining the Boolean values ​​of the prediction types corresponding to all preset weather types.

[0023] This embodiment provides a clear standard for determining whether a certain preset weather type should be identified as the final perspective weather result by using a preset type probability threshold. At the same time, it converts the threshold comparison result into a prediction type Boolean value, which facilitates subsequent data analysis and can quickly obtain perspective weather results, thereby improving data processing efficiency.

[0024] In one embodiment, the viewpoint weather results include multiple preset weather types corresponding to prediction type Boolean values, where the preset weather types include rainy weather and foggy weather; step S104, namely determining the image weather results based on the viewpoint weather results of all the acquired viewpoints, includes: S1041. When the Boolean value of the prediction type corresponding to rainy weather in the weather results of at least one of the acquisition perspectives is 1, the image weather result is determined to be rainy weather. S1042. When the prediction type Boolean value for rainy weather in all the weather results of the acquisition viewpoints is 0, and the prediction type Boolean value for foggy weather in at least one of the weather results of the acquisition viewpoints is 1, the image weather result is determined to be foggy weather.

[0025] Understandably, different acquisition perspectives may capture different weather features, resulting in different perspective weather results. For each acquisition perspective, the perspective weather result includes multiple Boolean values ​​corresponding to preset weather types. When the preset weather types include rainy and foggy weather, the electronic control unit compares and analyzes the Boolean values ​​of the prediction types for rainy and foggy weather in the perspective weather results of all acquisition perspectives to obtain the result regarding the identification of rainy or foggy weather in the image weather result. Specifically, when the Boolean value of the prediction type for rainy weather in the perspective weather result of at least one acquisition perspective is 1, it indicates that one or more acquisition perspectives have identified rainy weather from the image dimension, and the electronic control unit determines the image weather result as rainy weather. When the Boolean values ​​of the prediction types for rainy weather in the perspective weather results of all acquisition perspectives are 0, and the Boolean value of the prediction type for foggy weather in the perspective weather result of at least one acquisition perspective is 1, the electronic control unit determines the image weather result as foggy weather. When the Boolean values ​​of the prediction types for rainy and foggy weather in the perspective weather results of all acquisition perspectives are 0, the electronic control unit determines the image weather result as sunny weather.

[0026] This embodiment, by integrating weather identification results from multiple perspectives, can more comprehensively and accurately determine the overall weather conditions, reducing misjudgments caused by the limitations of a single perspective. Furthermore, by meticulously distinguishing the criteria for judging rainy and foggy weather, this embodiment provides targeted basis for further analysis of rainy and foggy weather, improving the accuracy of identifying severe weather types such as rain and fog.

[0027] S20. When the image weather result is rainy or foggy, determine the number of point cloud noise points based on radar point cloud data from several point cloud collection points within a preset collection range, and determine the radar weather result based on the number of point cloud noise points.

[0028] Understandably, the weather type in the image weather results is categorized as rainy, foggy, or sunny. When the weather type in the image weather results is rainy or foggy, the electronic control unit (ECU) needs to analyze the radar point cloud data to obtain the radar weather results, so that the final flight weather identification result can be determined by combining the image weather results and the radar weather results. When the weather type in the image weather results is sunny, the ECU directly determines the aircraft's flight weather identification result as sunny.

[0029] The preset acquisition range is the pre-defined scanning range of the radar sensor. Point cloud acquisition points refer to data acquisition points located within the scanning range of the radar sensor. Radar point cloud data is information used to characterize the position and reflection intensity of objects corresponding to each point cloud acquisition point in the environment. Point cloud noise count refers to the number of noise points in the point cloud acquisition points. Noise points are anomalous points in the point cloud acquisition points that deviate from the surrounding environmental point cloud data. In point cloud data, noise points are isolated points that do not conform to normal geometric structures. For example, raindrops appear as randomly distributed noise points in a point cloud; by identifying and statistically analyzing these noise points, rainfall can be analyzed. Radar weather results refer to the specific classification and judgment results made on the current weather conditions from the perspective of radar scan analysis.

[0030] In one embodiment, the number of point cloud noise points includes the number of noise points in several point cloud annular intervals; step S20, namely determining the number of point cloud noise points based on radar point cloud data from several point cloud collection points within a preset collection range, includes: S201. Perform flatness analysis on the radar point cloud data of each point cloud acquisition point and the radar point cloud data of a preset number of flattened point cloud acquisition points adjacent to the point cloud acquisition point, and obtain the acquisition point label of the point cloud acquisition point based on the flatness analysis results. S202. Based on the preset interval distance, the preset collection range is divided into several point cloud annular intervals. The number of all point cloud collection points with noise labels in each point cloud annular interval is determined as the interval noise number of each point cloud annular interval.

[0031] Understandably, within the preset acquisition range, each point cloud acquisition point represents a radar reflection signal at a spatial location. The electronic control unit classifies each point cloud acquisition point and adds an acquisition point label, dividing all point cloud acquisition points into two groups: "noise points" or "valid points." Acquisition point labels are classification identifiers used to characterize whether a point cloud acquisition point is noise, including noise labels and valid point labels. The preset flatness quantity is a pre-set number of surrounding acquisition points used to evaluate the flatness between a specific acquisition point in the point cloud and its surrounding acquisition points. A default value can be set, or it can be adjusted as needed. Flatness analysis is a process used to evaluate the degree of fluctuation between multiple acquisition points in the point cloud.

[0032] In one specific embodiment, flatness analysis is achieved using the horizontal resolution of the point cloud. The horizontal resolution of the point cloud refers to the average distance in the horizontal direction between two adjacent acquisition points in the point cloud, measured in centimeters or meters. The electronic control unit calculates the horizontal resolution of the point cloud by comparing the radar point cloud data of each acquisition point with the data of a preset number of adjacent flat point cloud acquisition points. The horizontal resolution of the point cloud is compared with a preset resolution threshold to determine the acquisition point label for that point cloud acquisition point. If the horizontal resolution of the point cloud reaches the preset resolution threshold, the acquisition point label is a noise label; if the horizontal resolution of the point cloud has not yet reached the preset resolution threshold, the acquisition point label is a valid point label.

[0033] The radar scan's preset acquisition range is a circular area. The electronic control unit (ECU) divides this range into several point cloud annular intervals based on a preset interval distance. The preset interval distance is a pre-defined value used to divide the circular acquisition range into multiple concentric rings. A point cloud annular interval refers to the multiple concentric rings formed after the preset acquisition range is divided. For example, if the preset interval distance is 10 meters and the preset acquisition range is a circular area with a diameter of 100 meters, then 10 point cloud annular intervals are formed: (0~10 meters), (10~20 meters), ..., (90~100 meters). The ECU determines the number of point cloud acquisition points labeled as noise points within each point cloud annular interval as the interval noise count for that annular interval. The sum of the interval noise counts for all point cloud annular intervals is the total point cloud noise count for the preset acquisition range.

[0034] This embodiment analyzes the flatness of each point cloud acquisition point and its adjacent acquisition points, enabling more accurate filtering of noise points in the point cloud acquisition points. Simultaneously, this embodiment divides the preset acquisition range into several point cloud annular intervals and counts the number of noise points within each annular interval. This facilitates comparative analysis based on the number and distribution of noise points in each annular interval, improving the efficiency of subsequent data analysis.

[0035] In one embodiment, step S20, namely determining the radar weather result based on the number of point cloud noise points, includes: S203. Determine the noise quantity comparison value based on the number of point cloud noise points within the first preset distance range and the number of point cloud noise points within the preset acquisition range. S204. If the noise number comparison value is greater than the preset comparison threshold, the radar weather result is determined to be foggy weather, and the radar fog visibility in the radar weather result is determined based on the radar point cloud data of all point cloud collection points whose collection point label is noise label. S205. If the noise number comparison value is less than or equal to the preset comparison threshold, the radar weather result is determined to be rainy weather, and the radar rainfall in the radar weather result is determined according to the number of noise points in all point cloud ring intervals within the second preset distance range. The radar visibility in foggy weather is determined based on the noise depth values ​​of all point cloud acquisition points labeled as noise points and a preset noise quantile threshold. The noise depth value of each point cloud acquisition point is determined based on the radar point cloud data of that point cloud acquisition point. The radar rainfall is determined based on the interval rainfall of all point cloud annular intervals. The interval rainfall of each point cloud annular interval is calculated by converting the interval point cloud noise density of that point cloud annular interval. The interval point cloud noise density of each point cloud annular interval is determined based on the interval noise number and annular interval volume of that point cloud annular interval.

[0036] Understandably, both rain and fog affect radar point clouds due to the scattering and absorption of laser light by particulate matter. However, due to differences in particle characteristics, the noise distribution exhibits different features. In rainy weather, noise is dispersed, while in foggy weather, noise tends to form clumps or patches, and the attenuation of the point cloud is more significant. In foggy weather, suspended particles in the air (such as water droplets or ice crystals) scatter and absorb radar laser light, causing a rapid decrease in laser transmittance.

[0037] In determining radar weather results based on the number of point cloud noise points, the electronic control unit first determines a noise quantity comparison value by comparing the number of point cloud noise points within a first preset distance range with the number of point cloud noise points within a preset acquisition range. The first preset distance range is a pre-defined specific distance interval near the aircraft used to distinguish weather types from the perspective of point cloud noise quantity. Different first preset distance ranges correspond to different noise quantity comparison values, which are the ratio of the number of point cloud noise points within a specific distance range to the total number of point cloud noise points within the preset acquisition range. Specifically, taking a ratio of 0.3 between the diameter of the first preset distance range and the diameter of the preset acquisition range as an example, if the first preset distance range is a circular area of ​​0-30 meters and the preset acquisition range is a circular area of ​​0-100 meters, then the noise quantity comparison value is the ratio of the sum of the noise points in the first three point cloud annular intervals to the sum of the noise points in the ten point cloud annular intervals.

[0038] Next, the electronic control unit compares the noise count with a preset comparison threshold to determine whether the weather type is rainy or foggy. When the noise count is greater than the preset comparison threshold, it indicates that the noise within the preset sampling range is concentrated in the close-range area near the aircraft, and the radar weather result is determined to be foggy. When the noise count is less than or equal to the preset comparison threshold, it indicates that the noise within the preset sampling range is not concentrated in the close-range area near the aircraft, and the radar weather result is determined to be rainy.

[0039] When the radar weather report indicates foggy weather, the electronic control unit (ECU) determines the radar fog visibility in the radar weather report based on radar point cloud data from all point cloud acquisition points labeled as noise. The ECU first determines the noise depth value for each point cloud acquisition point based on its radar point cloud data. The noise depth value refers to the distance between the noise point and the radar sensor. Based on the noise depth values ​​of all noise points and a preset noise quantile threshold, the radar fog visibility can be determined. The preset noise quantile threshold is a pre-set critical value for the abrupt change in noise distribution caused by fog droplet scattering and absorption. It can be set to a default value or adjusted as needed, such as a default value of 31.7% quantile. In dense fog, the noise depth value will exhibit a normal distribution. A quantile is the value at a specific position after sorting a set of data from smallest to largest; for example, the median (50th quantile) divides all data into two halves. When the preset noise quantile threshold is 31.7%, after sorting all noise depth values ​​from smallest to largest, the noise depth value at the 31.7% position is taken as the radar visibility in foggy weather.

[0040] When the radar weather report indicates rainy weather, the electronic control unit (ECU) first determines the interval point cloud noise density of each point cloud annular interval based on the number of noise points and the volume of the annular interval. Next, based on a pre-built rainfall noise formula, the ECU converts the interval point cloud noise density of each point cloud annular interval within a second preset distance range to obtain the interval rainfall for that annular interval. The interval rainfall is a quantified value characterizing the degree of rainfall within the point cloud annular interval. Then, the interval rainfall of all point cloud annular intervals within the second preset distance range is weighted and summed to obtain the radar rainfall in the radar weather report. The second preset distance range is a pre-defined specific distance range far from the radar sensor used to assess the degree of rainy weather from the perspective of point cloud noise density. Specifically, taking a ratio of 0.8 between the ring width of the second preset distance range and the diameter of the preset acquisition range as an example, where the second preset distance range is a ring area of ​​20~100 meters and the preset acquisition range is a circular area of ​​0~100 meters, the point cloud noise density of the eight point cloud ring intervals (20~30 meters), (30~40 meters)...(90~100 meters) can be processed by formula conversion to obtain the interval rainfall corresponding to each point cloud ring interval. After weighted summation, the radar rainfall is obtained.

[0041] The rainfall noise formula is a quantitative conversion relationship between the number of noise points in an interval and the rainfall amount in that interval, established by analyzing empirical data (such as historical and experimental data). Specifically: ,in, Indicates rainfall amount, Indicates the noise density of the point cloud. and This represents an empirical constant (obtained by fitting empirical data). As shown in the rainfall noise formula, noise density and rainfall are exponentially related; the higher the noise density, the greater the rainfall. When establishing the rainfall noise formula, a common rainfall noise formula can be obtained by uniformly fitting the noise quantity and rainfall of all point cloud annular intervals, or a separate rainfall noise formula can be obtained by individually fitting the noise quantity and rainfall of each point cloud annular interval.

[0042] In addition, in practical applications, the radar sensor also collects radar point cloud data at a preset fixed frequency (such as 10Hz). The electronic control unit obtains a smoothed value of radar rainfall by taking the average of the radar rainfall at the current collection time and the radar rainfall in the past 10 scans within 1 second, and uses the smoothed value of radar rainfall as the radar rainfall in the radar weather results.

[0043] This embodiment compares the number of point cloud noise points within a first preset distance range with the number of noise points within a preset collection range, and sets a preset comparison threshold. It can accurately distinguish between foggy days and rainy days based on the differences in noise distribution, and determine the visibility in foggy days and the rainfall in rainy days through different quantitative analysis methods. By combining qualitative identification and quantitative analysis, the accuracy of sky recognition is improved.

[0044] S30. Determine the flight weather identification result of the aircraft based on the image weather result and the radar weather result.

[0045] Understandably, flight weather identification results refer to the final weather condition information obtained through analysis of the environment surrounding the aircraft. Flight weather identification results include the weather type and the corresponding intensity quantification parameters. For example, when the weather type in the flight weather identification results is rainy, it also includes the corresponding rainfall level. When the weather type in the flight weather identification results is foggy, it also includes the corresponding fog visibility. Furthermore, in practical application scenarios, other weather types may also be included; for example, the identification of snowy weather can adopt the rainy weather identification process described in this embodiment.

[0046] On the embedded GPU platform (NVIDIA Jetson Orin), the average total time for a single recognition is 85ms, meeting real-time requirements. The image sensor and radar sensor employ a 10Hz acquisition frequency and a processing latency of less than 100ms, ensuring that the method in this embodiment can respond to weather changes in real time. In tests with 1000 sets of input data, the accuracy rate of the image recognition method alone was 78.5%, the accuracy rate of the radar point cloud recognition method alone was 82.3%, and the accuracy rate of the weather recognition method of this invention reached 94.2%. Multi-sensor fusion enables redundancy verification, significantly improving the accuracy of weather recognition.

[0047] This embodiment acquires environmental images from at least one acquisition perspective of the aircraft. Based on all environmental images, it determines the image weather result. When the image weather result indicates rainy or foggy weather, it determines the number of point cloud noise points based on radar point cloud data from several point cloud acquisition points within a preset acquisition range, and determines the radar weather result based on the number of point cloud noise points. Finally, it determines the aircraft's flight weather identification result based on both the image weather result and the radar weather result. This embodiment combines image sensors and radar sensors to acquire environmental data for weather identification, avoiding the reliability issues of single-sensor weather identification. Furthermore, this embodiment uses a redundancy mechanism to perform different data analysis processes on environmental image data and radar point cloud data, and obtains the flight weather identification result through comparison and fusion of the analysis results. This achieves multi-dimensional cross-validation, enabling real-time and accurate identification of environmental weather conditions, improving the reliability and fault tolerance of weather identification, and ensuring the flight safety of the aircraft.

[0048] In one embodiment, step S30, namely determining the flight weather identification result of the aircraft based on the image weather result and the radar weather result, includes: S301. When both the image weather result and the radar weather result are foggy weather, the flight weather identification result is determined to be foggy weather, and the radar fog visibility in the radar weather result is determined to be the fog visibility in the flight weather identification result. S302. When both the image weather result and the radar weather result are rainy weather, the flight weather identification result is determined to be rainy weather, and the rainfall level in the flight weather identification result is determined based on the radar rainfall in the radar weather result. S303. When the image weather result is rainy weather and the radar weather result is foggy weather, the flight weather identification result is determined to be rainy weather, and the rainfall level in the flight weather identification result is determined to be light rain level. S304. When the image weather result is foggy weather and the radar weather result is rainy weather, the flight weather identification result is determined to be foggy weather.

[0049] Understandably, when the electronic control unit (ECU) comprehensively analyzes image weather results and radar weather results, it needs to compare whether the weather type identifications in the image weather results and radar weather results are consistent, and uses different rules to determine the flight weather identification result according to different comparison results. When both image weather results and radar weather results indicate foggy weather, the flight weather identification result is determined to be foggy weather, and the radar fog visibility in the radar weather results is determined to be the fog visibility in the flight weather identification result. When both image weather results and radar weather results indicate rainy weather, the flight weather identification result is determined to be rainy weather, and the rainfall level in the flight weather identification result is determined based on the radar rainfall amount in the radar weather results. The rainfall level is divided into different levels according to the amount of rainfall, including light rain level, moderate rain level, heavy rain level, and torrential rain level. The ECU compares the radar rainfall amount in the radar weather results with the pre-set rainfall level range, and determines the rainfall level based on the comparison result. Specifically, the radar rainfall ranges for light rain are 0.1–4.9 mm, moderate rain is 5.0–14.9 mm, heavy rain is 15.0–29.9 mm, and torrential rain is 30.0–69.9 mm. When the image weather result indicates rain and the radar weather result indicates fog, the flight weather identification result is determined to be rain, and the rainfall level in the flight weather identification result is determined to be light rain. When the image weather result indicates fog and the radar weather result indicates rain, the flight weather identification result is determined to be fog. In tests with 20 sets of input rainfall levels for each category, the rainfall level identification accuracy reached 86.3%.

[0050] This embodiment combines image weather results and radar weather results to avoid weather identification bias caused by a single data source, and reasonably determines visibility and rainfall levels in foggy weather. Furthermore, this embodiment addresses situations where image and radar identification results are inconsistent, ensuring a scientifically sound basis for decision-making under various weather conditions, reducing identification uncertainty caused by conflicts, and improving the aircraft's ability to cope with complex weather.

[0051] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0052] In one embodiment, a weather recognition device is provided, which corresponds one-to-one with the weather recognition methods described in the above embodiments. For example... Figure 2 As shown, the weather recognition device includes an image recognition module 10, a radar recognition module 20, and a comparison analysis module. Detailed descriptions of each functional module are as follows: Image recognition module 10 is used to acquire environmental images collected from at least one acquisition viewpoint of the aircraft, and to determine image weather results based on all said environmental images; The radar identification module 20 is used to determine the number of point cloud noise points based on radar point cloud data from several point cloud collection points within a preset collection range when the image weather result is rainy or foggy, and to determine the radar weather result based on the number of point cloud noise points. The comparison and analysis module 30 is used to determine the flight weather identification result of the aircraft based on the image weather result and the radar weather result.

[0053] In one embodiment, the image recognition module 10 includes: The initial weather result determination unit is used to perform image recognition processing on the environmental images collected from each of the collection perspectives using a preset weather classification model, so as to obtain the initial weather results for each of the collection perspectives. The weather smoothing result determination unit is used to smooth the initial weather result and the initial historical weather result of each acquisition perspective to obtain the weather smoothing result of each acquisition perspective; the initial historical weather result refers to the initial weather result of the acquisition perspective at multiple historical moments within a preset historical time period. A perspective weather result determination unit is used to determine the perspective weather result for each of the acquisition perspectives based on the weather smoothing result for each acquisition perspective. The image weather result determination unit is used to determine the image weather result based on the view weather results of all the acquisition viewpoints.

[0054] In one embodiment, the image recognition module 10 further includes: The type probability acquisition unit is used to acquire the predicted type probability corresponding to each preset weather type in the weather smoothing result for each acquisition perspective. The type Boolean value determination unit is used to determine the prediction type Boolean value corresponding to each preset weather type based on the prediction type probability and the preset type probability threshold corresponding to each preset weather type. The type Boolean value analysis unit is used to determine the view weather result of the acquisition view based on the prediction type Boolean value corresponding to all the preset weather types.

[0055] In one embodiment, the image recognition module 10 further includes: The image rainy weather determination unit is used to determine that the image weather result is rainy weather when the prediction type Boolean value corresponding to rainy weather in the weather result of at least one of the acquisition viewpoints is 1. The image fog weather determination unit is used to determine that the image weather result is fog weather when the prediction type Boolean value corresponding to rain weather in the view weather results of all the acquisition viewpoints is 0, and the prediction type Boolean value corresponding to fog weather in the view weather results of at least one of the acquisition viewpoints is 1.

[0056] In one embodiment, the radar identification module 20 includes: The flatness analysis unit is used to perform flatness analysis on the radar point cloud data of each point cloud acquisition point, as well as the radar point cloud data of a preset number of flat point cloud acquisition points adjacent to the point cloud acquisition point, and obtain the acquisition point label of the point cloud acquisition point based on the flatness analysis results. The interval noise number determination unit is used to divide a preset collection range into several point cloud annular intervals based on a preset interval distance, and to determine the number of all point cloud collection points labeled with noise labels in each point cloud annular interval as the interval noise number of each point cloud annular interval.

[0057] In one embodiment, the radar identification module 20 further includes: The noise number comparison unit is used to determine the noise number comparison value based on the number of point cloud noise within a first preset distance range and the number of point cloud noise within a preset acquisition range. The radar fog weather determination unit is used to determine the radar weather result as fog weather if the noise number comparison value is greater than a preset comparison threshold, and to determine the radar fog visibility in the radar weather result based on the radar point cloud data of all point cloud collection points whose collection point label is a noise label. The radar rain weather determination unit is used to determine the radar weather result as rainy weather if the noise number comparison value is less than or equal to a preset comparison threshold, and to determine the radar rainfall in the radar weather result based on the number of noise points in all point cloud ring intervals within a second preset distance range. The radar visibility in foggy weather is determined based on the noise depth values ​​of all point cloud acquisition points labeled as noise points and a preset noise quantile threshold. The noise depth value of each point cloud acquisition point is determined based on the radar point cloud data of that point cloud acquisition point. The radar rainfall is determined based on the interval rainfall of all point cloud annular intervals. The interval rainfall of each point cloud annular interval is calculated by converting the interval point cloud noise density of that point cloud annular interval. The interval point cloud noise density of each point cloud annular interval is determined based on the interval noise number and annular interval volume of that point cloud annular interval.

[0058] In one embodiment, the comparison analysis module 30 includes: The first flight weather recognition unit is used to determine the flight weather recognition result as foggy weather when both the image weather result and the radar weather result are foggy weather, and to determine the radar fog visibility in the radar weather result as the fog visibility in the flight weather recognition result. The second flight weather identification unit is used to determine the flight weather identification result as rainy weather when both the image weather result and the radar weather result are rainy weather, and to determine the rainfall level in the flight weather identification result based on the radar rainfall in the radar weather result; The third flight weather identification unit is used to determine the flight weather identification result as rainy weather when the image weather result is rainy weather and the radar weather result is foggy weather, and to determine the rainfall level in the flight weather identification result as light rain level. The fourth flight weather identification unit is used to determine the flight weather identification result as foggy weather when the image weather result is foggy weather and the radar weather result is rainy weather.

[0059] Specific limitations regarding the weather recognition device can be found in the limitations of the weather recognition method described above, and will not be repeated here. Each module in the aforementioned weather recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0060] In one embodiment, an aircraft is provided; please refer to [reference needed]. Figure 3 The illustrated electronic device includes a memory 301 and a processor 302. The memory 301 stores computer programs; the processor 302 executes the programs stored in the memory 301 to implement the weather identification method described in any embodiment of this application. The aircraft can be a low-altitude aircraft, which is an aircraft flying in airspace below 1000 meters above the ground, including fixed-wing aircraft, rotary-wing aircraft, airships, hot air balloons, and electric vertical takeoff and landing (eVTOL) aircraft.

[0061] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media stores computer-readable instructions, which, when executed by one or more processors, perform the following steps: Acquire environmental images from at least one acquisition viewpoint of the aircraft, and determine image weather results based on all said environmental images; When the image weather result is rainy or foggy, the number of point cloud noise points is determined based on radar point cloud data from several point cloud collection points within a preset collection range, and the radar weather result is determined based on the number of point cloud noise points. Based on the image weather results and the radar weather results, the flight weather identification result of the aircraft is determined.

[0062] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0063] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0064] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A weather identification method, characterized in that, include: Acquire environmental images from at least one acquisition viewpoint of the aircraft, and determine image weather results based on all said environmental images; When the image weather result is rainy or foggy, the number of point cloud noise points is determined based on radar point cloud data from several point cloud collection points within a preset collection range, and the radar weather result is determined based on the number of point cloud noise points. Based on the image weather results and the radar weather results, the flight weather identification result of the aircraft is determined.

2. The weather identification method as described in claim 1, characterized in that, The step of determining the image weather result based on all the environmental images includes: By using a preset weather classification model, image recognition processing is performed on the environmental images collected from each of the aforementioned collection perspectives to obtain the initial weather results for each of the aforementioned collection perspectives. The initial weather results and historical initial weather results of each acquisition perspective are smoothed to obtain the smoothed weather results of each acquisition perspective; the historical initial weather results refer to the initial weather results of the acquisition perspective at multiple historical moments within a preset historical time period. Based on the weather smoothing results of each acquisition perspective, determine the view weather results for each acquisition perspective; Based on the viewpoint weather results from all the aforementioned acquisition perspectives, determine the image weather results.

3. The weather identification method as described in claim 2, characterized in that, The step of determining the view weather result for each acquisition viewpoint based on the weather smoothing result for each acquisition viewpoint includes: For each of the aforementioned acquisition perspectives, obtain the prediction type probability corresponding to each preset weather type in the weather smoothing results; Based on the prediction type probability and the preset type probability threshold corresponding to each preset weather type, determine the prediction type Boolean value corresponding to each preset weather type; Based on the Boolean values ​​of the prediction types corresponding to all the preset weather types, the viewpoint weather result for that acquisition viewpoint is determined.

4. The weather identification method as described in claim 2, characterized in that, The perspective weather results include multiple preset weather types corresponding to prediction type Boolean values, and the preset weather types include rainy weather and foggy weather; The step of determining the image weather result based on the view weather results from all the acquired viewpoints includes: When the Boolean value of the prediction type corresponding to rainy weather is 1 in the weather results of at least one of the acquisition perspectives, the image weather result is determined to be rainy weather. When the Boolean value of the prediction type for rainy weather in all the weather results of the collected viewpoints is 0, and the Boolean value of the prediction type for foggy weather in at least one of the weather results of the collected viewpoints is 1, the image weather result is determined to be foggy weather.

5. The weather identification method as described in claim 1, characterized in that, The number of point cloud noise points includes the number of noise points in several annular intervals of the point cloud. The step of determining the number of noise points in the point cloud based on radar point cloud data from several point cloud collection points within a preset collection range includes: For each point cloud acquisition point, the radar point cloud data and the radar point cloud data of a preset number of adjacent point cloud acquisition points are subjected to flatness analysis. Based on the flatness analysis results, the acquisition point label of the point cloud acquisition point is obtained. The preset collection range is divided into several point cloud annular intervals based on the preset interval distance. The number of all point cloud collection points labeled as noise points in each point cloud annular interval is determined as the interval noise number of each point cloud annular interval.

6. The weather identification method as described in claim 5, characterized in that, The step of determining the radar weather result based on the number of point cloud noise points includes: The noise quantity comparison value is determined based on the number of point cloud noise points within the first preset distance range and the number of point cloud noise points within the preset acquisition range. If the noise count comparison value is greater than the preset comparison threshold, the radar weather result is determined to be foggy weather, and the radar fog visibility in the radar weather result is determined based on the radar point cloud data of all point cloud collection points whose collection point label is noise label. If the noise count comparison value is less than or equal to the preset comparison threshold, the radar weather result is determined to be rainy weather, and the radar rainfall in the radar weather result is determined based on the number of noise points in all point cloud ring intervals within the second preset distance range. The radar visibility in foggy weather is determined based on the noise depth values ​​of all point cloud acquisition points labeled as noise points and a preset noise quantile threshold. The noise depth value of each point cloud acquisition point is determined based on the radar point cloud data of that point cloud acquisition point. The radar rainfall is determined based on the interval rainfall of all point cloud annular intervals. The interval rainfall of each point cloud annular interval is calculated by converting the interval point cloud noise density of that point cloud annular interval. The interval point cloud noise density of each point cloud annular interval is determined based on the interval noise number and annular interval volume of that point cloud annular interval.

7. The weather identification method as described in claim 1, characterized in that, The step of determining the flight weather identification result of the aircraft based on the image weather result and the radar weather result includes: When both the image weather result and the radar weather result are foggy weather, the flight weather identification result is determined to be foggy weather, and the radar fog visibility in the radar weather result is determined to be the fog visibility in the flight weather identification result; When both the image weather result and the radar weather result are rainy, the flight weather identification result is determined to be rainy, and the rainfall level in the flight weather identification result is determined based on the radar rainfall in the radar weather result. When the image weather result is rainy weather and the radar weather result is foggy weather, the flight weather identification result is determined to be rainy weather, and the rainfall level in the flight weather identification result is determined to be light rain level; When the image weather result is foggy weather and the radar weather result is rainy weather, the flight weather identification result is determined to be foggy weather.

8. A weather recognition device, characterized in that, include: An image recognition module is used to acquire environmental images collected from at least one acquisition viewpoint of the aircraft, and to determine the image weather result based on all the environmental images; The radar identification module is used to determine the number of point cloud noise points based on radar point cloud data from several point cloud collection points within a preset collection range when the image weather result is rainy or foggy, and to determine the radar weather result based on the number of point cloud noise points. The comparison and analysis module is used to determine the flight weather identification result of the aircraft based on the image weather result and the radar weather result.

9. An aircraft, characterized in that, Including processor and memory, among which, Memory, used to store computer programs; A processor for executing a program stored in memory to implement the weather recognition method according to any one of claims 1-7.

10. A computer-readable storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors cause the weather identification method as described in any one of claims 1 to 7 to be performed.

Images