Method for collecting multi-source heterogeneous data of aircraft safety data

Abstract

The application relates to the technical field of digital data processing, and discloses a kind of aircraft safety data multi-source heterogeneous data acquisition method, comprising: through single unmanned aerial vehicle platform, synchronous control multiple sensors concurrent execution physical obstacle three-dimensional survey, micro-meteorological wind field construction and space electromagnetic spectrum mapping operation, the three kinds of heterogeneous data collected are integrated into three-dimensional space-time data base with uniform time stamp in real time. The application ensures that the multi-source safety data has endogenous space-time consistency at the collection source, avoids information misplacement during traditional data fusion, so that the data base can truly reflect the real-time coupling state of multiple risks, and the exploration positioning ability of dynamic risk is also placed in the collection end, providing accurate and reliable decision basis for flight safety evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of aircraft safety data multi-source heterogeneous data acquisition method, belong to digital data processing technical field. BACKGROUND

[0002] To ensure the flight safety of low altitude airspace, it is usually necessary to collect and analyze data on key factors affecting flight, including physical obstacles, weather conditions and electromagnetic environment; The current situation of data collection and application in low altitude airspace is the demand for multiple types of low altitude airspace data. At present, the difficulties faced by data collection in low altitude airspace are weak data accuracy, separation of data collection mechanism at source level and complex data collection means. In the face of these difficulties, the problem of separation of data collection mechanism at source level is more prominent.

[0003] This separation of data collection mechanism at source level makes it difficult to unify various types of data in time reference and spatial position. When trying to integrate these data to assess the security situation, only the non-synchronous information of multiple independent risk factors at different times is obtained, which cannot accurately reflect the real environment state formed by the coupling effect of multiple risks at a specific moment for the aircraft.

[0004] Even if we try to improve the accuracy and update frequency of each single data, we cannot solve the fundamental problem of non-synchronous data. The reason is that flight safety risks are often triggered by multiple factors, such as obstacles in a static map, whose actual threat level may change due to real-time local wind shear. The analysis of such coupled risks is beyond the processing capacity of the existing technical framework based on non-synchronous data.

[0005] Specifically, the existing technology mainly has the following deficiencies:

[0006] 1. The data of different security dimensions are separated in the collection mechanism, and there is no unified time and space collection reference;

[0007] 2. There is an inherent time difference between data information and the real-time environment state faced by the aircraft, making it difficult to capture the instantaneous changes of risks;

[0008] 3. Due to the lack of synchronization and correlation, it is difficult to effectively analyze the coupling effect of obstacles, microclimate and electromagnetic interference and other multi-physical field risk factors.

[0009] Therefore, how to establish a new data collection method that can use a single mobile platform to synchronously collect multi-source heterogeneous safety data around the flight route environment at the moment of flight task execution, and integrate various types of data in time reference and spatial position data level, i.e. integrate into a unified data model with spatiotemporal consistency, has become a technical problem to be solved by the present application. SUMMARY

[0010] The application provides a kind of aircraft safety data multi-source heterogeneous data acquisition method, its main purpose is to solve the problem that the safety data in prior art is separated from source and acquisition mechanism, leading to the data cannot be synchronized in time and space dimensions, it is difficult to accurately assess the real-time coupling state of multiple risk factors.

[0011] To achieve the above object, the application provides a kind of aircraft safety data multi-source heterogeneous data acquisition method, including the following steps:

[0012] Step a, operate a UAV along the route, the UAV is equipped with optical perception equipment, wind speed and direction sensing equipment and electromagnetic monitoring equipment;

[0013] Step b, in the process of UAV flight, through the on-board processing unit synchronous control device, concurrent execution of the following data acquisition operation: real-time acquisition of the physical barrier information of the route environment, the acquisition operation combines the real-time image captured by the optical perception equipment and the pre-stored digital elevation model data, identifies the physical entity through an unsupervised target detection algorithm and automatically constructs a three-dimensional bounding box for the physical entity, generates a three-dimensional obstacle data subset with timestamp; real-time acquisition of the three-dimensional wind field information of the route environment, the acquisition operation obtains wind vector data containing horizontal component and vertical component through the wind speed and direction sensing equipment, constructs a micro-meteorological spatiotemporal data subset with timestamp; real-time acquisition of the spatial electromagnetic spectrum information of the route environment, the acquisition operation obtains multi-band electromagnetic field strength data through the electromagnetic monitoring equipment, constructs a spatiotemporal electromagnetic field data subset with timestamp; and align and integrate the three-dimensional obstacle data subset, micro-meteorological spatiotemporal data subset and spatiotemporal electromagnetic field data subset based on shared timestamp and spatial coordinates, generate a three-dimensional spatiotemporal data base with unified data specification.

[0014] Preferably, the operation of real-time acquisition of the physical barrier information of the route environment further comprises: for overhead cable type physical entities, repeated acquisition at multiple time points when physical form changes due to temperature or wind changes to obtain three-dimensional model data reflecting overhead cable sag changes; and for vegetation type physical entities, repeated acquisition in different growing seasons to obtain three-dimensional model data reflecting changes in vegetation canopy form, and incorporate the three-dimensional model data of overhead cables and vegetation into the three-dimensional obstacle data subset.

[0015] Preferably, the operation of collecting the three-dimensional wind field information of the flight route environment in real time further comprises: when the terrain curvature or the building density of a segment of the flight route exceeds a set threshold, increasing the spatial collection density and the time collection frequency of the wind vector data; and collecting the attitude data of the unmanned aerial vehicle itself during the execution of the collection operation in real time, and associating the attitude data of the unmanned aerial vehicle itself with the corresponding original wind vector data, so as to subsequently deduct the speed component introduced by the attitude of the unmanned aerial vehicle itself from the original wind vector data.

[0016] Preferably, the operation of collecting the spatial electromagnetic spectrum information of the flight route environment in real time further comprises: when an unknown interference signal is monitored, controlling the unmanned aerial vehicle to obtain the direction information of the unknown interference signal from at least two different positions; and based on an algorithm model deployed at the unmanned aerial vehicle end, using a triangulation method to solve the source position of the unknown interference signal, and calculating a certain position confidence of the source position , wherein , wherein is the minimum intersection distance of the direction pointing lines from the at least two different positions, is a pre-set reference distance, and the solved source position and the position confidence thereof are recorded in the time-space electromagnetic field data subset.

[0017] Preferably, the method further comprises the steps of obtaining and processing prior data before the unmanned aerial vehicle is operated to fly: collecting and integrating geographic information data, resource investigation data and planning control data in the flight route planning area, the geographic information data including a digital surface model (DSM), a digital orthographic image (DOM) and a three-dimensional model of oblique photography; and generating a restricted airspace data subset containing no-fly zones, restricted flight zones and temporary control zones based on the prior data, and fusing the same as background data and the three-dimensional time-space data base.

[0018] Preferably, the step of automatically constructing the three-dimensional bounding box further comprises: if the current collection frame is a sequence frame, applying a multi-target tracking algorithm to predict the three-dimensional bounding box of the current frame according to the three-dimensional bounding box generated in the previous frame; and fusing the predicted three-dimensional bounding box with the three-dimensional bounding box obtained by the unsupervised target detection algorithm to generate the final three-dimensional bounding box of the current frame.

[0019] Preferably, after the three-dimensional time-space data base is integrated and generated, the method further comprises the step of three-dimensionally visualizing the micro-meteorological time-space data subset: encoding the wind intensity grid data in the micro-meteorological time-space data subset into texture data; and using a graphic processing unit (GPU) to combine a reverse distance weighted interpolation algorithm to perform pixel-by-pixel sampling and shading rendering on the texture data to generate a wind field distribution map; and calculating and generating a batch of dynamic particle flow lines by using a multi-threading technology, so as to display the flow pattern of the wind field in the three-dimensional scene.

[0020] Preferably, the operation of collecting the spatial electromagnetic spectrum information of the route environment in real time further comprises: erecting a movable electromagnetic environment online monitoring system in advance at a substation or a key area of a high-voltage transmission line along the route; the online monitoring system is used to collect continuous change data of power frequency electric field and power frequency magnetic field, and the continuous change data are taken as ground reference to be integrated with the aerial electromagnetic field data collected by the unmanned aerial vehicle in the same area in time and space.

[0021] Preferably, the step of integrating to generate the three-dimensional space-time data base with unified data specification comprises: performing coordinate system conversion on the microclimate space-time data subset and the space-time electromagnetic field data subset to make them consistent with the geographic coordinate system adopted by the three-dimensional obstacle data subset; and attaching the microclimate information and the electromagnetic field strength information after the coordinate system conversion to the corresponding space grid or three-dimensional model in the three-dimensional space-time data base as attribute information.

[0022] Preferably, the raw data collected by the optical sensing device, the wind speed and direction sensing device and the electromagnetic monitoring device are all subjected to a data cleaning step by the on-board processing unit before generating respective data subsets, and the data cleaning step filters out invalid data points in the raw data due to sensor noise or communication anomaly according to preset data validity rules.

[0023] Compared with the prior art, the present application has the following beneficial effects:

[0024] 1. By concurrently performing the three operations of physical obstacle three-dimensional survey, microclimate wind field construction and spatial electromagnetic spectrum mapping on a single unmanned aerial vehicle platform, the three types of heterogeneous data collected are ensured to have endogenous consistency in time and space reference. In the prior art, these data are usually independently collected by different platforms at different times, and the inherent time and space misalignment limits the effectiveness of data fusion, making it difficult to accurately reflect the real coupling state of multiple risk factors at a specific moment. The present application generates a unified three-dimensional space-time data base by synchronous collection and integration, so that the risk assessment of the flight environment is no longer based on the superposition and deduction of static data, but is based on an instant snapshot of a dynamically changing multi-physical field environment, thereby providing data basis for subsequent route planning.

[0025] 2、The present application not only combines pre-stored geographic information and real-time optical perception in the obstacle data collection, but also introduces a repeated survey mechanism of multi-weather conditions and multi-season cycles for the two types of physical entities, overhead cables and vegetation, which dynamically change with the environment. Traditional three-dimensional real scene modeling often generates a static model at a certain moment, ignoring important safety variables such as cable sag due to temperature and wind load, and vegetation canopy shape changing with the season, which are crucial in actual flight. The method of the present application builds model data containing these dynamic change information, so that the final three-dimensional obstacle data subset goes beyond the static geometric description, and becomes a data model containing the dynamic behavior characteristics of a specific risk source. This processing method improves the fineness of flight safety assessment.

[0026] 3、When collecting spatial electromagnetic spectrum information, the present application not only records the field strength of unknown interference signals, but also uses the maneuverability of the unmanned aerial vehicle to obtain signal direction at different spatial positions, and performs real-time triangulation based on the airborne algorithm model. Ground-based monitoring or conventional aerial inspection often has slow positioning response or cannot locate when facing sudden or moving interference sources. The method of the present application changes the unmanned aerial vehicle from a passive signal receiver to an active interference source explorer, realizes rapid identification and spatial locking of unknown electromagnetic risk sources by positioning before data collection, and integrates this position information into the time-space electromagnetic field data subset, providing decision information containing the location coordinates of the interference source to avoid such safety hazards. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The figure is a schematic diagram of the system architecture of the present application.

[0028] Figure 2 The figure is a timing flow chart of the data collection method of the present application.

[0029] Figure 3 The figure is a comparison chart of the influence of data synchronization on system response and obstacle avoidance performance of the present application.

[0030] Figure 4 The figure is a particle streamline generation flow chart in the three-dimensional visualization of micro-meteorological data of the present application.

[0031] The purpose of the present application, functional characteristics and advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0032] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0033] The method provided by the present application is a kind of aircraft safety data multi-source heterogeneous data acquisition method, and the system architecture is to integrate and synchronously control multiple sensors through a single unmanned aerial vehicle platform, and to complete the spatio-temporal alignment and integration of heterogeneous data in real time at the source of data acquisition with the help of an on-board processing unit. The method process mainly includes three concurrent data acquisition jobs, corresponding to three-dimensional survey of physical obstacles, micro-meteorological wind field construction and spatial electromagnetic spectrum mapping, and a data integration step that fuses the results of the three jobs in real time, finally generating a three-dimensional spatio-temporal data base with a unified data specification. In order to cope with the spatio-temporal coupling and rapidly changing conditions of risk factors in low-altitude environment, this scheme shares a unified time stamp and spatial coordinate reference, so that the collected physical obstacle information, three-dimensional wind field information and spatial electromagnetic spectrum information have consistency in time and space at the source of collection, thereby providing a data basis that can reflect the real-time coupling state of multiple risks for subsequent flight safety evaluation.

[0034] Furthermore, the execution of the method begins with operating an unmanned aerial vehicle carrying optical perception equipment, wind speed and direction sensing equipment, and electromagnetic monitoring equipment to fly along a predetermined route. During the flight mission, a challenge is how to quantify physical entities whose form changes with the environment, such as the sag of overhead cables which changes due to temperature and wind load or the crown of vegetation which changes form with seasonal changes. To address this challenge, the physical obstacle information acquisition job of the present scheme adopts a multi-cycle repeated survey working method. Specifically, for overhead cable type physical entities, the system will preset multiple temperature or wind threshold values that trigger acquisition based on historical meteorological data during the mission planning stage. When the real-time meteorological data during flight meets these conditions, a targeted three-dimensional model acquisition is automatically performed to obtain three-dimensional model data under different sag states. Similarly, for vegetation type physical entities, the acquisition job will repeat the acquisition according to the preset seasonal cycle to obtain the form data of the crown in different growth stages. These dynamically acquired three-dimensional model data are then timestamped and integrated into the three-dimensional obstacle data subset, so that this data subset not only contains static geometric structures, but also records the form characteristics of specific risk sources under different conditions.

[0035] It can be understood that repeated collection of pictures of the three-dimensional environment can obtain three-dimensional model data reflecting the sag change of the overhead cable, and repeated collection in different growth seasons can obtain three-dimensional model data reflecting the change of the vegetation canopy shape, and the three-dimensional model data of the overhead cable and the vegetation are integrated into the three-dimensional obstacle data subset.

[0036] The three-dimensional obstacle data subset, the micro-meteorological spatiotemporal data subset, and the spatiotemporal electromagnetic field data subset can be aligned and integrated based on shared timestamps and spatial coordinates to generate a three-dimensional spatiotemporal data base with unified data specifications.

[0037] Meanwhile, to solve the problem that regional weather forecasting cannot reflect the local micro-meteorological environment around the flight route, the micro-meteorological spatiotemporal data subset construction operation of the method uses the airborne wind speed and direction sensing device to obtain three-dimensional wind vector data containing horizontal and vertical components in real time. Given that complex terrain or dense building groups are factors that cause local airflow turbulence, the collection operation adopts a dynamic sampling density adjustment strategy. Before the UAV flies, the system analyzes the flight path based on the pre-stored digital elevation model data, automatically identifies flight segments with terrain curvature or building density exceeding the preset threshold, and automatically increases the spatial collection density and temporal collection frequency of wind vector data when the UAV enters these flight segments. In addition, to avoid interference of the UAV's own motion on wind field measurement, the operation records the UAV's own body attitude data during the execution of the collection in real time, and performs a speed component deduction procedure, that is, uses the associated body attitude data to calculate the speed component introduced by the body motion at each sampling time, and deducts it from the original wind vector data, thereby obtaining the wind field information deducted from the body motion interference. The processed data is integrated into the micro-meteorological spatiotemporal data subset with timestamps.

[0038] In the electromagnetic environment data collection layer, considering that there are unknown or temporary mobile interference sources in the low-altitude airspace, which pose a threat to flight safety, the spatial electromagnetic spectrum information collection operation in this scheme is designed to be able to actively explore. When the electromagnetic monitoring device captures an unknown interference signal, the onboard processing unit triggers a triangulation procedure. The procedure first controls the UAV to obtain the direction information of the unknown interference signal from the current position, and then autonomously flies to another position to again obtain the direction information of the signal from the second different position. Based on the algorithm model deployed on the UAV, the source position of the unknown interference signal is calculated in real time using at least two direction pointing lines through the triangulation method. In order to quantify the reliability of the positioning result, the system also calculates a position confidence , which is determined as , where is the minimum intersection distance of the direction pointing lines from at least two different positions, and is a reference distance preset according to a typical operation scene, the calculation provides a dimensionless confidence index by normalizing the intersection error of the direction line, finally, the calculated source position and the corresponding position confidence are recorded in the time-space electromagnetic field data subset, realizing the identification and spatial locking of the unknown electromagnetic risk source.

[0039] Specifically, in the operation of collecting the spatial electromagnetic spectrum information of the flight route environment in real time, the spatial electromagnetic environment of the flight route environment is relatively complex, and part of the unknown interference signal can be collected in real time.

[0040] While the above three data collection operations continue, all raw data are subjected to a unified data cleaning step by the airborne processing unit before generating respective data subsets, which filters out invalid data points in the raw data due to sensor noise or communication abnormalities according to preset data validity rules. Subsequently, the cleaned three-dimensional obstacle data subset, micro-meteorological time-space data subset and time-space electromagnetic field data subset are aligned and integrated based on the shared unified time stamp and spatial coordinates, and the integration step specifically includes coordinate system conversion of the micro-meteorological time-space data subset and the time-space electromagnetic field data subset to make them consistent with the geographic coordinate system adopted by the three-dimensional obstacle data subset, and the micro-meteorological information and electromagnetic field strength information after coordinate conversion are attached to the corresponding spatial grid or three-dimensional model in the three-dimensional time-space data base as attribute information. Thus, a three-dimensional time-space data base with unified data specification is generated. As an application and presentation method of the generated data, the method further includes a step of three-dimensional visualization of the micro-meteorological time-space data subset after generating the three-dimensional time-space data base, which first encodes the wind intensity grid data in the micro-meteorological time-space data subset into texture data, and uses a graphic processing unit (GPU) to perform pixel-by-pixel sampling and shading rendering on the texture data by combining a inverse distance weighted interpolation algorithm, to generate a wind field distribution map. Further, a batch of dynamic particle flow lines are calculated and generated by a multi-thread technology to show the wind field flow pattern in the three-dimensional scene, so that the invisible wind field data is presented in an intuitive visual form.

[0041] Specifically, the micro-meteorological time-space data in the micro-meteorological time-space data subset can be converted to the geographic coordinate system adopted by the three-dimensional obstacle data subset, and the micro-meteorological information after coordinate system conversion is attached to the corresponding spatial grid or three-dimensional model in the three-dimensional time-space data base as attribute information.

[0042] The multi-band electromagnetic field strength data in the spatiotemporal electromagnetic field data subset can be made consistent with the geographic coordinate system used by the three-dimensional obstacle data subset through coordinate transformation. The electromagnetic field strength information after coordinate system transformation is then attached as attribute information to the corresponding spatial grid or three-dimensional model in the three-dimensional spatiotemporal data base.

[0043] For real-time acquired information on physical obstacles in the flight path environment, the unsupervised target detection specifically employs a density-based spatial clustering algorithm, DBSCAN. This algorithm segments physical entities by identifying density-connected regions in point cloud data, and its required neighborhood radius... Minimum nearest neighbors of the core object These two parameters are directly determined by the performance indicators of the onboard optical sensing equipment, namely... The value is taken as the theoretical minimum number of point clouds that can form the smallest identifiable obstacle (such as a single utility pole) at a typical operating distance. The value is then taken as the minimum safe distance that can distinguish two adjacent independent obstacles (such as two trees). After completing the point cloud cluster segmentation, the tracking and updating of the identified targets in the sequence frames is achieved by fusing the 3D bounding boxes predicted by the multi-target tracking algorithm with the 3D bounding boxes detected in the current frame. This fusion strategy adopts an intersection-union ratio (IU / U)-based approach. The weighted average, As a numerical value between 0 and 1, it is used to quantify the degree of overlap between the predicted bounding box and the detected bounding box, and is directly used as the weight. Substitute The formula calculates the parameters of the final bounding box, where... Represents the center point location, size, and orientation parameters, when When the value is below a preset lower threshold of 0.3, the weight... It is forced to be set to 1, meaning the detection results are fully adopted to reset the tracking.

[0044] The terrain curvature threshold, used to increase the density of wind vector data acquisition in complex terrain flight segments, has a calibration procedure that begins with acquiring digital elevation model data of the operational airspace and then gridding this data. Next, the Laplace operator value for each grid cell is calculated. This operator, as a mathematical tool for measuring the second derivative of the Earth's surface, effectively quantifies the severity of terrain undulation. Finally, the cumulative distribution function of all Laplace operator values ​​is calculated, and the value corresponding to its 90th percentile is taken as the terrain curvature threshold. Similarly, the reference distance used to calculate position reliability in space electromagnetic spectrum information acquisition... , which is derived from a field calibration procedure conducted in a test field with known position reference electromagnetic emission sources, by instructing the UAV to repeatedly perform triangulation at different baseline distances and recording the average positioning error and operational time consumption corresponding to each baseline distance, and finally selecting the baseline distance that can complete positioning within the preset time limit of 15 seconds and the average positioning error first reaches the convergence plateau as the calibration value of . In addition, for the timestamp deviation between the wind speed and direction sensing device and the UAV's own inertial measurement unit due to inconsistent data output frequency, the system aligns the time stamps by using linear interpolation method before performing speed component deduction, that is, according to the two closest inertial measurement unit speed readings before and after the wind vector data sampling time , the algorithm model deployed on the UAV calculates the accurate airframe speed vector at time , thereby ensuring the accuracy of the wind field data.

[0045] Example 1: In an application of UAV patrol inspection of high-voltage transmission corridors in mountainous areas, the UAV flies along a route close to the ridge. The terrain and meteorological records in this area show that sudden downdraft and crosswind shear are prone to occur in the afternoon. A large transformer substation and its surrounding area have complex known and unknown electromagnetic radiation environments. When the UAV enters the valley section, the three-dimensional wind field information collected by the wind speed and direction sensing device carried by the UAV shows that the vertical Z-axis wind vector component appears a continuous negative value, and the horizontal XY-axis component fluctuates more than the preset threshold value in a short time, which is identified by the onboard processing unit as a microburst event. At the same time, the real-time images captured by the optical sensing device combined with the pre-stored digital elevation model data are analyzed by an unsupervised target detection algorithm, and a three-dimensional bounding box of the power cable under the flight route is obtained, with the lowest point lower than the static sag lowest point recorded in the database. This result of the physical obstacle information collection operation verifies the immediate morphological influence of the airflow identified by the micro-meteorological wind field construction operation on physical entities.

[0046] When the UAV approaches the transformer substation area, the electromagnetic monitoring device captures a strong field interference signal of a non-recorded frequency band, which is strong enough to suppress the quality of the satellite navigation signal of the UAV. At this moment, the system does not switch to the backup inertial navigation, but automatically triggers the triangulation procedure by the onboard processing unit, while maintaining the tracking of the direction of the interference signal, the UAV is controlled to deviate from the current position to a second position outside the predetermined reference distance, and the direction lines of the interference signal are obtained from the two different positions. Immediately, the algorithm model deployed on the UAV uses the two direction information to calculate the source position of the interference signal and the position confidence close to 1, indicating that the positioning result has high reliability. The collection operation provides the unknown electromagnetic risk source coordinates for the unmanned aerial vehicle, and realizes the locking of the spatial position. Specifically, the spatial electromagnetic spectrum information of the real-time collection route environment is collected, and the electromagnetic monitoring equipment is used to collect multi-band electromagnetic field strength data. The collected electromagnetic field strength data is a kind of aircraft safety data, and the collected data is used to improve the safety and efficiency of flight.

[0047] Finally, in the on-board processing unit of the unmanned aerial vehicle, the cable dynamic sag reflected by the physical obstacle data subset, the local wind shear area identified by the micro-meteorological spatiotemporal data subset, and the unknown interference source coordinates calibrated by the spatiotemporal electromagnetic field data subset are integrated into a unified three-dimensional spatiotemporal data base based on shared timestamps and spatial coordinates. The data base constitutes a multi-physical field risk factor real-time coupling digital scene, based on which the flight control system replans a flight trajectory that ascends in the vertical direction to avoid dynamic cables and circumvents the influence range of known interference sources in the horizontal direction, thereby continuing to perform the inspection task under multiple concurrent risks. The implementation of this process is based on a working method at the data collection level, that is, from discrete and asynchronous surveying of each independent risk factor to instant and synchronous overall data snapshot of a dynamically changing multi-physical field environment on a single platform. The internal data consistency of the generated three-dimensional spatiotemporal data base is not obtained through post-processing and passive alignment, but is derived from the unity of the collection operation in the time and space reference.

[0048] Embodiment 2: In this embodiment, in order to objectively verify the effectiveness of the synchronous collection and real-time integration mechanism of the method of the present application in processing multi-source heterogeneous data in dealing with coupled risks relative to the traditional time-sharing and source-separated data fusion mode, a hardware-in-the-loop simulation test platform is built. The platform builds a digital twin environment, including a high-precision terrain, a dynamic weather field generator, a controllable electromagnetic interference source, and a power cable model based on a physical engine and whose sag can change in real time with wind load. The test is divided into two groups. Test group A adopts the multi-source heterogeneous data collection method of the aircraft safety data of the present application, and the physical obstacle, micro-meteorological and electromagnetic environment data are generated and integrated synchronously in the on-board processing unit, and the data timestamp difference is negligible. The control group B simulates the traditional data acquisition mode, and the three types of data are from different systems, and there is a artificially set time delay parameter to simulate the asynchrony of data arrival and fusion processing.

[0049] The test key parameter, the time delay parameter in the control group B of the data update frequency and the risk evolution rate. The main factors affecting the value of this parameter are the evolution speed of the key risk in the scenario and the response time of the aircraft. The technical trade-off of setting this parameter is that too large will cause the flight decision to lag behind the real state of the environment, while too small will not reflect the inherent delay defect of the discrete data system. Therefore, the decision rule of setting should cover the range from the ideal state to beyond the safety response window of the aircraft to test the failure boundary of the system. In this test, the value of is set to a set of discrete values, including 1 second, 3 seconds, and 5 seconds, for comparative analysis.

[0050] The test process is as follows. The UAV simulation model enters the digital twin environment with the same initial route. At the 10th second of flight, the dynamic weather field generator and the electromagnetic interference source are activated at the same time to simulate the coupling risk scenario of microburst and navigation signal interference. The system continuously records the planned route calculated by the flight controller according to the data base obtained, and the real-time minimum safety distance from the power cable model in the environment. Repeat the test multiple times, record the system response time and obstacle avoidance effect under two methods, and a set of representative data is shown in Table 1.

[0051] Table 1: Comparison of system response and obstacle avoidance performance under different data synchronization conditions.

[0052]

[0053] The test data shows that test group A completes the route re-planning within 0.2 seconds after the risk is activated and always maintains a safety distance of more than 15 meters. The response time of control group B is directly related to the data time delay . When is 3 seconds, the UAV eventually makes an avoidance action, but its minimum safety distance is below the engineering safety threshold, and when is 5 seconds, the UAV collides because it does not complete effective avoidance before the cable sag changes to the lowest point. The mechanism of this data trend is that the three-dimensional spatiotemporal data base generated by test group A has coupled data on the timestamp of the internal wind field change and the cable shape change. The flight controller can make decisions based on the fact that wind shear and cable sag reduction occur at the same time, while the controller of control group B has not updated the obstacle database when it receives the wind shear data. Its decision is based on an invalid environment model, resulting in a lag in the avoidance action.

[0054] Example 3: This example combines Figures 1 to 4 This paper describes the implementation of a method for acquiring multi-source heterogeneous aircraft safety data. For example... Figure 1 As shown in the diagram, this system revolves around the core low-altitude electromagnetic situation processing chain, logically divided into three stages: situation element acquisition, situation understanding, and 3D situation visualization. In the situation element acquisition stage, the system, based on the low-altitude electromagnetic environment input, performs radiation source identification and electromagnetic environment construction. This process is supported by a data support module containing a radiation source feature library, low-altitude environmental data, and mathematical models. Subsequently, in the situation understanding stage, the system performs electromagnetic effect analysis, electromagnetic situation assessment, and electromagnetic situation prediction, and conducts quantitative analysis based on an indicator system and behavioral models. Finally, all the processed information is output in a visualized form during the 3D situation visualization stage.

[0055] like Figure 2 As shown, the process begins with the operator issuing a flight mission start command to the UAV platform. The UAV platform then executes the initialization system, and the onboard processing unit issues commands to the optical sensing device, wind speed and direction sensor, and electromagnetic monitoring device to start obstacle monitoring, wind field monitoring, and spectrum monitoring, respectively. During the loop operation in flight, the three types of sensors transmit the collected real-time image data, wind vector data, and electromagnetic field strength data back to the onboard processing unit. The onboard processing unit performs data cleaning and spatiotemporal alignment operations on the received heterogeneous data, thereby integrating the three types of datasets and finally generating a unified spatiotemporal data base and updating it.

[0056] like Figure 3 As shown, the horizontal axis represents the distance from... to Data time delay The left vertical axis represents the route replanning time. The right-hand vertical axis represents the minimum safe distance. The solid line in the graph, representing route replanning times, shows an upward trend, indicating that replanning times increase with data delay. The delay increases linearly with data latency. The dashed line, representing the minimum safe distance, shows a decreasing trend, indicating that the minimum safe distance increases with data latency. The increase leads to a sharp decrease, when achieve When the minimum safe distance becomes negative, it means a collision failure has occurred.

[0057] like Figure 4As shown, the process begins with acquiring wind field grid data collected by a drone. It then splits into two parallel branches. The main computational process on the left first creates position, color transparency, and index memory, then enters a loop. This loop starts by randomly generating wind particles, then iterates through the particle list, checking if the age of each particle is greater than its lifespan. If not, it calculates the particle position for the next moment and updates the particle position memory, ultimately generating wind direction particle streamlines. The parallel branch on the right is used for transparency calculation. It creates a particle flow transparency calculation thread, calculates the particle flow transparency, and updates the particle transparency memory, providing support for the final visualization rendering.

[0058] Example 4: Before deploying the method of this invention in an operational airspace with complex mountainous and urban edge terrain, a system calibration and verification procedure was performed to determine the values ​​of a series of adaptive parameters to match the system performance with the specific environmental characteristics. The primary purpose of this procedure was to determine an objective terrain curvature threshold for the dynamic sampling density adjustment strategy, given that drastic terrain changes are a factor causing low-altitude micro-meteorological turbulence, and excessively high sampling frequencies would increase the load on the airborne processing unit. To perform this calibration, digital elevation model data covering the entire predetermined operational airspace was first acquired. The airborne processing unit then performed a gridded traversal of the dataset, calculated the terrain curvature value for each grid cell, and statistically analyzed the probability distribution of all curvature values. Based on this distribution histogram, the curvature threshold was set at the position corresponding to a cumulative distribution function value of 0.95, i.e., selecting the five percent of the most rugged terrain in the airspace as the flight segments where the data acquisition frequency needed to be increased. This statistical method directly correlates the definition of terrain complexity with the geographical characteristics of the operational area.

[0059] Secondly, regarding the calculation of location reliability in the acquisition of spatial electromagnetic spectrum information... Reference distance parameters A field calibration procedure was performed. This procedure was conducted in an open test area with a clean electromagnetic background, with a reference electromagnetic source at a known location positioned at the center. The UAV was instructed to perform triangulation around the source at a series of increasing reference distances, from 20 meters to 100 meters, in 10-meter increments. For each reference distance, the triangulation was repeated multiple times, and the average error between the calculated source position and the actual position, as well as the average time required to complete a single triangulation, were recorded. Finally, the reference distance that minimized the average positioning error while maintaining a positioning time below a preset upper limit of 15 seconds was selected as the reference distance for this operational environment. The calibration value.

[0060] Moreover, to correct the raw wind vector data acquired by the wind speed and direction sensor, a time stamp alignment step is performed before the velocity component introduced by the body motion is deducted. Given that the wind speed and direction sensor and the UAV's own inertial measurement unit have different data output frequencies, at any sampling moment , the system will acquire the wind vector reading at this moment, retrieve the two closest inertial measurement unit velocity readings before and after the moment , and calculate the precise velocity vector of the UAV body at the moment by linear interpolation, and then perform vector subtraction to improve the accuracy of the wind field data.

[0061] Finally, in the physical obstacle information acquisition, for the fusion of the predicted three-dimensional bounding box in the sequence frame and the three-dimensional bounding box obtained by the unsupervised target detection algorithm, a weighted average strategy based on the intersection over union is adopted. The onboard processing unit calculates the intersection over union value between the predicted box and the detected box, which is used as a confidence weight, and the final fusion result of the two bounding boxes, the center point position, size and orientation, etc. are obtained by weighted average of the predicted value and the detected value, and the weight is positively correlated with the value. If the value is lower than the preset lower limit of 0.3, it is considered that the tracking deviates, and the result of the unsupervised target detection is directly used as the output of the current frame. This procedure provides a clear quantitative rule for data fusion.

[0062] Embodiment 5: Before applying the method of the present application to specific route planning and flight operations, a procedure for constructing baseline data of route environment and electromagnetic spectrum is performed. The procedure integrates the geographic information data of no-fly zones and restricted flight zones officially released by the route planning area, and superimposes the personnel intensive area data provided by the local management department including schools and hospitals. For the temporary control area information, an interface is established with the airspace management system to automatically query and update the data at a preset time frequency of 12 hours, so as to generate a restricted airspace data subset covering all known fixed and temporary restricted areas. At the same time, the UAV performs a general survey flight of the electromagnetic environment in the planned route area, records the center frequency, bandwidth and field strength of all stable existing electromagnetic emission facilities such as communication base stations and broadcast television towers, and stores these information in a known emission source database. In subsequent formal flight, any signal that is not recorded in the database and has a signal strength exceeding the predetermined threshold of local background noise is defined as an unknown interference signal by the onboard processing unit.

[0063] To address the dynamic changes in physical entities such as overhead cables and vegetation, the system employs a repetitive data acquisition triggering mechanism based on preset rules to maintain the timeliness of the 3D obstacle data subset. For vegetation, leveraging the predictability of its growth cycle, the system automatically generates repetitive data acquisition tasks based on the phenological calendar of the work area, setting two time points each year: the early spring leaf unfolding stage and the peak summer growth period. For physical entities like overhead cables, which are affected by weather conditions, the system continuously monitors real-time weather data. When the difference between the ambient temperature and the temperature at the time of the last survey exceeds a certain temperature difference threshold, the system will trigger a repetitive data acquisition. Or the wind speed is consistently higher than the wind speed threshold. And after a predetermined time, the system will trigger a resurvey of the overhead cable targets in the affected flight segment, thereby achieving a combination of periodic and event-based updates to dynamic risks.

[0064] Example 6: To further clarify the implementation path of the specific algorithm in the method of this invention and the system's operation strategy under boundary conditions, the following detailed procedure is executed. Specifically, for the step of unsupervised target detection of the original real-world 3D point cloud data of non-sequence frames in the acquisition of physical obstacle information, a density-based spatial clustering algorithm, DBSCAN, is used. The neighborhood radius of this algorithm... Minimum nearest neighbors of the core object The parameters are pre-calibrated based on the point cloud density of the onboard optical sensing device at a typical operating distance. During algorithm execution, the input point cloud data is first clustered, and each independent point cloud cluster is identified as a physical entity. Then, the minimum volume bounding box that can completely cover the point cloud cluster is calculated, and this bounding box is output as the 3D bounding box of the physical entity.

[0065] To perform 3D visualization of a subset of micro-meteorological spatiotemporal data, a particle advection calculation method based on Euler integrals is employed to generate batch dynamic particle streamlines to represent wind field flow patterns. At the beginning of each rendering frame, particles are randomly scattered across the visible 3D wind field grid data. For each particle, its flow pattern in the next time frame is... Location From its current moment Location and the wind vector at its location They jointly determined that their relationship is as follows: At the same time, a fixed lifespan is set for each particle. When a particle moves out of the visible area or its lifespan ends, the particle is removed, and the visual density of the particle stream is maintained by re-seeding new particles in the visible area.

[0066] In addition, in order to deal with the abnormal situation of sensor failure in flight, the method is built-in with an online fault-tolerant mechanism. Taking the failure of wind speed and direction sensor as an example, when the airborne processing unit continuously receives invalid data points beyond the preset physical range or empty for a predetermined number of times, it is determined that the sensor is failed. The system then enters a safety mode, in which the micro-meteorological spatio-temporal data subset is marked as unavailable, the flight controller no longer dynamically adjusts the safety distance of obstacles according to real-time wind field information when planning the flight route, but uniformly increases a fixed safety redundancy to the entity safety boundary in all three-dimensional obstacle data subsets, and simultaneously sends alarm information and the current system running state to the ground station.

[0067] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0068] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. An aircraft safety data multi-source heterogeneous data collection method, characterized in that, The method comprises the following steps: Step a, operating a UAV to fly along a flight route, the UAV carrying an optical perception device, a wind speed and direction sensing device, and an electromagnetic monitoring device; the flight safety data comprising a three-dimensional obstacle data subset, a micro-meteorological spatiotemporal data subset, and a spatiotemporal electromagnetic field data subset; Step b, during the flight of the UAV, synchronously controlling the devices by the on-board processing unit, and concurrently performing the following data acquisition operations: acquiring real-time physical obstacle information of the environment along the flight route, the acquisition operation combining real-time images captured by the optical perception device and pre-stored digital elevation model data, identifying physical entities by an unsupervised target detection algorithm and automatically constructing three-dimensional bounding boxes for the physical entities, and generating a three-dimensional obstacle data subset with time stamps; acquiring real-time three-dimensional wind field information of the environment along the flight route, the acquisition operation obtaining wind vector data comprising horizontal and vertical components by the wind speed and direction sensing device, and constructing a micro-meteorological spatiotemporal data subset with time stamps; in a flight segment where the terrain curvature or building density exceeds a set threshold, increasing the spatial acquisition density and temporal acquisition frequency of the wind vector data; and acquiring real-time UAV body attitude data during the acquisition operation, and correlating the body attitude data with the corresponding original wind vector data for subsequent deduction of the speed component introduced by the body motion from the original wind vector data; acquiring real-time spatial electromagnetic spectrum information of the environment along the flight route, the acquisition operation obtaining multi-band electromagnetic field strength data by the electromagnetic monitoring device, and constructing a spatiotemporal electromagnetic field data subset with time stamps; and aligning and integrating the three-dimensional obstacle data subset, the micro-meteorological spatiotemporal data subset, and the spatiotemporal electromagnetic field data subset based on shared time stamps and spatial coordinates, and generating a three-dimensional spatiotemporal data base with unified data specifications.

2. The method of claim 1, wherein, The operation of acquiring real-time physical obstacle information of the environment along the flight route further comprises: for overhead cable physical entities, repeatedly acquiring at multiple time points when the physical form changes due to temperature or wind changes to obtain three-dimensional model data reflecting the change in overhead cable sag; and for vegetation physical entities, repeatedly acquiring in different growing seasons to obtain three-dimensional model data reflecting the change in vegetation canopy form, and incorporating the three-dimensional model data of overhead cables and vegetation into the three-dimensional obstacle data subset.

3. The method of claim 1, wherein, The operation of collecting the spatial electromagnetic spectrum information of the route environment in real time further comprises: when the unknown interference signal is monitored, the unmanned aerial vehicle is controlled to obtain the direction information of the unknown interference signal from at least two different positions; and based on the algorithm model deployed at the unmanned aerial vehicle end, the source position of the unknown interference signal is calculated by using the triangulation method, and a certain position reliability of the source position is calculated wherein wherein, is the minimum intersection distance of the direction pointing lines from the at least two different positions, is a preset reference distance, and the calculated source position and the position reliability thereof are recorded in the space-time electromagnetic field data subset.

4. The method of claim 1, wherein, The method further comprises, before operating the UAV to fly, the steps of acquiring and processing prior data: collecting and integrating geographic information data, resource survey data, and planning control data within the flight route planning area, the geographic information data comprising a digital surface model (DSM), a digital orthographic image (DOM), and a tilt photography three-dimensional model; and generating a restricted airspace data subset comprising no-fly zones, restricted flight zones, and temporary control zones based on the prior data, and fusing the same as background data with the three-dimensional spatiotemporal data base.

5. The method of claim 1, wherein, The step of automatically constructing the three-dimensional bounding box further comprises: if the current acquisition frame is a sequence frame, applying a multi-target tracking algorithm to predict the three-dimensional bounding box of the current frame according to the three-dimensional bounding box generated in the previous frame; and fusing the predicted three-dimensional bounding box with the three-dimensional bounding box obtained by the unsupervised target detection algorithm to generate the final three-dimensional bounding box of the current frame.

6. The aircraft safety data multi-source heterogeneous data collection method of claim 1, wherein, After the three-dimensional spatio-temporal data base is generated, the method further comprises the step of three-dimensional visualization of the micro-meteorological spatio-temporal data subset: encoding the wind intensity grid data in the micro-meteorological spatio-temporal data subset as texture data; and using a graphic processing unit (GPU) to combine the inverse distance weighted interpolation algorithm to perform pixel-by-pixel sampling and shading rendering on the texture data to generate a wind field distribution map; and calculating and generating a batch of dynamic particle flow lines by using a multi-thread technology to show the flow pattern of the wind field in the three-dimensional scene.

7. The method of claim 3, wherein, The operation of collecting the spatial electromagnetic spectrum information of the flight route environment in real time further comprises: pre-installing a mobile electromagnetic environment online monitoring system at a substation or a high-voltage transmission line key area along the flight route; the online monitoring system is used to collect continuous change data of power frequency electric field and power frequency magnetic field, and the continuous change data is used as a ground reference to be spatio-temporally aligned and integrated with the air electromagnetic field data collected by the unmanned aerial vehicle in the same area.

8. The aircraft safety data multi-source heterogeneous data collection method of claim 1, wherein, The step of integrating the three-dimensional spatio-temporal data base generated according to the unified data specification comprises: performing coordinate system conversion on the micro-meteorological spatio-temporal data subset and the spatio-temporal electromagnetic field data subset to make them consistent with the geographic coordinate system used by the three-dimensional obstacle data subset; and attaching the micro-meteorological information and electromagnetic field strength information converted by the coordinate system conversion as attribute information to the corresponding spatial grid or three-dimensional model in the three-dimensional spatio-temporal data base.

9. The aircraft safety data multi-source heterogeneous data collection method of claim 1, wherein, The raw data collected by the optical perception device, the wind speed and direction sensing device and the electromagnetic monitoring device are subjected to a data cleaning step by the on-board processing unit before the respective data subsets are generated, and the data cleaning step filters out invalid data points in the raw data caused by sensor noise or communication abnormalities according to preset data validity rules.

Images