A Smart Fusion and Cleaning Method for Multi-Source Heterogeneous Data

Abstract

This invention discloses an intelligent fusion and cleaning method for multi-source heterogeneous data, relating to the field of intelligent data processing technology. The method includes: acquiring multi-source heterogeneous initial data; generating predicted trajectory data using the initial data; performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase; cleaning the first and second data according to preset first and second processing strategies respectively; and fusing the cleaning results of the first and second processing strategies to obtain a fused dataset. Its advantages include: solving the anomaly problems caused by differences in data sampling frequency, timestamp misalignment, and environmental interference during UAV flight; and improving data reliability and processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent data processing technology, and in particular to an intelligent fusion and cleaning method for multi-source heterogeneous data. Background Technology

[0002] With the widespread application of drone technology in military reconnaissance, logistics transportation, agricultural inspection, urban monitoring and other fields, flight control systems are becoming increasingly intelligent. More and more sensors are being integrated into drone platforms, including IMU, GPS, visual image sensors, lidar, barometers and so on. The multidimensional data collected by these sensors provides key support for drone path planning, attitude control, target recognition and obstacle avoidance decisions. However, due to the diversity of sensor types, heterogeneous data sources, and different communication protocols and sampling methods, these data have significant differences in sampling frequency, timestamp annotation, unit standards and other aspects, which brings great technical challenges to data fusion and cleaning.

[0003] Currently, commonly used data fusion and cleaning methods include Kalman filtering, complementary filtering, time window-based time-series alignment algorithms, and machine learning-based data anomaly identification and reconstruction methods. These technologies perform well in stable environments and in flight missions with stable environments and gradual state changes, effectively achieving synchronization, registration, and anomaly detection of multi-sensor data. However, in practical applications, UAV flight is characterized by high speed, nonlinearity, and drastic dynamic changes. Its attitude, trajectory, and environmental state may fluctuate rapidly in a short period of time. At the same time, to ensure power output, the battery needs to continuously discharge at high power during flight, causing the airframe temperature to rise rapidly. This leads to performance fluctuations in some temperature-sensitive sensors, further causing problems such as null values, abrupt changes, drift, or timestamp misalignment in the data, seriously affecting the accuracy and consistency of the data. If conventional methods are used to process this data, it is easy to encounter problems such as state estimation errors and decreased control accuracy, and even cause flight safety risks.

[0004] Therefore, an intelligent fusion and cleaning method for multi-source heterogeneous data is proposed. Summary of the Invention

[0005] In view of the above-mentioned prior art, this application is hereby filed. Embodiments of this application provide an intelligent fusion and cleaning method for multi-source heterogeneous data, which can improve the accuracy and reliability of multi-source heterogeneous data processing during UAV flight.

[0006] According to one aspect of this application, an intelligent fusion and cleaning method for multi-source heterogeneous data is provided, comprising: acquiring initial multi-source heterogeneous data within a target time period; generating predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly identification and synchronization correction based on attitude change features and image temporal consistency; and fusing the processing results of the first processing strategy and the second processing strategy to obtain a fused dataset.

[0007] According to another aspect of this application, an intelligent fusion and cleaning system for multi-source heterogeneous data is provided, comprising: a data acquisition module for acquiring initial multi-source heterogeneous data within a target time period; a trajectory prediction module for generating predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; a data alignment module for performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; a data classification module for dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; a first cleaning module for cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; a second cleaning module for cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly identification and synchronization correction based on attitude change features and image temporal consistency; and a data fusion module for fusing the processing results of the first and second processing strategies to obtain a fused dataset.

[0008] According to another aspect of this application, an electronic device is provided, including a memory and a processor, the memory being used to store computer-executable instructions, and the processor being used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method described above.

[0009] According to another aspect of this application, a computer storage medium is provided that stores computer-executable instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0010] Compared with the prior art, the intelligent fusion and cleaning method and system for multi-source heterogeneous data according to the embodiments of this application can realize the spatiotemporal alignment of multi-source heterogeneous data by predicting trajectory data, and adopt different cleaning strategies for the stable flight stage and the dynamic transformation stage respectively. It effectively solves the abnormal problems caused by data sampling frequency differences, timestamp misalignment and environmental interference during the flight of UAV, and has the advantages of improving data reliability and processing efficiency. Attached Figure Description

[0011] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is a flowchart of an intelligent fusion and cleaning method for multi-source heterogeneous data according to the present invention.

[0013] Figure 2 This is a flowchart illustrating the re-cleaning process of an intelligent fusion and cleaning method for multi-source heterogeneous data according to the present invention.

[0014] Figure 3 This is a block diagram of an intelligent fusion and cleaning system for multi-source heterogeneous data according to the present invention.

[0015] Figure 4 This is a block diagram of an electronic device according to the present invention. Detailed Implementation

[0016] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0017] Exemplary methods

[0018] like Figures 1-2As shown, an intelligent fusion and cleaning method for multi-source heterogeneous data according to an embodiment of this application includes: acquiring initial multi-source heterogeneous data within a target time period; generating predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly identification and synchronization correction based on attitude change features and image temporal consistency; and fusing the processing results of the first and second processing strategies to obtain a fused dataset.

[0019] Multi-source heterogeneous data refers to data from different sensors or subsystems that differ in data format, structure, sampling method, etc. Specifically, it can include data acquired by sensors such as inertial measurement units (IMU), global positioning systems (GPS), image acquisition devices, lidar, and barometric altimeters.

[0020] Among them, the flight trajectory prediction model refers to the algorithm model that predicts the future trajectory change trend of the UAV based on the historical flight status and sensor observation data. Specifically, it can be implemented using the LSTM model based on recurrent neural networks or the Transformer time series prediction model with attention mechanism, and is used to generate the benchmark reference data of the flight trajectory.

[0021] Spatiotemporal alignment refers to the process of unifying data from different sensors with different timestamps and spatial coordinate systems into a unified time base and spatial reference system in order to achieve synchronization and comparability of multi-source data.

[0022] Among them, the fusion of processing results refers to integrating the cleaned datasets from different flight phases to form a unified, continuous and consistent optimized fusion dataset, which can be achieved by using a time-axis-based sequence overlay and window stitching algorithm.

[0023] The core innovation of this application lies in addressing the characteristics of high speed, nonlinearity, and high dynamic change during UAV flight. By generating benchmark reference data through flight trajectory prediction, it achieves accurate spatiotemporal alignment of multi-source heterogeneous sensor data. Based on the division of flight state stages, it adopts differentiated data cleaning strategies, combining trajectory continuity constraints in the stable flight stage with attitude temporal consistency detection in the dynamic transformation stage. This effectively solves the problem of multi-source data anomalies caused by environmental factors such as high-power discharge, flight thermal disturbance, and signal interference, significantly improving the accuracy of data fusion and the reliability of flight behavior recognition.

[0024] The working process and principle of this application are as follows: First, multi-source heterogeneous initial data within the target time period is acquired. This data comes from different sensors and devices, including but not limited to information such as velocity, acceleration, angular velocity, and position. A pre-trained trajectory prediction model is used to generate predicted trajectory data within the target time period based on the initial data. This predicted trajectory data serves as a unified spatiotemporal reference framework for subsequent data alignment and processing. It is a unified temporal spatial trajectory constructed based on multimodal inputs, filling in gaps and interruptions in the original data. Simultaneously, it integrates spatial paths predicted from multiple modal signals, avoiding single-modal errors, and can serve as a "unified alignment reference" for each time point.

[0025] Then, based on the predicted trajectory data, the initial data is spatiotemporally aligned to obtain aligned data. During the spatiotemporal alignment process, the timestamps and spatial location information of each modality data are calibrated with the predicted trajectory, solving problems such as inconsistent sampling frequencies of different devices and timestamp drift.

[0026] The aligned data is then divided into two phases based on preset classification rules: the first phase (stable flight) and the second phase (dynamic transformation). This phased processing strategy fully considers the differences in flight characteristics and sensor data performance across different states during UAV flight. Because stable flight and dynamic transformations (such as turning, takeoff and landing, and sudden avoidance) differ significantly in aerodynamic state, sensor response, and data distribution patterns, a uniform data processing approach often fails to adequately address the characteristics of each phase, potentially leading to anomaly detection bias or data cleaning distortion. The first phase of data exhibits trajectory continuity and attitude stability, making it suitable for anomaly detection and compensation based on trajectory smoothness and state logic. The second phase, however, exhibits drastic changes in attitude angle and abrupt changes in image and inertial data features, making it more suitable for identification and correction based on attitude change rate and multimodal temporal consistency. This refined processing strategy based on flight phases not only improves the accuracy of data cleaning but also enhances the system's data adaptability and robustness in complex dynamic flight environments.

[0027] The first data is processed using anomaly detection and compensation based on flight trajectory continuity and flight control status feedback. Specifically, by analyzing whether the positional changes of the UAV's trajectory on the time axis during the stable flight phase are continuous and smooth, and combining this with state parameters such as altitude, acceleration, and thrust output provided by the flight control system, the system identifies any abrupt changes, drifts, or unreasonable jumps in the flight path. When the trajectory suddenly drifts or is interrupted, and the flight control data at the corresponding moment does not support the possibility of sudden acceleration or drastic attitude changes, the system determines that the data segment is abnormal. Then, combining the trajectory trends of the preceding and following moments, compensation and repair are performed using interpolation, smoothing filtering, or fitting methods based on physical dynamics models. This processing strategy targets the characteristics of the stable flight phase, identifying and correcting abnormal data points by analyzing the continuous changes in the trajectory and state feedback signals, thereby improving the accuracy and stability of the data during this phase.

[0028] The second data employs anomaly identification and synchronous correction based on attitude change characteristics and the consistency of image / LiDAR timing. Specifically, by analyzing the attitude angle change trend recorded by the inertial measurement unit during the UAV's dynamic flight maneuvers (such as turning, pitching, and hovering), the continuity and amplitude characteristics of these changes are extracted. This is then combined with the timing characteristics of synchronously acquired image sequences or LiDAR point cloud data, such as displacement / rotation behavior in visual keyframes and the consistency of background changes in image contours or radar echoes, to determine whether the sensor data remains consistent across time and state levels. When attitude angle data exhibits abrupt changes without a corresponding visual transformation, or when the image / point cloud indicates continuous attitude change while the inertial data is interrupted, it is identified as abnormal data. Synchronous correction is then performed on this segment of data based on visual / radar information, including realigning timestamps, compensating for missing attitude angles, or using image / point cloud-guided attitude change trends for fitting and interpolation, ensuring the consistency and integrity of multimodal data during the dynamic phase. This processing strategy addresses the complexity of flight dynamic transformation phases by fusing attitude sensor data with visual / radar information for cross-modal anomaly detection and time synchronization, effectively resolving data misalignment and information loss issues during high-speed maneuvers.

[0029] Finally, the processing results from the two stages are merged to obtain the final fused dataset. The fusion process takes into account the continuity between different stages, avoiding data gaps.

[0030] This phased, multi-strategy processing method solves the problem of fusion and cleaning of multi-source heterogeneous data during UAV flight, and improves the accuracy and consistency of the data.

[0031] The above-described solution addresses the technical challenge of fusing and cleaning multi-source heterogeneous data during UAV flight. By introducing a predicted trajectory as a unified spatiotemporal reference, it effectively resolves the issues of timestamp drift and spatial inconsistency between different devices. The phased processing strategy employs different anomaly detection and cleaning methods tailored to the characteristics of stable flight and dynamic transformation phases, improving the targeting and accuracy of data processing.

[0032] In some of the above-mentioned schemes in this application, spatiotemporal alignment of the initial data specifically includes: using the predicted trajectory data as a reference trajectory, performing reverse calibration on the timestamps and spatial location information of each modality data in the initial data to obtain aligned data that is spatiotemporally consistent with the trajectory data; wherein, the reverse calibration includes filling in the missing time points with differences, and performing temporal adjustment and spatial remapping on the offset data.

[0033] Among them, interpolation is used to generate supplementary data at missing time points through interpolation algorithms, such as linear interpolation or cubic spline interpolation; time series adjustment is used to nonlinearly stretch or compress the data sequence through dynamic time warping algorithms to eliminate phase differences on the time axis; spatial remapping uses coordinate transformation matrices to uniformly transform data from different sensor coordinate systems to the global coordinate system of the reference trajectory, such as three-dimensional spatial mapping through Euler angle rotation matrices or quaternion transformations.

[0034] Specifically, the predicted trajectory data is first used as the baseline time axis. For each data point in the initial data, the time difference between it and the most recent time point in the predicted trajectory data is calculated. If the time difference exceeds a preset threshold, it is determined to be a missing data point. For missing data points, a linear interpolation method is used to fill in the missing data.

[0035] Secondly, for data points with time offsets, a dynamic time warping algorithm is used for time series adjustment. The initial data sequence is aligned with the predicted trajectory data sequence to find the optimal time correspondence, and then the timestamps of the initial data are corrected according to the correspondence.

[0036] Finally, for data points with spatial offsets, a spatial remapping method is used for calibration. The deviation vector between the spatial coordinates in the initial data and the corresponding time points in the predicted trajectory data is calculated. Based on the deviation vector, the spatial coordinates of the initial data are translated and rotated to align them spatially with the predicted trajectory data.

[0037] Through the above technical solution, this application achieves accurate spatiotemporal alignment of multi-source heterogeneous data. By using predicted trajectory data as a reference to perform reverse calibration on the initial data, the problems of inconsistent sampling frequencies, inconsistent timestamps, and inconsistent spatial coordinate systems between different data sources are solved. Interpolation is used to fill in missing data points, ensuring data continuity and integrity. Through temporal adjustment and spatial remapping, spatiotemporal deviations caused by sensor delays or drift are corrected. This improves the accuracy of subsequent data analysis and processing, providing a reliable data foundation for intelligent monitoring and analysis of UAVs in flight.

[0038] While some of the solutions described above achieve data cleaning and optimization for different flight states of the UAV through stage-based classification processing, the transition from stable flight to high-speed maneuvering flight, or the return to stable flight after a dynamic transformation, is not an instantaneous abrupt change but rather exhibits a certain degree of dynamic continuity during actual flight. Directly splicing and fusing the data from the two divided stages during data processing can easily lead to problems such as trajectory abrupt changes, discontinuous attitude angles, unreasonable thrust changes, or non-conservation of system energy at the state transition boundaries. Direct fusion may affect the continuity of the fused data on the time axis and the consistency with the physical model, thereby reducing the overall data quality and the reliability of subsequent flight decision algorithms.

[0039] This application further proposes that before fusing the processing results of the first processing strategy and the second processing strategy, the process includes: obtaining the first time boundary and the second time boundary of the first data and the second data on the time axis; extracting the trajectory state parameters of the corresponding time points from the predicted trajectory data according to the first time boundary and the second time boundary; determining the first data and the second data in the time sequence position on the time axis as the previous stage and the second data as the next stage; establishing a state transition function between the previous stage and the next stage according to the trajectory state parameters; and performing continuous correction on the data corresponding to the next stage within the preset time period of the first time boundary / second time boundary according to the state transition function.

[0040] Among them, the state transition function adopts a polynomial fitting method, and cubic polynomial models are established for position, velocity and attitude angle respectively.

[0041] Through the above technical solution, this application achieves a smooth data transition between the stable flight phase and the dynamic transformation phase, avoiding the problem of data discontinuity during phase switching. By establishing a state transition function, the continuity of data near key time points is corrected, improving the temporal consistency and physical rationality of the data, and providing a more reliable data foundation for subsequent motion analysis and performance evaluation.

[0042] In some of the solutions described above in this application, although the processing of UAV flight data is achieved through multi-stage data segmentation and cleaning strategies, in practical applications, the cleaned data may still deviate from the identification results and predicted trajectories of key flight maneuvers due to environmental interference or sensor anomalies, affecting the accuracy of subsequent training analysis and safety assessment.

[0043] This application further proposes: using a pre-trained action recognition model to identify key flight actions within a target time period based on a fusion dataset; performing consistency analysis between the key flight actions and the motion trajectories at corresponding time points in the predicted trajectory data to obtain a deviation degree representing the degree of deviation between the two; determining whether the deviation degree exceeds a preset deviation threshold; if so, backtracking the first and / or second data based on the time period corresponding to the deviation degree; dynamically adjusting the cleaning parameters in the first and / or second processing strategies based on the deviation degree, and re-cleaning and compensating the backtracked data until the deviation degree is less than the preset deviation threshold.

[0044] The flight action recognition model employs an architecture combining convolutional neural networks (CNN) and long short-term memory networks (LSTM). It takes multimodal time-series sensor data from the fusion dataset as input and outputs flight action categories and corresponding timestamps. The consistency analysis includes the quantitative calculation of time deviation, spatial position deviation, and attitude angle deviation, and generates a comprehensive deviation degree through weighted summation. The dynamic adjustment strategy selects corresponding cleaning parameter correction methods for different flight stages. For example, it increases the window length for trajectory continuity detection during steady-state cruise and increases the tolerance threshold for attitude angle change rate during high-maneuver flight.

[0045] Specifically, the fused dataset is first processed using a pre-trained flight action recognition model to identify the time points when the drone completes key flight actions such as turning, acceleration maneuvers, and hovering. Then, these identification results are compared with the predicted trajectory data to calculate the deviation. For example, if the drone is identified as having completed an acceleration maneuver at a certain moment, but the predicted trajectory shows that the drone was still in a stable cruising state at that moment, a large deviation will occur.

[0046] Furthermore, the system will determine whether this deviation exceeds a preset threshold. If it does, it indicates a potential problem in the data processing. In this case, the system will revert to the time period in which the deviation occurred and re-examine the original data.

[0047] Therefore, based on the identified problems, the data cleaning strategy is dynamically adjusted. For example, for data in the jump phase, the interpolation completion is strengthened, or the criteria for outlier identification are adjusted. Then, the adjusted strategy is used to reprocess this data, including data completion and noise reduction.

[0048] This process is repeated until the consistency between the processed data and the predicted trajectory reaches the required level, i.e., the deviation is reduced to below a threshold. In this way, it can be ensured that the final data not only conforms to physical laws but also accurately reflects the actual performance of the drone.

[0049] In some of the schemes described above in this application, the consistency analysis specifically includes: extracting the recognition time point of the key flight maneuver and the theoretical time point corresponding to the key flight maneuver in the predicted trajectory data, and calculating the time deviation between the recognition time point and the theoretical time point; extracting the coordinates of the core spatial position point in the key flight maneuver recognition result at the recognition time point and the corresponding point coordinates in the predicted trajectory data, and calculating the Euclidean distance deviation between the core spatial position coordinates and the corresponding point coordinates; extracting the main body attitude angle in the key flight maneuver recognition result at the recognition time point and the corresponding attitude angle in the predicted trajectory data, calculating the angle difference vector between the main body attitude angle and the corresponding attitude angle, and calculating the L2 norm of the angle difference vector as the attitude angle deviation; and calculating the deviation degree based on the time deviation, spatial position deviation, and attitude angle deviation. :

[0050] ;

[0051] in , , These represent time deviation, spatial position deviation, and attitude angle deviation, respectively. , , These are the preset time deviation threshold, spatial position deviation threshold, and attitude angle deviation threshold, respectively. , , For the preset weighting coefficients, satisfy .

[0052] In some of the solutions described above in this application, the setting of the preset deviation threshold includes: acquiring environmental state data at the identification time point; extracting environmental impact factors from a preset environment-action-influence factor mapping table based on environmental state factors and key flight actions; extracting the standard deviation threshold corresponding to the key flight actions from a preset action-deviation threshold mapping table based on the key flight actions; and correcting the standard deviation threshold based on the environmental impact factors to obtain the preset deviation threshold.

[0053] The environmental status data includes at least one of temperature, light intensity, and battery level; the environment-action-influence factor mapping table is pre-established using experimental data, recording the quantitative factors of the influence of different environmental factors on specific action deviations; the action-deviation threshold mapping table is generated through training with historical data, storing the standard deviation thresholds corresponding to different action types, and the formula for correcting the standard deviation thresholds is: ;in, The standard deviation threshold, As an environmental impact factor, , This is the preset minimum deviation threshold.

[0054] Specifically, after acquiring environmental state data, the degree of deviation impact of the current environment on key flight maneuvers is determined by querying the environment-action-influence factor mapping table. For example, in low-temperature environments, sensors may experience data drift; in this case, the environmental influence factor is set to a positive value to increase the deviation threshold. In highly reflective environments, image data may be distorted; the corresponding factor is dynamically adjusted. The standard deviation threshold is extracted from the action-deviation threshold mapping table and dynamically adjusted according to a correction formula to ensure that the preset deviation threshold adapts to environmental changes while avoiding oversensitivity due to excessively low thresholds. For example, when... When the value is 0.2, the standard threshold is magnified to 1.2 times, but if the calculated result is lower than... Then adopt As a lower limit, the threshold is not excessively relaxed due to environmental factors. Therefore, under different flight environment conditions, the deviation threshold can dynamically adapt to the actual error range of sensor data, improving the robustness of anomaly detection.

[0055] Through the above technical solution, this application achieves dynamic adjustment of the deviation threshold based on environmental conditions and flight maneuver type, avoiding the problems of oversensitivity or overly lenient thresholds caused by fixed thresholds, and improving the accuracy and robustness of anomaly detection. Simultaneously, the introduction of a minimum threshold ensures basic data quality requirements. This method can better adapt to complex and changing environments, providing a reliable foundation for subsequent data processing and analysis.

[0056] Exemplary System

[0057] Figure 3The illustration depicts an intelligent fusion and cleaning system for multi-source heterogeneous data according to an embodiment of this application, comprising: a data acquisition module for acquiring initial multi-source heterogeneous data within a target time period; a trajectory prediction module for generating predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; a data alignment module for performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; a data classification module for dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to preset classification rules; a first cleaning module for cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; a second cleaning module for cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly identification and synchronization correction based on attitude change features and image temporal consistency; and a data fusion module for fusing the processing results of the first and second processing strategies to obtain a fused dataset.

[0058] In one example, the data alignment module performs spatiotemporal alignment of the initial data by using the predicted trajectory data as a reference trajectory and performing reverse calibration on the timestamps and spatial location information of each modality data in the initial data to obtain aligned data that is spatiotemporally consistent with the trajectory data. The reverse calibration includes filling in the missing time points with interpolation and performing temporal adjustment and spatial remapping on the offset data.

[0059] In one example, before fusing the processing results of the first and second processing strategies, the data fusion module further includes: obtaining the first and second adjacent time boundaries of the first and second data on the time axis; extracting trajectory state parameters at corresponding time points from the predicted trajectory data based on the first and second time boundaries; determining the data with the earlier temporal position on the time axis as the previous stage and the data with the later position as the next stage; establishing a state transition function between the previous and next stages based on the trajectory state parameters; and performing continuity correction on the data corresponding to the next stage within a preset time period of the first / second time boundary based on the state transition function.

[0060] In one example, a reprocessing module is also included: using a pre-trained action recognition model, key flight actions within a target time period are identified based on the fused dataset; consistency analysis is performed between the key flight actions and the motion trajectories at corresponding time points in the predicted trajectory data to obtain a deviation degree representing the degree of deviation between the two; it is determined whether the deviation degree exceeds a preset deviation threshold, and if so, the first and / or second data are backtracked based on the time period corresponding to the deviation degree; the cleaning parameters in the first and / or second processing strategies are dynamically adjusted based on the deviation degree, and the backtracked data is re-cleaned and compensated until the deviation degree is less than the preset deviation threshold.

[0061] In one example, the consistency analysis performed by the reprocessing module includes: extracting the recognition time point of the key flight maneuver and the corresponding theoretical time point in the predicted trajectory data, and calculating the time deviation between the recognition time point and the theoretical time point; extracting the core spatial position coordinates in the key flight maneuver recognition result at the recognition time point and the corresponding point coordinates in the predicted trajectory data, and calculating the Euclidean distance deviation between the core spatial position coordinates and the corresponding point coordinates; extracting the main body attitude angle in the key flight maneuver recognition result at the recognition time point and the corresponding attitude angle in the predicted trajectory data, calculating the angle difference vector between the main body attitude angle and the corresponding attitude angle, and calculating the L2 norm of the angle difference vector as the attitude angle deviation; and calculating the deviation degree based on the time deviation, spatial position deviation, and attitude angle deviation. :

[0062] ;

[0063] in , , These represent time deviation, spatial position deviation, and attitude angle deviation, respectively. , , These are the preset time deviation threshold, spatial position deviation threshold, and attitude angle deviation threshold, respectively. , , For the preset weighting coefficients, satisfy .

[0064] In one example, the setting of the preset deviation threshold in the reprocessing module includes: acquiring environmental state data at the identification time point; extracting environmental impact factors from a preset environment-action-influence factor mapping table based on environmental state factors and key flight actions; extracting the standard deviation threshold corresponding to the key flight actions from a preset action-deviation threshold mapping table based on the key flight actions; and correcting the standard deviation threshold based on the environmental impact factors to obtain the preset deviation threshold.

[0065] In one example, the formula used by the reprocessing module to correct the standard deviation threshold is:

[0066] ;

[0067] in, The standard deviation threshold, As an environmental impact factor, , This is the preset minimum deviation threshold.

[0068] Exemplary electronic devices

[0069] Figure 4 An electronic device according to an embodiment of this application is illustrated. The electronic device may be the mobile device itself, or a standalone device independent of it, which may communicate with the mobile device to receive collected input signals from it and send selected target driving behaviors to it.

[0070] Figure 4 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0071] like Figure 4 As shown, the electronic device includes one or more processors and memory.

[0072] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0073] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement the driving behavior decision-making methods of the various embodiments of this application described above, and / or other desired functions.

[0074] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0075] Of course, for the sake of simplicity, Figure 4Only some of the components of the electronic device relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device may include any other suitable components depending on the specific application.

[0076] Exemplary computer-readable media

[0077] Embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the driving behavior decision-making methods according to various embodiments of this application as described in the "Exemplary Methods" section above.

[0078] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0079] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0080] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0081] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0082] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0083] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for intelligent fusion and cleaning of multi-source heterogeneous data, characterized in that, The method comprises: acquiring initial data of multiple sources and different types in a target time period; generating predicted trajectory data in the target time period according to the initial data through a pre-trained trajectory prediction model; aligning the initial data in time and space according to the predicted trajectory data to obtain aligned data; dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; cleaning the first data according to a preset first processing strategy, the first processing strategy comprising abnormality detection and compensation processing based on trajectory continuity and acceleration stability; cleaning the second data according to a preset second processing strategy, the second processing strategy comprising abnormality identification and synchronization correction based on attitude change characteristics and image time sequence consistency; fusing the processing results of the first processing strategy and the second processing strategy to obtain a fused data set. 2.The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 1, characterized in that, The aligning the initial data in time and space comprises: performing reverse calibration on the timestamps and spatial position information of each modality data in the initial data by taking the predicted trajectory data as a reference trajectory to obtain aligned data consistent in time and space with the trajectory data; wherein the reverse calibration comprises difference filling for missing time points and time sequence adjustment and spatial remapping for offset data. 3.The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 1, characterized in that, Before the fusing the processing results of the first processing strategy and the second processing strategy, the method further comprises: acquiring adjacent first and second time boundaries of the first data and the second data on a time axis; extracting trajectory state parameters of corresponding time points from the predicted trajectory data according to the first and second time boundaries; determining the first data and the second data in front of the time sequence position on the time axis as a previous stage and determining the first data and the second data behind the time sequence position on the time axis as a next stage; establishing a state transition function between the previous stage and the next stage according to the trajectory state parameters; continuously correcting the data corresponding to the next stage within a preset time period at the first time boundary / second time boundary according to the state transition function.

4. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 1, characterized in that, The method further comprises: identifying key flight actions in the target time period according to the fused data set through a pre-trained action recognition model; performing consistency analysis on the key flight actions and the motion trajectory of the corresponding time points in the predicted trajectory data to obtain a deviation degree representing the deviation degree of the two; judging whether the deviation degree exceeds a preset deviation threshold, and if so, backtracking the first data and / or the second data according to the time period corresponding to the deviation degree; dynamically adjusting cleaning parameters in the first processing strategy and / or the second processing strategy based on the deviation degree, and re-cleaning and compensating the backtracked data until the deviation degree is less than the preset deviation threshold.

5. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 4, characterized in that, The consistency analysis comprises: extracting the identification time points of the key flight actions and the corresponding theoretical time points of the key flight actions in the predicted trajectory data, and calculating the time deviation between the identification time points and the theoretical time points; extracting a core spatial position point coordinate in the key flight action recognition result at the recognition time point and a corresponding point coordinate in the predicted trajectory data, and calculating an Euclidean distance deviation of the core spatial position point coordinate and the corresponding point coordinate; extracting a subject posture angle in the key flight action recognition result at the recognition time point and a corresponding posture angle in the predicted trajectory data, calculating an angle difference vector of the subject posture angle and the corresponding posture angle, and calculating an L2 norm of the angle difference vector as a posture angle deviation; calculating a degree of deviation from the time deviation, the spatial position deviation, and the attitude angle deviation : ； wherein , , respectively represent the time deviation, the spatial position deviation and the attitude angle deviation, , , respectively are preset time deviation threshold value, spatial position deviation threshold value and attitude angle deviation threshold value, , , is a preset weight coefficient, satisfying .

6. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 5, characterized in that, the preset deviation threshold value comprises: obtaining environment state data at the recognition time point; extracting an environment influence factor from a preset environment-action-influence factor mapping table according to the environment state data and the key flight action; extracting a standard deviation threshold value corresponding to the key flight action from a preset action-deviation threshold value mapping table according to the key flight action; correcting the standard deviation threshold value according to the environment influence factor to obtain the preset deviation threshold value.

7. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 6, characterized in that, the formula for correcting the standard deviation threshold value is: ； wherein, is the standard deviation threshold value, is the environmental impact factor, , is a preset minimum deviation threshold value.

8. A multi-source heterogeneous data-oriented intelligent fusion and cleaning system, characterized in that, comprises: a data acquisition module configured to acquire initial data of multiple sources and different structures in a target time period; a trajectory prediction module configured to generate predicted trajectory data in the target time period according to the initial data by using a pre-trained trajectory prediction model; a data alignment module configured to perform spatio-temporal alignment on the initial data according to the predicted trajectory data to obtain aligned data; a data classification module configured to divide the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; a first cleaning module configured to clean the first data according to a preset first processing strategy, the first processing strategy comprising abnormality detection and compensation processing based on trajectory continuity and acceleration stability; a second cleaning module configured to clean the second data according to a preset second processing strategy, the second processing strategy comprising abnormality identification and synchronous correction based on posture change characteristics and image time sequence consistency; a data fusion module configured to fuse processing results of the first processing strategy and the second processing strategy to obtain a fused data set. 9.An electronic device comprising a memory and a processor, the electronic device characterized by: The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the method according to any one of claims 1-7.

10. A computer storage medium having stored thereon computer- executable instructions, comprising: The computer executable instructions, when executed by the processor, implement the steps of the method according to any one of claims 1-7.

Images