Intelligent fusion and cleaning method for multi-source heterogeneous data

Abstract

The invention discloses an intelligent fusion and cleaning method for multi-source heterogeneous data, and relates to the technical field of data intelligent processing, and the method comprises the steps: obtaining multi-source heterogeneous initial data, generating prediction trajectory data through the initial data, carrying out the space-time alignment of the initial data according to the prediction trajectory data, obtaining alignment data, and carrying out the fusion and cleaning of the multi-source heterogeneous data. Dividing the aligned data into first data corresponding to a stationary flight stage and second data corresponding to a dynamic transformation stage, cleaning the first data and the second data according to a preset first processing strategy and a preset second processing strategy, and fusing cleaning results of the first processing strategy and the second processing strategy to obtain a fused data set; the method has the beneficial effects that the abnormal problems caused by data sampling frequency difference, timestamp dislocation and environmental interference in the flight process of the unmanned aerial vehicle are solved, and the method has the advantages of improving data reliability and processing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent data processing, and in particular to an intelligent fusion and cleaning method for multi-source heterogeneous data. Background Art

[0002] With the widespread application of drone technology in military reconnaissance, logistics and transportation, agricultural inspection, urban monitoring and other fields, flight control systems are becoming increasingly intelligent, and more and more sensors are being integrated into drone platforms, including IMUs, GPS, visual image sensors, lidars, barometers, etc. The multi-dimensional data collected by these sensors provides key support for drone path planning, attitude control, target recognition and obstacle avoidance decisions. However, due to factors such as the diversity of sensor types, heterogeneous data sources, different communication protocols and sampling methods, these data have significant differences in sampling frequency, timestamp annotation, unit standards, etc., which brings great technical challenges to data fusion and cleaning.

[0003] Currently, commonly used data fusion and cleaning methods include Kalman filtering, complementary filtering, time series alignment algorithms based on time windows, and data anomaly identification and reconstruction methods based on machine learning. These technologies perform well in stable environments and in flight missions with stable environments and gentle state changes. They can effectively realize the synchronization, alignment and anomaly detection of multi-sensor data. However, in actual applications, UAV flight is characterized by high speed, nonlinearity and drastic dynamic changes. Its attitude, trajectory and environmental state may fluctuate rapidly in a short period of time. At the same time, in order to ensure power output, the battery needs to discharge continuously at high power during flight, causing the body temperature to rise rapidly, thereby causing performance fluctuations of some temperature-sensitive sensors, further causing problems such as null values, mutations, drift or timestamp misalignment of the data, seriously affecting the accuracy and consistency of the data. If conventional methods are used to process these data, state estimation errors and reduced control accuracy are likely to occur, and even cause flight safety risks.

[0004] Therefore, an intelligent fusion and cleaning method for multi-source heterogeneous data is proposed. Summary of the Invention

[0005] In view of the above-mentioned existing technical conditions, this application is proposed. The embodiments of this application provide an intelligent fusion and cleaning method for multi-source heterogeneous data, which can improve the accuracy and reliability of multi-source heterogeneous data processing during drone flight.

[0006] According to one aspect of the present application, an intelligent fusion and cleaning method for multi-source heterogeneous data is provided, comprising: acquiring multi-source heterogeneous initial data within a target time period; generating predicted trajectory data within the target time period based on the initial data through a pre-trained trajectory prediction model; performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to preset classification rules; cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly recognition and synchronous correction based on posture change characteristics and image temporal consistency; and fusing the processing results of the first processing strategy and the second processing strategy to obtain a fused data set.

[0007] According to another aspect of the present application, an intelligent fusion and cleaning system for multi-source heterogeneous data is provided, including: a data acquisition module for acquiring multi-source heterogeneous initial data within a target time period; a trajectory prediction module for generating predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; a data alignment module for performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; a data classification module for dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to preset classification rules; a first cleaning module for cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; a second cleaning module for cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly recognition and synchronous correction based on posture change characteristics and image time series consistency; and a data fusion module for fusing the processing results of the first processing strategy and the second processing strategy to obtain a fused data set.

[0008] According to another aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which implement the steps of the above-described method when executed by the processor.

[0009] According to another aspect of the present application, a computer storage medium is provided, on which computer executable instructions are stored. When the computer executable instructions are executed by a processor, the steps of the above method are implemented.

[0010] Compared with the existing technology, the intelligent fusion and cleaning method and system for multi-source heterogeneous data according to the embodiment of the present application can achieve spatiotemporal alignment of multi-source heterogeneous data by predicting trajectory data, and adopt different cleaning strategies for the stable flight phase and the dynamic transformation phase respectively, effectively solving the anomaly problems caused by data sampling frequency differences, timestamp misalignment and environmental interference during the flight of the drone, and has the advantages of improving data reliability and processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is a flow chart of an intelligent fusion and cleaning method for multi-source heterogeneous data of the present invention.

[0013] Figure 2 This is a re-cleaning flow chart of an intelligent fusion and cleaning method for multi-source heterogeneous data according to the present invention.

[0014] Figure 3 This is a block diagram of an intelligent fusion and cleaning system for multi-source heterogeneous data according to the present invention.

[0015] Figure 4 The present invention is a block diagram of an electronic device. DETAILED DESCRIPTION

[0016] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0017] Exemplary Methods

[0018] like Figure 1~Figure 2As shown, according to an embodiment of the present application, an intelligent fusion and cleaning method for multi-source heterogeneous data includes: obtaining multi-source heterogeneous initial data within a target time period; generating predicted trajectory data within the target time period based on the initial data through a pre-trained trajectory prediction model; performing spatiotemporal alignment on the initial data according to the predicted trajectory data to obtain aligned data; dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly recognition and synchronous correction based on posture change characteristics and image time series consistency; fusing the processing results of the first processing strategy and the second processing strategy to obtain a fused data set.

[0019] Among them, multi-source heterogeneous data refers to data from different sensors or subsystems, and has differences in data format, structure, sampling method, etc. Specifically, it may include data obtained by sensors such as inertial measurement unit (IMU), global positioning system (GPS), image acquisition device, lidar, and barometric altimeter.

[0020] Among them, the flight trajectory prediction model refers to an algorithm model that predicts the future trajectory change trend of the UAV based on its historical flight status and sensor observation data. It can be implemented using an LSTM model based on a recurrent neural network or a Transformer time series prediction model using an attention mechanism to generate benchmark reference data for the flight trajectory.

[0021] Among them, spatiotemporal alignment refers to the process of unifying data from different sensors with different timestamps and spatial coordinate systems into a unified time base and spatial reference system to achieve synchronization and comparability of multi-source data.

[0022] Among them, the fusion of processing results refers to the integration of cleaned data sets in different flight phases to form a unified, continuous and consistency-optimized fusion data set, which can be achieved by using a time-axis-based sequence superposition and window splicing algorithm.

[0023] The core innovation of this application lies in the fact that, in view of the high-speed, nonlinear and highly dynamic characteristics of UAV flight, it generates benchmark reference data through flight trajectory prediction to achieve accurate spatiotemporal alignment of multi-source heterogeneous sensor data, and adopts differentiated data cleaning strategies based on the flight state stage division. It combines the trajectory continuity constraint in the stable flight stage with the attitude timing consistency detection in the dynamic transformation stage to effectively solve the multi-source data anomaly problems caused by environmental factors such as high-power discharge, flight thermal disturbance, and signal interference, and significantly improves the accuracy of data fusion and the reliability of flight behavior recognition.

[0024] The working process and principle of this application are as follows: first, multi-source heterogeneous initial data within the target time period is obtained. These data come from different sensors and devices, including but not limited to speed, acceleration, angular velocity, position and other information. Through the pre-trained trajectory prediction model, the predicted trajectory data within the target time period is generated according to the initial data. The predicted trajectory data serves as a unified spatiotemporal reference framework for subsequent data alignment and processing. It is a unified temporal spatial trajectory constructed based on multimodal input, which makes up for the omissions and interruptions in the original data. At the same time, it integrates the spatial path predicted by multiple modal signals to avoid single modal errors, and can be used as a "unified alignment reference" at each time point.

[0025] The initial data is then spatially and temporally aligned based on the predicted trajectory data to produce aligned data. This alignment process aligns the timestamps and spatial position information of each modal data with the predicted trajectory, resolving issues such as inconsistent sampling frequencies and timestamp drift across different devices.

[0026] Next, the aligned data is divided into the first data from the stable flight phase and the second data from the dynamic transformation phase according to preset classification rules. This phased processing strategy fully accounts for the differences in flight characteristics and sensor data performance between different UAV flight phases. Since stable flight and dynamic transformations (such as turning, takeoff and landing, and sudden avoidance) exhibit significant differences in aerodynamic state, sensor response, and data distribution patterns, a unified data processing approach often fails to account for the characteristics of each phase, potentially leading to anomaly detection bias and cleaning distortion. The first data after division exhibits trajectory continuity and attitude stability, making it suitable for anomaly detection and compensation based on trajectory smoothness and state logic. The second data, however, exhibits dramatic changes in attitude angle and sudden changes in image and inertial data features, making it more suitable for identification and correction based on attitude change rate and multimodal temporal consistency. This refined processing strategy, based on flight phases, not only improves cleaning accuracy but also enhances the system's data adaptability and robustness in complex dynamic flight environments.

[0027] The first data is processed using anomaly detection and compensation based on the continuity of the flight trajectory and flight control state feedback. Specifically, by analyzing whether the position changes of the drone's trajectory on the time axis during the stable flight phase are continuous and smooth, combined with the state parameters such as altitude, acceleration and thrust output provided by the flight control system, it is identified whether there are abnormal conditions such as mutations, drifts or unreasonable jumps in the flight path. When the trajectory suddenly drifts or is interrupted, and the flight control data at the corresponding moment does not support the possibility of sudden acceleration or drastic attitude changes, the system will determine that this segment of data is abnormal, and combined with the trajectory trends of the previous and next moments, it uses interpolation, smoothing filtering or fitting based on physical dynamics models to perform compensation and repair. This processing strategy targets the characteristics of the stable flight phase, and by analyzing the continuity changes of the trajectory and the state feedback signal, it identifies and corrects abnormal data points, thereby improving the accuracy and stability of the data in this phase.

[0028] The second data is subjected to anomaly identification and synchronous correction based on attitude change characteristics and image / lidar temporal consistency. Specifically, by analyzing the attitude angle change trends recorded by the inertial measurement unit during dynamic flight maneuvers (such as turning, pitching, and hovering transitions), the continuity and amplitude characteristics of these changes are extracted. Combined with temporal change characteristics in the simultaneously acquired image sequences or lidar point cloud data, such as the displacement / rotation behavior of visual keyframes and the consistency of background changes in image outlines or radar echoes, the consistency of the sensor data at the time and state levels is determined. When the attitude angle data exhibits an abnormal sudden change without a corresponding change in the visual information, or when the image / point cloud indicates a continuous attitude change while the inertial data is interrupted, this data is identified as abnormal. Synchronous correction is then performed on this data segment based on the visual / radar information. This includes realigning timestamps, compensating for missing attitude angles, or fitting and interpolating the attitude change trends guided by the image / point cloud, ensuring the consistency and integrity of the multimodal data during the dynamic phase. This processing strategy targets the complexity of the dynamic transformation phase of flight, integrating attitude sensor data with vision / radar information for cross-modal anomaly detection and time synchronization, effectively solving the problems of data misalignment and information loss during high-speed maneuvers.

[0029] Finally, the processing results of the two stages are fused to obtain the final fused dataset. The fusion process takes into account the continuity between different stages and avoids data gaps.

[0030] This phased, multi-strategy processing method solves the problem of multi-source heterogeneous data fusion and cleaning during drone flight, and improves the accuracy and consistency of the data.

[0031] Through the above-mentioned solution, this application solves the technical challenges of fusing and cleaning multi-source heterogeneous data during drone flight. By introducing predicted trajectories as a unified spatiotemporal reference, it effectively addresses the issues of timestamp drift and inconsistent spatial references between different devices. A phased processing strategy, tailored to the characteristics of the stable flight phase and the dynamic transformation phase, employs different anomaly detection and cleaning methods, improving the pertinence and accuracy of data processing.

[0032] In some of the above-mentioned schemes of the present application, the spatiotemporal alignment of the initial data specifically includes: using the predicted trajectory data as the reference trajectory, reversely calibrating the timestamps and spatial position information of each modal data in the initial data, and obtaining aligned data that is consistent with the trajectory data in time and space; wherein, the reverse calibration includes interpolating missing time points, and performing timing adjustment and spatial remapping of the offset data.

[0033] Among them, difference filling generates supplementary data at missing time points through interpolation algorithms, such as linear interpolation or cubic spline interpolation methods; timing adjustment uses dynamic time warping algorithms to nonlinearly stretch or compress the data sequence to eliminate phase differences on the time axis; spatial remapping uses coordinate transformation matrices to uniformly convert data in different sensor coordinate systems into the global coordinate system of the reference trajectory, such as through Euler angle rotation matrices or quaternion transformations to achieve three-dimensional spatial mapping.

[0034] Specifically, the predicted trajectory data is first used as the reference time axis. For each data point in the initial data, the time difference between it and the nearest time point in the predicted trajectory data is calculated. If the time difference exceeds a preset threshold, it is considered a missing data point. Missing data points are filled using linear interpolation.

[0035] Secondly, for data points with time offsets, a dynamic time warping algorithm is used to adjust the timing. The initial data sequence is aligned with the predicted trajectory data sequence to find the best time correspondence, and then the timestamps of the initial data are corrected based on the correspondence.

[0036] Finally, spatial remapping is used to calibrate data points with offset spatial positions. The deviation vector between the spatial coordinates of the initial data and the spatial coordinates of the corresponding time point in the predicted trajectory data is calculated. Based on the deviation vector, the spatial coordinates of the initial data are translated and rotated to align them spatially with the predicted trajectory data.

[0037] Through the above technical solution, this application achieves precise spatiotemporal alignment of multi-source heterogeneous data. By using the predicted trajectory data as a reference to reversely calibrate the initial data, the problems of inconsistent sampling frequencies, inconsistent timestamps, and inconsistent spatial coordinate systems between different data sources are resolved. Missing data points are interpolated to ensure data continuity and integrity. Through timing adjustment and spatial remapping, spatiotemporal deviations caused by sensor delays or drift are corrected. This improves the accuracy of subsequent data analysis and processing, providing a reliable data foundation for intelligent monitoring and analysis of drones during flight.

[0038] In some of the above-mentioned solutions of this application, although data cleaning and optimization under different flight states of the drone are achieved through stage classification processing, in actual flight, the transition from stable flight to high-speed maneuvering flight, or the recovery to stable flight after the dynamic transformation action is completed, is not an instantaneous mutation behavior, but rather there is a certain degree of dynamic continuity. If the data of the two divided stages are directly spliced ​​and fused during the data processing process, it is easy to cause trajectory mutations, discontinuous attitude angles, unreasonable thrust changes, or non-conservation of system energy at the state switching boundary. Direct fusion may affect the continuity of the fused data on the time axis and the consistency of the physical model, thereby reducing the overall data quality and the reliability of the subsequent flight decision algorithm.

[0039] The present application further proposes that before fusing the processing results of the first processing strategy and the second processing strategy, the method also includes: obtaining the adjacent first time boundary and second time boundary of the first data and the second data on the time axis; extracting the trajectory state parameters of the corresponding time point from the predicted trajectory data based on the first time boundary and the second time boundary; determining the first data and the second data with an earlier time sequence position on the time axis as the previous stage, and determining the later data as the next stage; establishing a state transition function between the previous stage and the next stage based on the trajectory state parameters; and performing continuity correction on the data corresponding to the next stage within the preset time period of the first time boundary / second time boundary according to the state transition function.

[0040] Among them, the state transition function adopts the polynomial fitting method to establish cubic polynomial models for position, velocity and attitude angle respectively.

[0041] Through the above technical solution, this application achieves a smooth data transition between the stable flight phase and the dynamic transformation phase, avoiding data discontinuity during phase switching. By establishing a state transition function, the data near key time points is corrected for continuity, improving the temporal consistency and physical rationality of the data, providing a more reliable data foundation for subsequent motion analysis and performance evaluation.

[0042] In some of the above-mentioned solutions of this application, although the processing of drone flight data is achieved through multi-stage data division and cleaning strategies, in actual applications, the cleaned data may still cause deviations between the recognition results of key flight actions and the predicted trajectory due to environmental interference or sensor abnormalities, affecting the accuracy of subsequent training analysis and safety assessment.

[0043] The present application further proposes: through a pre-trained action recognition model, identifying key flight actions within a target time period based on a fusion data set; performing consistency analysis based on the key flight actions and the motion trajectories at corresponding time points in the predicted trajectory data to obtain a deviation degree representing the degree of deviation between the two; judging whether the deviation degree exceeds a preset deviation threshold, and if so, backtracking the first data and / or the second data according to the time period corresponding to the deviation degree; dynamically adjusting the cleaning parameters in the first processing strategy and / or the second processing strategy based on the deviation degree, and re-cleaning and compensating the backtracked data until the deviation degree is less than the preset deviation threshold.

[0044] Among them, the flight action recognition model adopts an architecture that combines convolutional neural networks (CNN) and long short-term memory networks (LSTM). It inputs multimodal time series sensor data in the fused data set and outputs flight action categories and corresponding timestamp tags; consistency analysis includes quantitative calculation of time deviation, spatial position deviation and attitude angle deviation, and generates a comprehensive deviation degree through weighted summation; the dynamic adjustment strategy selects corresponding cleaning parameter correction methods for different flight phases, such as increasing the window length of trajectory continuity detection in the steady-state cruise phase, and increasing the tolerance threshold of the attitude angle change rate in the high-maneuverability flight phase.

[0045] Specifically, the fused dataset is first processed using a pre-trained flight maneuver recognition model to identify the moments when the drone completes key flight maneuvers, such as turns, acceleration maneuvers, and hovering. These recognition results are then compared with the predicted trajectory data to calculate the degree of deviation. For example, if the drone is identified as completing an acceleration maneuver at a certain moment, but the predicted trajectory shows that the drone was still in a stable cruise state at that moment, a large deviation will occur.

[0046] The system then determines whether the deviation exceeds a pre-set threshold. If so, it indicates a possible problem with the data processing. The system then retraces the data to the time period where the deviation occurred and rechecks the original data.

[0047] Based on the identified issues, the data cleaning strategy can be dynamically adjusted. For example, for data with jumps, interpolation and completion can be increased, or the criteria for identifying outliers can be adjusted. The data can then be reprocessed using the adjusted strategy, including data completion and denoising.

[0048] This process is repeated until the processed data meets the required consistency with the predicted trajectory, that is, the deviation is reduced to below a threshold. In this way, the final data can be guaranteed to conform to the laws of physics and accurately reflect the actual performance of the drone.

[0049] In some of the above-mentioned schemes of the present application, the consistency analysis specifically includes: extracting the recognition time point of the key flight action and the theoretical time point corresponding to the key flight action in the predicted trajectory data, and calculating the time deviation between the recognition time point and the theoretical time point; extracting the core spatial position point coordinates in the key flight action recognition result at the recognition time point and the corresponding point coordinates in the predicted trajectory data, and calculating the Euclidean distance deviation between the core spatial position coordinates and the corresponding point coordinates; extracting the main body posture angle in the key flight action recognition result at the recognition time point and the corresponding posture angle in the predicted trajectory data, calculating the angle difference vector between the main body posture angle and the corresponding posture angle, and calculating the L2 norm of the angle difference vector as the posture angle deviation; calculating the deviation degree according to the time deviation, spatial position deviation and posture angle deviation. :

[0050] ;

[0051] in 、 、 Represent the time deviation, spatial position deviation and attitude angle deviation respectively, 、 、 They are respectively the preset time deviation threshold, spatial position deviation threshold and attitude angle deviation threshold, 、 、 is the preset weight coefficient, satisfying .

[0052] In some of the above-mentioned schemes of the present application, the setting of the preset deviation threshold includes: obtaining environmental status data at the identification time point; extracting environmental impact factors from a preset environment-action-influence factor mapping table based on environmental status factors and key flight actions; extracting the standard deviation threshold corresponding to the key flight action from a preset action-deviation threshold mapping table based on the key flight action; and correcting the standard deviation threshold according to the environmental impact factor to obtain the preset deviation threshold.

[0053] The environmental status data includes at least one of temperature, light intensity, and battery power. The environment-action-influence factor mapping table is pre-established using experimental data, and the table records the quantitative factors of the impact of different environmental factors on the deviation of specific actions. The action-deviation threshold mapping table is generated through historical data training, and the table stores the standard deviation thresholds corresponding to different action types. The formula for correcting the standard deviation threshold is: ;in, is the standard deviation threshold, is the environmental impact factor, , is the preset minimum deviation threshold.

[0054] Specifically, after the environmental status data is acquired, the degree of deviation of the current environment on the key flight action is determined by querying the environment-action-influence factor mapping table. For example, in a low-temperature environment, the sensor may experience data drift. At this time, the environmental impact factor is set to a positive value to increase the deviation threshold; in a highly reflective environment, the image data may be distorted, and the corresponding factor is dynamically adjusted. After the standard deviation threshold is extracted from the action-deviation threshold mapping table, it is dynamically adjusted according to the correction formula to ensure that the preset deviation threshold not only adapts to environmental changes, but also avoids excessive sensitivity caused by too low a threshold. For example, when When the value is 0.2, the standard threshold is magnified to 1.2 times, but if the calculated result is lower than , then use As a lower limit, it avoids excessive relaxation of the threshold due to environmental factors. Therefore, under different flight environment conditions, the deviation threshold can dynamically adapt to the actual error range of sensor data, improving the robustness of anomaly detection.

[0055] Through the above technical solution, this application achieves dynamic adjustment of the deviation threshold based on environmental conditions and flight maneuver type, avoiding the oversensitivity or over-leniency caused by fixed thresholds and improving the accuracy and robustness of anomaly detection. At the same time, the introduction of a minimum threshold ensures basic data quality requirements. This method can better adapt to complex and changing environments and provide a reliable foundation for subsequent data processing and analysis.

[0056] Exemplary Systems

[0057] Figure 3The diagram illustrates an intelligent fusion and cleaning system for multi-source heterogeneous data according to an embodiment of the present application, comprising: a data acquisition module for acquiring multi-source heterogeneous initial data within a target time period; a trajectory prediction module for generating predicted trajectory data within a target time period based on the initial data using a pre-trained trajectory prediction model; a data alignment module for performing spatiotemporal alignment on the initial data based on the predicted trajectory data to obtain aligned data; a data classification module for dividing the aligned data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to preset classification rules; a first cleaning module for cleaning the first data according to a preset first processing strategy, the first processing strategy including anomaly detection and compensation processing based on trajectory continuity and acceleration stability; a second cleaning module for cleaning the second data according to a preset second processing strategy, the second processing strategy including anomaly recognition and synchronous correction based on posture change characteristics and image temporal consistency; and a data fusion module for fusing the processing results of the first processing strategy and the second processing strategy to obtain a fused data set.

[0058] In one example, the data alignment module performs spatiotemporal alignment on the initial data, including: using the predicted trajectory data as the reference trajectory, and reversely calibrating the timestamps and spatial position information of each modal data in the initial data to obtain aligned data that is consistent with the trajectory data in time and space; wherein, the reverse calibration includes interpolating missing time points, and performing timing adjustment and spatial remapping on the offset data.

[0059] In one example, before the data fusion module fuses the processing results of the first processing strategy and the second processing strategy, it also includes: obtaining the adjacent first time boundary and second time boundary of the first data and the second data on the time axis; extracting the trajectory state parameters of the corresponding time point from the predicted trajectory data based on the first time boundary and the second time boundary; determining the first data and the second data with the earlier time sequence position on the time axis as the previous stage, and determining the later stage as the next stage; establishing a state transition function between the previous stage and the next stage based on the trajectory state parameters; and performing continuity correction on the data corresponding to the next stage within the preset time period of the first time boundary / second time boundary according to the state transition function.

[0060] In one example, a reprocessing module is also included: through a pre-trained action recognition model, key flight actions within the target time period are identified based on the fusion data set; consistency analysis is performed based on the key flight actions and the motion trajectory of the corresponding time point in the predicted trajectory data to obtain a deviation degree representing the degree of deviation between the two; it is determined whether the deviation degree exceeds a preset deviation threshold, and if so, the first data and / or the second data are backtracked according to the time period corresponding to the deviation degree; the cleaning parameters in the first processing strategy and / or the second processing strategy are dynamically adjusted based on the deviation degree, and the backtracked data is re-cleaned and compensated until the deviation degree is less than the preset deviation threshold.

[0061] In one example, the consistency analysis performed by the reprocessing module includes: extracting the recognition time point of the key flight action and the theoretical time point corresponding to the key flight action in the predicted trajectory data, and calculating the time deviation between the recognition time point and the theoretical time point; extracting the core space position point coordinates in the key flight action recognition result at the recognition time point and the corresponding point coordinates in the predicted trajectory data, and calculating the Euclidean distance deviation between the core space position coordinates and the corresponding point coordinates; extracting the main body posture angle in the key flight action recognition result at the recognition time point and the corresponding posture angle in the predicted trajectory data, calculating the angle difference vector between the main body posture angle and the corresponding posture angle, and calculating the L2 norm of the angle difference vector as the posture angle deviation; calculating the deviation degree according to the time deviation, spatial position deviation and posture angle deviation. :

[0062] ;

[0063] in 、 、 Represent the time deviation, spatial position deviation and attitude angle deviation respectively, 、 、 They are respectively the preset time deviation threshold, spatial position deviation threshold and attitude angle deviation threshold, 、 、 is the preset weight coefficient, satisfying .

[0064] In one example, the setting of the preset deviation threshold in the reprocessing module includes: obtaining environmental status data at the identification time point; extracting environmental impact factors from a preset environment-action-influence factor mapping table based on environmental status factors and key flight actions; extracting the standard deviation threshold corresponding to the key flight action from a preset action-deviation threshold mapping table based on the key flight action; and correcting the standard deviation threshold according to the environmental impact factor to obtain the preset deviation threshold.

[0065] In one example, the formula for correcting the standard deviation threshold by the reprocessing module is:

[0066] ;

[0067] in, is the standard deviation threshold, is the environmental impact factor, , is the preset minimum deviation threshold.

[0068] Exemplary electronic devices

[0069] Figure 4 The figure shows an electronic device according to an embodiment of the present application. The electronic device can be the mobile device itself, or a stand-alone device independent of the mobile device, which can communicate with the mobile device to receive collected input signals from the mobile device and send the selected target driving behavior to the mobile device.

[0070] Figure 4 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0071] like Figure 4 As shown, the electronic device includes one or more processors and memory.

[0072] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0073] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the driving behavior decision-making method of each embodiment of the present application described above and / or other desired functions.

[0074] In one example, the electronic device may further include an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0075] Of course, to simplify, Figure 4Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0076] Exemplary computer-readable media

[0077] An embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the driving behavior decision-making method according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0078] Computer-readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0079] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0080] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0081] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0082] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0083] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An intelligent fusion and cleaning method for multi-source heterogeneous data, characterized by: include: Acquire multi-source heterogeneous initial data within the target time period; Generate predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; Performing spatiotemporal alignment on the initial data according to the predicted trajectory data to obtain aligned data; dividing the alignment data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; Cleaning the first data according to a preset first processing strategy, wherein the first processing strategy includes anomaly detection and compensation processing based on trajectory continuity and acceleration stability; Cleaning the second data according to a preset second processing strategy, wherein the second processing strategy includes anomaly recognition and synchronous correction based on posture change characteristics and image time sequence consistency; The processing results of the first processing strategy and the second processing strategy are fused to obtain a fused data set.

2. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 1 is characterized in that: The performing spatiotemporal alignment on the initial data comprises: Using the predicted trajectory data as a reference trajectory, reversely calibrating the timestamps and spatial position information of each modal data in the initial data to obtain aligned data that is consistent with the trajectory data in time and space; The reverse calibration includes filling in the difference of missing time points, and performing time sequence adjustment and spatial remapping on the offset data.

3. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 1 is characterized in that: Before fusing the processing results of the first processing strategy and the second processing strategy, the method further includes: Acquire a first time boundary and a second time boundary adjacent to the first data and the second data on a time axis; extracting a trajectory state parameter corresponding to a time point from the predicted trajectory data according to the first time boundary and the second time boundary; Determine the first data and the second data that have an earlier time sequence position on the time axis as the previous stage, and determine the later time sequence position as the next stage; Establishing a state transition function between a previous stage and a next stage according to the trajectory state parameters; According to the state transition function, the data corresponding to the preset time period of the first time boundary / the second time boundary in the subsequent stage is corrected in continuity.

4. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 1 is characterized in that: Also includes: identifying key flight actions within the target time period based on the fused dataset using a pre-trained action recognition model; Performing consistency analysis on the key flight action and the motion trajectory at the corresponding time point in the predicted trajectory data to obtain a deviation degree indicating the degree of deviation between the two; Determine whether the deviation exceeds a preset deviation threshold, and if so, backtrack the first data and / or the second data according to a time period corresponding to the deviation; The cleaning parameters in the first processing strategy and / or the second processing strategy are dynamically adjusted based on the deviation, and the backtracked data is re-cleaned and compensated until the deviation is less than the preset deviation threshold.

5. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 4 is characterized in that: The consistency analysis includes: Extracting the recognition time point of the key flight action and the theoretical time point corresponding to the key flight action in the predicted trajectory data, and calculating the time deviation between the recognition time point and the theoretical time point; Extracting the core space position point coordinates in the key flight action recognition result at the recognition time point and the corresponding point coordinates in the predicted trajectory data, and calculating the Euclidean distance deviation between the core space position coordinates and the corresponding point coordinates; Extracting the subject attitude angle in the key flight action recognition result at the recognition time point and the corresponding attitude angle in the predicted trajectory data, calculating an angle difference vector between the subject attitude angle and the corresponding attitude angle, and calculating the L2 norm of the angle difference vector as the attitude angle deviation; Calculate the deviation degree according to the time deviation, spatial position deviation and attitude angle deviation : ； in 、 、 Represent the time deviation, spatial position deviation and attitude angle deviation respectively, 、 、 They are respectively the preset time deviation threshold, spatial position deviation threshold and attitude angle deviation threshold, 、 、 is the preset weight coefficient, satisfying .

6. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 5 is characterized in that: The setting of the preset deviation threshold includes: Acquire environmental status data at the identified time point; Extracting environmental impact factors from a preset environment-action-influence factor mapping table according to the environmental state factors and the key flight actions; Extracting a standard deviation threshold value corresponding to the key flight action from a preset action-deviation threshold mapping table according to the key flight action; The standard deviation threshold is corrected according to the environmental impact factor to obtain the preset deviation threshold.

7. The intelligent fusion and cleaning method for multi-source heterogeneous data according to claim 6 is characterized in that: The formula for correcting the standard deviation threshold is: ； in, is the standard deviation threshold, is the environmental impact factor, , is the preset minimum deviation threshold.

8. An intelligent fusion and cleaning system for multi-source heterogeneous data, characterized by: include: The data acquisition module is used to obtain the initial data of multiple sources and heterogeneity within the target time period; A trajectory prediction module, configured to generate predicted trajectory data within the target time period based on the initial data using a pre-trained trajectory prediction model; a data alignment module, configured to perform spatiotemporal alignment on the initial data according to the predicted trajectory data to obtain aligned data; a data classification module, configured to classify the alignment data into first data corresponding to a stable flight phase and second data corresponding to a dynamic transformation phase according to a preset classification rule; a first cleaning module, configured to clean the first data according to a preset first processing strategy, wherein the first processing strategy includes anomaly detection and compensation processing based on trajectory continuity and acceleration stability; a second cleaning module, configured to clean the second data according to a preset second processing strategy, wherein the second processing strategy includes anomaly recognition and synchronous correction based on posture change characteristics and image temporal consistency; The data fusion module is used to fuse the processing results of the first processing strategy and the second processing strategy to obtain a fused data set.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer storage medium having computer-executable instructions stored thereon, characterized in that: When the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images