A track fusion method, device, equipment and storage medium

Abstract

The application discloses a track fusion method, device and equipment and a storage medium, and is applied to an unmanned equipment cluster scene. The method first collects multi-source target data of a target scene, extracts appearance and motion features of a target object from the multi-source target data, combines a historical track database, completes target association through feature matching, obtains first track data containing a unique target ID, recognizes abnormal track data in the first track data by using a graph convolutional neural network, and finally optimizes the abnormal track data to obtain second track data. The application fully excavates the value of multi-source data, improves the target matching accuracy, the completeness and the precision of the track data, truly reflects the actual motion state of the target object, and provides reliable track data support for the collaborative decision of the unmanned equipment cluster and the target management and control.

Description

Technical Field

[0001] This application relates to the field of information processing and target tracking technology, and in particular to a track fusion method, apparatus, device and storage medium. Background Technology

[0002] With the widespread adoption of applications such as collaborative drone operations, unmanned equipment platooning, and multi-target security monitoring, it is necessary to collect multi-source observation data of multiple target objects within a target scene and perform track fusion processing to form track data reflecting the target's motion state. However, existing track fusion methods still face many technical challenges in practical applications.

[0003] Existing technologies, when processing multi-source observation data, often extract only single-dimensional information for matching and associating targets with historical tracks, failing to fully explore the value of multi-source data. This results in decreased matching accuracy in scenarios with dense targets and mutual occlusion, and significant deviations between the initially generated track data and the actual target trajectory.

[0004] Meanwhile, existing track fusion methods lack effective quality detection and correction mechanisms for the initially generated track data. On the one hand, they cannot detect anomalies such as track interruptions and trajectory jumps in a timely manner; on the other hand, they lack effective repair methods for detected anomalies. This results in a large amount of abnormal data directly entering the subsequent processing or output stages, leading to poor integrity of the final output track data. The reliability of subsequent operations based on such data is difficult to guarantee. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a track fusion method, apparatus, device, and storage medium, with the aim of improving the accuracy and completeness of track fusion.

[0006] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides a track fusion method, the method comprising: Collect multi-source target data in the target scene; the multi-source target data includes multi-source observation data of multiple target objects within the target scene; Based on the multi-source target data, the appearance features and motion features of each target object are obtained; the appearance features characterize the visual appearance of the target object, and the motion features characterize the position and motion state of the target object in the current frame. Based on the appearance and motion characteristics of each target object, target matching and association are performed from the historical track database to obtain the associated track information of each target object; the historical track database stores established historical track data, and each historical track data includes the appearance characteristics, motion characteristics, and unique target ID corresponding to the historical track data; the associated track information includes the unique target ID matched and associated with each target object and the first track data; Abnormal track data in the first track data is identified using a graph convolutional neural network; The abnormal track data in the first track data is optimized to obtain the second track data.

[0007] In an optional implementation, the step of performing target matching and association from a historical track database based on the appearance and motion characteristics of each target object to obtain the associated track information of each target object includes: Based on the target fusion weight, the appearance features and motion features of each target object are weighted and fused to obtain the first fusion feature corresponding to each target object; The historical appearance features and historical motion features of the historical target objects corresponding to each historical track data are read from the historical track database, and the predicted motion features of the current frame are predicted based on the historical motion features of each historical target object. Based on the target fusion weight, the historical appearance features and predicted motion features of each historical target object are weighted and fused to obtain the second fusion feature corresponding to each historical target object; Based on the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object, target matching and association are performed to obtain the associated trajectory information of each target object.

[0008] In an optional implementation, the step of performing target matching and association based on the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object to obtain the associated track information of each target object includes: Determine the similarity between the first fusion feature corresponding to each of the target objects and the second fusion feature corresponding to each of the historical target objects; Based on the similarity, the historical track data associated with each target object is determined, and the track association confidence level corresponding to each matching association is determined based on the similarity; the track association confidence level characterizes the accuracy of the matching association. Based on the historical track data associated with the matching, the associated track information of each target object is obtained.

[0009] In an optional implementation, the target fusion weights are updated periodically with a first dynamic duration; the method for determining the first dynamic duration includes: Obtain the target environment features in the target scene; Based on the target environment features, a scene complexity score for the current frame is obtained; the scene complexity score for the current frame indicates the complexity of the target scene environment in the current frame. Based on the scene complexity score of the current frame and a preset mapping relationship, a first dynamic duration is determined; the preset mapping relationship includes: When the scene complexity score of the current frame is greater than the first preset threshold, the first dynamic duration is set to the first duration value; when the scene complexity score is less than or equal to the first preset threshold, the first dynamic duration is set to the second duration value; the first duration value is less than the second duration value.

[0010] In an optional implementation, the target fusion weights are adjusted based on the track association confidence level in the previous update cycle, and the adjustment methods include: If the track association confidence level is lower than a preset confidence threshold and continues to exceed a preset number of frames in the previous update cycle, the allocation ratio of appearance feature weight and motion feature weight in the target fusion weight is adjusted according to the target environment characteristics.

[0011] In an optional implementation, the abnormal track data includes track break segments and track jump segments; the step of identifying abnormal track data in the first track data using a graph convolutional neural network includes: The first track data is constructed into a track graph structure, and the track data corresponding to each frame in the first track data is used as a graph node. The node features of the graph node include the appearance features, motion features and unique target ID of the target object corresponding to the graph node. Based on the motion state of the target object represented by the motion features, the consistency of motion direction between each node is determined, and the consistency of motion direction is used as the edge weight between nodes. The spatiotemporal dependencies of each node in the trajectory graph structure are learned by graph convolutional neural network, and the normal motion pattern of the target object is determined by combining the motion features in the node features. Based on the normal motion pattern, track break segments and track jump segments are identified in the first track data. The track break segment is a range of multiple consecutive frames without valid track data under the same unique target ID, and valid track data exists in the frames before and after. The track jump segment is track frame data where the position of the target object represented by the motion feature in the node feature deviates from the normal motion pattern.

[0012] In an optional implementation, optimizing the abnormal trajectory data to obtain the second trajectory data includes: For the broken track segment, extract the effective track data of the preceding consecutive frames of the broken track segment, and predict the filling motion features of the target object in each missing frame of the broken track segment by combining the normal motion law of the target object with the graph convolutional neural network. Fill the broken track segment with the filling motion features and complete the broken track segment. For the aforementioned track jump segment, based on the normal track nodes of multiple frames before and after the track jump segment, the position of the track jump segment is processed by weighted average smoothing through graph attention mechanism; The completed track break segments and the smoothed track jump segments are integrated into the first track data to obtain the second track data.

[0013] A second aspect of this application provides a track fusion apparatus, the apparatus comprising: The data acquisition module is used to collect multi-source target data in the target scene; the multi-source target data includes multi-source observation data of multiple target objects in the target scene; The feature extraction module is used to obtain the appearance features and motion features of each target object based on the multi-source target data; the appearance features characterize the visual appearance of the target object, and the motion features characterize the position and motion state of the target object in the current frame; The track association module is used to perform target matching and association from the historical track database based on the appearance and motion characteristics of each target object to obtain the associated track information of each target object; the historical track database stores established historical track data, and each historical track data includes the appearance characteristics, motion characteristics, and unique target ID corresponding to the historical track data; the associated track information includes the unique target ID matched and associated with each target object and the first track data; An anomaly detection module is used to identify abnormal track data in the first track data using a graph convolutional neural network; The trajectory optimization module is used to optimize the abnormal trajectory data in the first trajectory data to obtain the second trajectory data.

[0014] A third aspect of this application provides a trajectory fusion device, which includes: a processor and a memory. The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the track fusion method described in any implementation of the first aspect according to the instructions in the program code.

[0015] A fourth aspect of this application provides a computer-readable storage medium for storing program code for performing the steps of the track fusion method described in any implementation of the first aspect.

[0016] Compared with the prior art, this application has the following advantages: In the technical solution of this application, firstly, multi-source target data in the target scene is collected, including multi-source observation data of multiple target objects within the target scene; secondly, based on the multi-source target data, the appearance features and motion features of each target object are obtained, wherein the appearance features characterize the visual appearance of the target object, and the motion features characterize the position and motion state of the target object in the current frame; then, based on the appearance features and motion features of each target object, target matching and association are performed from a historical track database to obtain the associated track information of each target object, wherein the historical track database stores established historical track data, and each historical track data includes the appearance features, motion features, and unique target ID corresponding to the historical track data, and the associated track information includes the unique target ID matched and associated with each target object and first track data; then, abnormal track data in the first track data is identified through a graph convolutional neural network; finally, the abnormal track data in the first track data is optimized to obtain second track data. As can be seen, this application fully leverages the effective value of multi-source data by extracting the appearance and motion features of the target object from multi-source target data and combining them with historical trajectory databases for target matching and association. This significantly improves the accuracy of target matching and association, making the initially generated first trajectory data more closely match the actual motion trajectory of the target. Simultaneously, a graph convolutional neural network is used to accurately identify abnormal trajectory data in the first trajectory data, and targeted optimization processing is performed on these abnormal trajectory data. This effectively corrects and improves the trajectory data, resulting in a significant increase in the completeness and accuracy of the final second trajectory data. This data truly reflects the actual motion state of the target object, providing reliable data support for subsequent collaborative decision-making and target control operations based on trajectory data, effectively ensuring the reliability of subsequent operations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1A flowchart of a track fusion method provided in an embodiment of this application; Figure 2 This application provides a schematic diagram of trajectory fusion delay comparison in a multi-target intersection scenario. Figure 3 This application provides a schematic diagram of track fusion accuracy in a multi-target intersection scenario. Figure 4 This is a schematic diagram of the structure of a track fusion device provided in an embodiment of this application. Detailed Implementation

[0019] As described earlier, with the widespread adoption of applications such as collaborative drone operations, unmanned equipment platooning, and multi-target security monitoring, it is necessary to collect multi-source observation data of multiple target objects within a target scenario and perform track fusion processing to form track data reflecting the target's motion state. However, existing track fusion methods still face many technical challenges in practical applications.

[0020] Existing technologies, when processing multi-source observation data, often extract only single-dimensional information for matching and associating targets with historical tracks, failing to fully explore the value of multi-source data. This results in decreased matching accuracy in scenarios with dense targets and mutual occlusion, and significant deviations between the initially generated track data and the actual target trajectory.

[0021] Meanwhile, existing track fusion methods lack effective quality detection and correction mechanisms for the initially generated track data. On the one hand, they cannot detect anomalies such as track interruptions and trajectory jumps in a timely manner; on the other hand, they lack effective repair methods for detected anomalies. This results in a large amount of abnormal data directly entering the subsequent processing or output stages, leading to poor integrity of the final output track data. The reliability of subsequent operations based on such data is difficult to guarantee.

[0022] To address the aforementioned problems, the inventors have proposed a track fusion method, device, equipment, and storage medium after research.

[0023] First, multi-source target data is collected in the target scene, including multi-source observation data of multiple target objects within the target scene. Second, based on the multi-source target data, the appearance features and motion features of each target object are obtained, wherein the appearance features characterize the visual appearance of the target object, and the motion features characterize the position and motion state of the target object in the current frame. Next, based on the appearance features and motion features of each target object, target matching and association are performed from a historical track database to obtain associated track information for each target object. The historical track database stores established historical track data, and each historical track data entry includes the appearance features, motion features, and unique target ID corresponding to that historical track data. The associated track information includes the unique target ID matched and associated with each target object and first track data. Then, abnormal track data in the first track data is identified using a graph convolutional neural network. Finally, the abnormal track data in the first track data is optimized to obtain second track data. As can be seen, this application fully leverages the effective value of multi-source data by extracting the appearance and motion features of the target object from multi-source target data and combining them with historical trajectory databases for target matching and association. This significantly improves the accuracy of target matching and association, making the initially generated first trajectory data more closely match the actual motion trajectory of the target. Simultaneously, a graph convolutional neural network is used to accurately identify abnormal trajectory data in the first trajectory data, and targeted optimization processing is performed on these abnormal trajectory data. This effectively corrects and improves the trajectory data, resulting in a significant increase in the completeness and accuracy of the final second trajectory data. This data truly reflects the actual motion state of the target object, providing reliable data support for subsequent collaborative decision-making and target control operations based on trajectory data, effectively ensuring the reliability of subsequent operations.

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0025] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] The trajectory fusion method provided in this application can be applied to various unmanned equipment cluster scenarios, such as drones in drone swarms, unmanned vehicles in unmanned vehicle convoys, and unmanned vessels in unmanned vessel formations. Each unmanned equipment acts as an independent processing entity, completing core processing actions such as local trajectory detection, association, and optimization of its own collected data. The core node only acts as a federated learning server to coordinate the aggregation and updating of model parameters, and does not directly receive raw sensor data or perform basic trajectory fusion.

[0027] See Figure 1 This figure is a flowchart of a trajectory fusion method provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: S101. Collect multi-source target data in the target scene.

[0028] In this embodiment, multi-source target data is synchronously collected by multiple types of sensors locally mounted on each unmanned equipment. The main data collector is a single unmanned equipment such as a drone or an autonomous vehicle. During the data collection process, the raw data is not transmitted to the core node or other unmanned equipment; data storage and subsequent processing are only completed locally.

[0029] In this embodiment, the multiple types of sensors include radar sensors, photoelectric sensors, and inertial measurement units. The corresponding multi-source target data includes physical feature data of the target environment and motion attitude data of the unmanned equipment itself, providing full-dimensional data support for subsequent feature extraction and trajectory association.

[0030] In one example implementation, a radar sensor acquires distance and velocity data of the target object, an optoelectronic sensor acquires color or grayscale image data of the target object, and an inertial measurement unit acquires attitude data such as pitch angle, roll angle, heading angle, and speed of the unmanned equipment.

[0031] In another example implementation, for unmanned vehicle platooning scenarios, a lidar sensor can be added to collect 3D point cloud data of the target object and supplement it to the multi-source target data, thereby improving the ability to perceive targets under complex road conditions.

[0032] This application embodiment collects multi-source sensor data locally and independently, avoiding the transmission of raw data across nodes. This protects data privacy and core sensor parameters, reduces data transmission latency, and provides a rich data foundation for subsequent multi-feature fusion, adapting to the complex perception needs of multi-target cross-scenes.

[0033] S102. Based on multi-source target data, obtain the appearance features and motion features of each target object.

[0034] In this embodiment of the application, appearance features characterize the visual appearance of the target object, including visual attributes such as the outline, texture, and color of the target object; motion features characterize the position and motion state of the target object in the current frame, including physical attributes such as the real-time coordinates, motion speed, and motion direction of the target object.

[0035] In this embodiment, feature extraction is performed on preprocessed multi-source target data based on the improved YOLO-Transformer architecture. This architecture uses the C2f module as the backbone network for feature extraction, enhances the multi-scale target adaptation capability through SPPF spatial pyramid pooling, and introduces a Transformer encoder in the output layer to focus on the target region using a self-attention mechanism, which can simultaneously output high-precision appearance features and motion features of the target object.

[0036] See Figure 2 , Figure 2 This application provides a schematic diagram of trajectory fusion delay comparison in a multi-target cross-scene, as shown in the embodiments of this application. Figure 2 As shown, the method in this application relies on a lightweight improved YOLO-Transformer architecture and a fully local processing mode. The trajectory fusion processing latency remains relatively stable as the amount of data processed increases. When the data processing volume is 10 lines / frame, the processing latency is only 20ms. When the data processing volume reaches 50 lines / frame, the processing latency is still within 40ms. In contrast, the processing latency of the traditional Kalman filter fusion method increases rapidly with the amount of data, reaching 160ms at 50 lines / frame. The processing latency of the non-lightweight YOLO-Transformer fusion method is always higher than 60ms. The trajectory fusion method provided in this application significantly reduces the overall processing latency of trajectory fusion, achieves tactical-level real-time processing efficiency, and effectively improves the real-time response capability of unmanned equipment formation collaborative operations.

[0037] In one example implementation, multi-source target data is first preprocessed locally. Gaussian filtering removes electromagnetic interference noise from radar data, and median filtering eliminates salt-and-pepper noise from electro-optical images. Attitude correction is performed on the electro-optical images based on attitude data from the inertial measurement unit to eliminate image distortion caused by the motion of the unmanned equipment itself. All sensor data is uniformly converted into a tensor format with a preset resolution, and data augmentation is performed through random cropping, flipping, and brightness adjustment. The preprocessed tensor data is then input into an improved YOLO-Transformer architecture for feature extraction.

[0038] In another example implementation, the lidar point cloud data is converted into a tensor dimension that matches the photoelectric image through voxelization. After multimodal fusion with the image data, it is then input into the feature extraction network to improve the dimensionality and recognizability of the appearance features.

[0039] This application's embodiments improve the accuracy and robustness of feature extraction by combining the fusion and preprocessing of multi-source sensor data, providing a reliable feature basis for subsequent target matching and association.

[0040] S103. Based on the appearance and motion characteristics of each target object, target matching and association are performed from the historical track database to obtain the associated track information of each target object.

[0041] In this embodiment of the application, the historical track database is a database stored locally by each unmanned equipment. It stores all historical track data of targets that have been established by the equipment. Each historical track data includes the appearance features, motion features and unique target ID corresponding to the historical track data. The historical track data is updated in real time during the local processing of the equipment.

[0042] The associated track information includes the unique target ID associated with each target object and the first track data. The first track data is the initial track data that is obtained by integrating the target object in the current frame with the historical tracks, which includes the target object's real-time position, speed, and direction of motion.

[0043] In this embodiment, the unique target ID refers to a globally unique identifier assigned to each detected target object, used to distinguish different target objects and avoid track confusion in multi-target overlapping scenarios. The first track data refers to the unoptimized raw track data formed by combining the current frame features with historical track data after the target object is matched and associated, and includes the target's real-time status information.

[0044] In one alternative implementation, step S103 includes: S1031. Based on the target fusion weight, the appearance features and motion features of each target object are weighted and fused to obtain the first fusion feature corresponding to each target object.

[0045] In this embodiment, the target fusion weights include appearance feature weights. and motion feature weights Both satisfy It is generated by the local attention network of the unmanned equipment and dynamically updated based on scene complexity and trajectory correlation accuracy.

[0046] Appearance features The feature extraction layer output from the improved YOLO-Transformer is then converted into a fixed-dimensional vector through global average pooling, representing motion features. This is the target motion state vector extracted based on multi-source data of the current frame.

[0047] In one example implementation, feature weighted fusion is achieved through a multi-feature attention fusion mechanism, with the following calculation formula: ; in, The first fusion feature, This is the appearance feature vector of the target object in the current frame. The motion feature vector of the target object in the current frame. , The real-time target fusion weights are output by the local attention network.

[0048] S1032. Read the historical appearance features and historical motion features of the historical target objects corresponding to each historical track data from the historical track database, and predict the motion features of the current frame based on the historical motion features of each historical target object.

[0049] In this embodiment of the application, the historical appearance feature is the appearance feature vector of the target object in previous frames stored in the historical track database, and the historical motion feature is the motion state data such as the position and speed of the target object in previous frames.

[0050] In this embodiment, the Kalman filter algorithm can be used to predict the state of historical motion features to obtain the predicted motion features of the historical target object in the current frame, thereby achieving accurate estimation of the target motion trend and adapting to the association requirements in target maneuvering scenarios.

[0051] In one example implementation, the predicted motion features are calculated using the state prediction formula of the Kalman filter, as follows: ; ; in, Let k be the predicted motion state at time k, i.e., the current frame, and A be the state transition matrix. The historical best estimated motion state at time k-1. For the control matrix, The control quantity at time k-1, Let k be the prediction error covariance. Estimate the error covariance at time k-1. Let be the process noise covariance.

[0052] S1033. Based on the target fusion weight, the historical appearance features and predicted motion features of each historical target object are weighted and fused to obtain the second fusion feature corresponding to each historical target object.

[0053] In this embodiment, the target fusion weight is consistent with the real-time weight used in step S1031 to ensure that the calculation standard of the fusion features of the target object in the current frame and the target object in the past is unified.

[0054] Historical appearance features are in the form of fixed-dimensional vectors. Predicted motion features are converted into vectors that match the dimensions of historical appearance features after being output by Kalman filtering, ensuring the feasibility of fusion calculation.

[0055] In one example implementation, the same multi-feature attention fusion mechanism as in step S1031 is used, and the calculation formula is as follows: ; in, This is the second fusion feature. This refers to the historical appearance feature vector of the target object. Predict the motion feature vector for the current frame of the historical target object. , Weights are fused for real-time targets.

[0056] S1034. Based on the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object, target matching and association are performed to obtain the associated trajectory information of each target object.

[0057] In this embodiment of the application, the matching relationship between the target object in the current frame and the target object in the past can be determined by calculating the similarity between the first fusion feature and the second fusion feature.

[0058] In one alternative implementation, step S1034 includes: Step 1: Determine the similarity between the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object.

[0059] In this embodiment of the application, the cosine similarity algorithm can be used to calculate the similarity between two fused feature vectors. The value range of cosine similarity is [-1, 1]. The closer the value is to 1, the higher the similarity between the two feature vectors, and the greater the probability that the target object in the current frame and the target object in the past are the same target.

[0060] In one example implementation, the formula for calculating cosine similarity is: ; in, For cosine similarity, The first fusion feature, This is the second fusion feature.

[0061] Step 2: Determine the historical track data associated with each target object based on similarity, and determine the track association confidence level corresponding to each matching relationship based on similarity.

[0062] In the embodiments of this application, the track association confidence level characterizes the accuracy of the matching association.

[0063] In this embodiment, the similarity between the first fusion feature of the target object in the current frame and the second fusion feature of all historical target objects is sorted, and the historical target object corresponding to the maximum similarity is selected as the matching object. If the maximum similarity is higher than the similarity threshold of the model's adaptive learning, the matching is determined to be successful, and the corresponding historical track data is associated; if it is lower than the threshold, it is determined to be a new target, a new unique target ID is assigned to it, and new track data is created.

[0064] In one example implementation, the cosine similarity value is directly mapped to the track association confidence score. The similarity value is the confidence score; the higher the confidence score, the higher the accuracy of the matching association and the stronger the reliability of subsequent track data. For a new target, its initial track association confidence score is set to a preset baseline value, such as 0.8, and is dynamically updated based on the matching results of subsequent frames.

[0065] Step 3: Based on the historical track data matched and associated, obtain the associated track information of each target object.

[0066] In this embodiment of the application, if a match is successful, the real-time status data of the target object in the current frame is added to the matched historical track data to form first track data containing a unique target ID, historical status, current real-time status and track association confidence; if it is determined to be a new target, based on the features and status data of the target object in the current frame, a new first track data containing a new unique target ID is created and the track data is stored in the local historical track database.

[0067] In one example implementation, the first track data is stored in a structured manner, with data fields including: target ID, frame number, real-time position coordinates, motion speed, motion direction, track association confidence, appearance feature vector, and motion feature vector, which facilitates subsequent track optimization and historical data retrieval.

[0068] In one optional implementation, to improve the adaptability of the target fusion weights to dynamic scenarios and avoid a decrease in track association accuracy caused by sudden scene changes, the target fusion weights are updated periodically with a first dynamic duration. The determination of the first dynamic duration includes: Step 1: Obtain the target environment features in the target scene.

[0069] In this embodiment of the application, the target environment features can be extracted by a lightweight CNN network based on multi-source target data collected locally by unmanned equipment.

[0070] In one example implementation, the target environment features include electromagnetic interference intensity, target maneuvering coefficient, and illumination abrupt change degree, wherein the electromagnetic interference intensity is extracted based on the noise fluctuation level of radar data, the target maneuvering coefficient is extracted based on the rate of change of historical motion features of the target object, and the illumination abrupt change degree is extracted based on the brightness variance of the photoelectric image.

[0071] Step 2: Based on the characteristics of the target environment, obtain the scene complexity score of the current frame; the scene complexity score of the current frame indicates the complexity of the target scene environment in the current frame.

[0072] In this embodiment, the scene complexity score is calculated by weighted summation. The weight coefficients of each environmental feature are determined by local training to adapt to the sensor characteristics and application scenarios of different unmanned equipment.

[0073] In one example implementation, the formula for calculating the scene complexity score is: ; in, Score the scene complexity of the current frame. Scoring the intensity of electromagnetic interference. The target maneuver coefficient, The degree of change in light intensity, , , For the weighting coefficients, satisfying .

[0074] Step 3: Determine the first dynamic duration based on the scene complexity score of the current frame and the preset mapping relationship.

[0075] In this embodiment, the preset mapping relationship is a rule for the correspondence between scene complexity and weight iteration cycle, which is stored locally in each unmanned equipment and can be flexibly adjusted according to the actual application scenario.

[0076] The first dynamic duration is the iterative update cycle of the target fusion weights, measured in frames. The shorter the cycle, the more frequently the weights are updated, and the more timely the response to scene changes.

[0077] In this embodiment, the preset mapping relationship includes: when the scene complexity score of the current frame is greater than a first preset threshold, setting the first dynamic duration to a first duration value, such as 5 frames; when the scene complexity score is less than or equal to the first preset threshold, setting the first dynamic duration to a second duration value, such as 20 frames. The first duration value is less than the second duration value, achieving rapid weight iteration in complex scenes and saving computational resources in simple scenes.

[0078] In one optional implementation, to optimize the target fusion weights based on track association accuracy feedback and further improve association accuracy, the target fusion weights are adjusted based on the track association confidence level in the previous update cycle. The adjustment methods include: When the track association confidence level is lower than the preset confidence threshold and continues to exceed the preset number of frames in the previous update cycle, the allocation ratio of appearance feature weight and motion feature weight in the target fusion weight is adjusted according to the target environment characteristics.

[0079] In one alternative implementation, the specific adjustment formula is as follows: ; ; in, , These are the appearance feature weights before and after the iteration, respectively. The weights of the motion features after iteration. The learning rate set for the local model. The average track association confidence level over the previous update period. To preset the signal threshold, when < When the frame rate exceeds the preset frame rate, the weights are automatically adjusted based on environmental characteristics. For example, when electromagnetic interference is strong, the weights of motion features are increased, and when there are sudden changes in lighting, the weights of appearance features are increased.

[0080] This application's embodiments solve the track confusion problem in multi-target scenarios by replacing the traditional manual threshold association method with dynamic target matching and association based on multi-feature fusion. Simultaneously, by combining scene perception to achieve dynamic iterative updates of target fusion weights, it adapts to instantaneous changes in scenarios such as electromagnetic interference, target maneuvering, and sudden changes in illumination, avoiding the decrease in association accuracy caused by static weights and providing high-precision initial track data for subsequent track optimization.

[0081] S104. Identify abnormal track data in the first track data using a graph convolutional neural network.

[0082] In this application embodiment, abnormal track data refers to track data that deviates from the normal movement pattern of the target object. It is a track problem that is prone to occur in scenarios with multiple targets crossing and dense occlusion. It mainly includes track break segments and track jump segments. Such data will lead to poor track continuity and low accuracy, and cannot provide reliable support for unmanned equipment decision-making.

[0083] Graph convolutional neural networks employ an architecture that combines graph convolutional layers and graph attention layers to construct a graph structure from flight track data. By learning the spatiotemporal dependencies between nodes, they can accurately identify flight track anomalies.

[0084] In this embodiment, the training and inference of the graph convolutional neural network are both completed locally on each unmanned equipment. The model is fine-tuned based on local historical flight track data to make the network more suitable for the sensor acquisition characteristics and application scenarios of the equipment, while avoiding cross-node latency in model inference and ensuring real-time performance.

[0085] In one alternative implementation, the abnormal track data includes track break segments and track jump segments, and step S104 includes: S1041. Construct the first track data into a track graph structure, and take the track data corresponding to each frame in the first track data as a graph node. The node features of the graph node include the appearance features, motion features and unique target ID of the target object corresponding to the graph node.

[0086] In this embodiment, the track graph structure is an undirected graph. Subgraphs are constructed based on continuous frame tracks of a single target object, and the track subgraphs of multiple targets together form the global track graph. Node features are high-dimensional vectors, which are concatenated with the encoded vectors of appearance features, motion features, and unique target IDs to provide a basis for subsequent learning of dependencies between nodes.

[0087] In one example implementation, the unique target ID is one-hot encoded and converted into a vector that matches the dimensions of appearance features and motion features. Then, the three are concatenated according to their dimensions to form the final node feature vector. , where i is the node number.

[0088] S1042. Based on the motion state of the target object represented by motion features, determine the consistency of motion direction between nodes, and use the consistency of motion direction as the edge weight between nodes.

[0089] In this embodiment, motion direction consistency is an index for measuring the similarity of motion patterns between adjacent frame nodes or different target nodes in the same scene. The value range is [0,1]. The closer the value is to 1, the more consistent the motion directions between nodes are and the stronger the correlation.

[0090] Edge weights are used to characterize the degree of dependence between nodes in the trajectory graph, providing a weight basis for feature learning in graph convolutional neural networks.

[0091] In one example implementation, motion direction consistency is calculated based on the motion direction angle of the target object. Let the motion direction angle of node i be... The direction angle of movement of node j is Then the directions of motion of the two are consistent. This value is then used directly as the edge weight between node i and node j.

[0092] S1043. Learn the spatiotemporal dependencies of each node in the trajectory map structure through graph convolutional neural network, and determine the normal motion law of the target object by combining the motion features in the node features.

[0093] In this embodiment, the node features of the track map are updated by graph convolutional layers to learn the correlation between the node's own features and the features of its neighboring nodes, thereby mining the spatiotemporal patterns of the track data.

[0094] Based on the updated node features, clustering and fitting algorithms are used to obtain motion law models of target objects in unobstructed and interference-free scenarios, such as uniform linear motion and uniformly accelerated motion, which serve as the benchmark for abnormal track identification.

[0095] In one example implementation, the spatiotemporal dependencies of nodes are learned through a graph convolution feature update formula, which is: ; in, Let i be the feature vector of node i in the (l+1)th layer. Let be the set of neighboring nodes of node i. , Let i be the degree of nodes i and j, i.e., the number of nodes in the neighborhood. Let j be the feature vector of node j in the l-th layer. The weights are the convolutional weights of the l-th layer. For the l-th layer bias term, This is the ReLU activation function.

[0096] After multi-layer graph convolution calculation, the motion characteristics of the nodes are fitted with a polynomial to obtain the normal motion law of the target object.

[0097] S1044. Based on normal motion patterns, identify track break segments and track jump segments in the first track data.

[0098] In this embodiment, the broken track segment is a range of consecutive frames without valid track data under the same unique target ID, and where valid track data exists in the preceding and following frames. This is mostly caused by building obstruction or sensor detection failure.

[0099] Track jump segments are track frame data in which the position of the target object deviates from the normal motion pattern, as represented by the motion features in the node features. They are mostly caused by sensor noise and electromagnetic interference.

[0100] In this embodiment, track break segments are identified by detecting the continuity of track data, and track jump segments are identified by detecting the position deviation threshold. Both detections are performed locally, and the detection thresholds are obtained by graph convolutional neural networks based on historical data adaptive learning.

[0101] In one example implementation, for a broken track segment, the track data of consecutive frames with the same target ID is traversed. If there are N consecutive frames without valid data, and the frames before and after have valid data, then the interval is determined to be a broken track segment, where N is an adaptive threshold, such as 3 frames.

[0102] For a track jump segment, calculate the Euclidean distance between the actual position of the current frame node and the predicted position based on normal motion patterns. If the distance is greater than a preset deviation threshold, the track data of that frame is determined to be a track jump segment.

[0103] This application embodiment uses graph convolutional neural network for track anomaly identification, converting track data into a graph structure to fully explore the spatiotemporal dependencies of multi-target tracks; at the same time, it uses the normal motion patterns of target objects as the identification benchmark, improving the accuracy and reliability of abnormal track identification and clarifying the processing objects for subsequent track optimization.

[0104] S105. Optimize the abnormal track data in the first track data to obtain the second track data.

[0105] In the embodiments of this application, the optimization process is completed locally on each unmanned equipment. A motion law prediction and completion strategy is adopted for broken track segments, and a graph attention mechanism smoothing strategy is adopted for track jump segments. Both strategies are based on the learning results of graph convolutional neural networks, ensuring that the optimized track data conforms to the normal motion law of the target, while retaining the real motion characteristics of the target and avoiding over-optimization bias.

[0106] In one alternative implementation, the optimization methods specifically include: For a broken track segment, valid track data from multiple consecutive frames preceding the broken track segment is extracted. By combining graph convolutional neural networks with the normal motion patterns of the target object, the filling motion features of the target object in each missing frame of the broken track segment are predicted. The filling motion features are then filled into the broken track interval to complete the broken track segment.

[0107] In this embodiment, the number of valid track data frames selected from consecutive preceding frames is determined by the target's motion pattern; for example, the first two frames are selected for uniform motion, and the first three frames are selected for uniformly accelerated motion. The motion features to be filled include the target position, velocity, and direction of motion in the missing frames, maintaining consistency with the feature dimensions of the first track data to ensure the uniformity of the completed track data.

[0108] In one example implementation, the location coordinates of missing frames are predicted using a weighted summation method to fill in the core parameters of the motion features, as shown in the formula: ; in, Let be the predicted location coordinates of the missing frame at time t. , Let be the true position coordinates of the two adjacent frames before time t. The prediction weights are generated by a graph convolutional neural network based on the normal motion patterns of the target, satisfying... .

[0109] Based on the predicted position coordinates, the velocity and direction of motion of the missing frames are deduced by combining normal motion patterns, thus forming complete motion features for filling in the missing frames.

[0110] For track jump segments, the position of the track jump segment is smoothed by weighted average based on the normal track nodes of multiple frames before and after the track jump segment through a graph attention mechanism.

[0111] In this embodiment, the normal track nodes in the preceding and following frames are the track nodes without abnormalities adjacent to the jump frame, and the number selected is 2-3 frames before and after. The graph attention mechanism learns the attention weights between nodes and performs weighted correction on the position of the jump frame, so that the corrected position conforms to the normal motion law of the target, while preserving the target's maneuverability.

[0112] In one example implementation, trajectory smoothing is performed using a graph attention-weighted average formula, which is: ; in, Let i be the position coordinates of the transition node after smoothing. Let i be the set of normal track nodes in the neighborhood of the jump node i. Let be the graph attention weights between node i and node j, which are generated by the graph attention network based on node feature similarity. Let be the position coordinates of a normal node j in the domain.

[0113] The completed track break segments and the smoothed track jump segments are integrated into the first track data to obtain the second track data.

[0114] In this embodiment of the application, the trajectory data is checked for consistency during the integration process to ensure that the completed and smoothed trajectory data does not conflict with the valid trajectory data of the preceding and following frames. At the same time, the confidence of the trajectory data is updated. The confidence of the completed trajectory frame is linearly reduced based on the confidence of the preceding frame, and the confidence of the smoothed trajectory frame is the average of the confidence of the neighboring nodes.

[0115] In one example implementation, the integrated track data is structurally reconstructed, retaining core fields such as target ID, frame number, and real-time status, while deleting abnormal track markers. The reconstructed track data is then used as the second track data, which is anomaly-free, highly continuous, and highly accurate, and can directly support the decision-making and control of unmanned equipment.

[0116] This application's embodiments address the issues of track breakage and track jumps in multi-target intersection scenarios through targeted abnormal track optimization processing, improving the continuity and accuracy of track data. Simultaneously, the optimization strategy based on graph convolutional neural networks and graph attention mechanisms ensures that the optimized track data conforms to the actual movement patterns of the targets, avoiding over-optimization and providing reliable track data support for unmanned equipment formation coordination and autonomous obstacle avoidance.

[0117] To improve the global trajectory fusion accuracy of the entire unmanned equipment cluster while protecting data privacy, and to enhance the cluster's overall adaptability to complex scenarios, each unmanned equipment acts as a federated learning client, and the core node acts as a federated learning server, thus constructing a distributed collaborative fusion architecture.

[0118] The specific implementation is as follows: First, each unmanned equipment (UAV) trains a trajectory fusion model locally based on locally collected multi-source target data and processing results. This includes improved YOLO-Transformer models and graph convolutional neural network models, but only the model parameters are trained; no raw data is transmitted. Second, the core node periodically sends model parameter collection requests to each client, and each client uploads its locally trained model parameters to the core node. Then, the core node assigns dynamic aggregation weights based on the sensor reliability of each UAV, such as data integrity, noise level, and trajectory association accuracy; higher reliability results in a larger aggregation weight. Next, the core node updates the global model parameters using a weighted aggregation algorithm and distributes the updated global model parameters to each UAV. Finally, each UAV fine-tunes its local model based on the global model parameters, optimizing and upgrading the local model to improve local trajectory fusion accuracy.

[0119] The formula for modeling the loss of the client-side local model is as follows: ; The formula for aggregating global parameters of the core node is: ; In the above formula, Let the local model loss be for the i-th client. The total loss for target detection and tracking for the i-th client is consistent with the loss of the improved YOLO-Transformer, including classification loss, localization loss, and confidence loss. The regularization coefficient is . For the local model parameters of the i-th client, These are global model parameters. Here are the global model parameters after aggregation in round t+1, and n is the number of federated learning clients, i.e., the number of unmanned equipment. The dynamic aggregation weight of the i-th client satisfies , These are the local model parameters after t rounds of training for the i-th client.

[0120] In addition, the federated learning architecture has high fault tolerance. Even if some unmanned devices go offline, the core nodes only skip the parameter collection and aggregation of that node, while the remaining online nodes can still complete local model training and global parameter aggregation, maintaining the local fusion calculation of the cluster. Moreover, the global model accuracy only decreases slightly and will not cause the overall fusion to fail.

[0121] This application's embodiments achieve global model collaborative optimization through federated learning, enabling co-training of models across unmanned equipment clusters while eliminating the need for cross-node transmission of raw data. This protects data privacy and core sensor parameters while improving the accuracy of global trajectory fusion. Simultaneously, the high fault tolerance of the distributed architecture avoids single-node failure issues inherent in centralized data processing, further enhancing the robustness of the entire cluster in trajectory fusion under multi-target scenarios.

[0122] The trajectory fusion method provided in this application independently completes the entire process of multi-source data acquisition, feature extraction, trajectory matching and association, and anomaly optimization locally. It combines a dynamic association strategy for multi-feature fusion, trajectory optimization methods using graph neural networks, a weight iteration mechanism for dynamic scene perception, and a distributed global model collaborative architecture based on federated learning. This achieves a comprehensive improvement in the local accuracy, real-time processing, environmental adaptability, and global robustness of trajectory fusion in multi-objective cross-scenario scenarios. It reduces the error accumulation of traditional step-by-step processing, improves the accuracy and continuity of trajectory decoupling and fusion, and reduces the overall processing latency through a lightweight model and local processing architecture, adapting to the tactical-level real-time decision-making needs of unmanned equipment. At the same time, it achieves a synergistic improvement in data privacy protection and global model accuracy through federated learning, and enhances the overall operational capability of unmanned equipment clusters with the high fault tolerance of the distributed architecture. It provides stable, reliable, and high-precision trajectory data support for collaborative perception, autonomous decision-making, and formation control of various unmanned equipment clusters such as UAV swarms, unmanned vehicle formations, and unmanned vessel formations.

[0123] See Figure 3 , Figure 3 This application provides a schematic diagram of track fusion accuracy in a multi-target cross-scene, as illustrated in the embodiments of this application. Figure 3 As shown, the method in this application, by combining a dynamic scene-aware weight iteration mechanism, an attention mechanism, and an adaptive similarity threshold design, maintains a high level of track fusion accuracy across different environmental dimensions. The fusion accuracy reaches 98% under normal conditions, 94% under complex lighting conditions such as strong light and heavy rain, and even 92% under strong electromagnetic interference. The accuracy degradation rate under each environment does not exceed 8%. In contrast, while traditional fusion methods achieve 90% accuracy under normal conditions, this drops to 75% under strong light, 72% under heavy rain, and only 68% under strong electromagnetic interference, with an accuracy degradation rate of 20%-30% under various adverse environments. The method in this application significantly improves the adaptability to adverse environments such as complex lighting and electromagnetic interference, significantly improves track fusion accuracy under various complex environments, effectively reduces the impact of adverse environments on track fusion results, and greatly enhances the robustness of track fusion.

[0124] Based on the trajectory fusion method provided in the foregoing embodiments, this application also provides a trajectory fusion apparatus. Figure 4 This is a schematic diagram of a track fusion device provided in an embodiment of this application. Figure 4 As shown, the trajectory fusion device includes: a data acquisition module 401, a feature extraction module 402, a trajectory association module 403, an anomaly identification module 404, and a trajectory optimization module 405.

[0125] The data acquisition module 401 is used to acquire multi-source target data in the target scene; the multi-source target data includes multi-source observation data of multiple target objects in the target scene.

[0126] The feature extraction module 402 is used to obtain the appearance features and motion features of each target object based on multi-source target data; the appearance features represent the visual appearance of the target object, and the motion features represent the position and motion state of the target object in the current frame.

[0127] The track association module 403 is used to perform target matching and association from the historical track database based on the appearance and motion characteristics of each target object to obtain the associated track information of each target object. The historical track database stores established historical track data, and each historical track data contains the appearance characteristics, motion characteristics and unique target ID corresponding to the historical track data. The associated track information includes the unique target ID matched and associated with each target object and the first track data.

[0128] Anomaly detection module 404 is used to identify abnormal track data in the first track data through graph convolutional neural network.

[0129] The trajectory optimization module 405 is used to optimize abnormal trajectory data in the first trajectory data to obtain the second trajectory data.

[0130] This application embodiment improves the accuracy and completeness of track fusion through the coordinated operation of the data acquisition module 401, feature extraction module 402, track association module 403, anomaly identification module 404, and track optimization module 405.

[0131] In an optional implementation, the track association module 403 includes a first fusion unit, a feature prediction unit, a second fusion unit, and a matching association unit.

[0132] The first fusion unit is used to perform weighted fusion of the appearance features and motion features of each target object based on the target fusion weight, so as to obtain the first fusion feature corresponding to each target object.

[0133] The feature prediction unit is used to read the historical appearance features and historical motion features of the historical target objects corresponding to each historical track data from the historical track database, and predict the motion features of the current frame based on the historical motion features of each historical target object.

[0134] The second fusion unit is used to perform weighted fusion of the historical appearance features and predicted motion features of each historical target object based on the target fusion weight, so as to obtain the second fusion feature corresponding to each historical target object.

[0135] The matching and association unit is used to perform target matching and association based on the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object, so as to obtain the associated track information of each target object.

[0136] In the optional implementation, the matching association unit is specifically used for: Determine the similarity between the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object. Based on the similarity, determine the historical track data to be matched and associated with each target object, and determine the track association confidence corresponding to each matching relationship based on the similarity; the track association confidence represents the accuracy of the matching relationship. Based on the historical track data to be matched and associated, obtain the associated track information of each target object.

[0137] In an optional implementation, the target fusion weights used in the track association module 403 are updated periodically with a first dynamic duration. The track association module 403 also includes a duration determination unit.

[0138] The duration determination unit is used to acquire target environment features in the target scene. Based on the target environment features, the scene complexity score of the current frame is obtained; the scene complexity score of the current frame indicates the complexity of the scene environment in the target scene of the current frame. Based on the scene complexity score of the current frame and a preset mapping relationship, a first dynamic duration is determined; the preset mapping relationship includes: when the scene complexity score of the current frame is greater than a first preset threshold, the first dynamic duration is set to a first duration value; when the scene complexity score is less than or equal to the first preset threshold, the first dynamic duration is set to a second duration value; the first duration value is less than the second duration value.

[0139] In an optional implementation, the target fusion weights used in the track association module 403 are adjusted based on the track association confidence in the previous update cycle, and the track association module 403 also includes a weight adjustment unit.

[0140] The weight adjustment unit is used to adjust the allocation ratio of appearance feature weight and motion feature weight in the target fusion weight according to the target environment characteristics when the track association confidence is lower than the preset confidence threshold and continues to exceed the preset number of frames in the current update cycle.

[0141] In the optional implementation, the abnormal track data includes track break segments and track jump segments, and the anomaly identification module 404 is specifically used for: The first track data is constructed into a track graph structure, with each frame of track data in the first track data serving as a graph node. The node features of the graph node include the appearance features, motion features, and unique target ID of the target object corresponding to that node. Based on the motion state of the target object represented by the motion features, the consistency of motion direction between nodes is determined, and the consistency of motion direction is used as the edge weight between nodes. The spatiotemporal dependencies of each node in the track graph structure are learned through a graph convolutional neural network, and the normal motion pattern of the target object is determined by combining the motion features in the node features. Based on the normal motion pattern, track break segments and track jump segments in the first track data are identified. Track break segments are intervals with multiple consecutive frames without valid track data under the same unique target ID, but with valid track data in the preceding and following frames. Track jump segments are track frames where the target object position deviates from the normal motion pattern, as represented by the motion features in the node features.

[0142] In the optional implementation, the trajectory optimization module 405 is specifically used for: For broken track segments, valid track data from multiple consecutive frames preceding the broken segment is extracted. A graph convolutional neural network, combined with the normal motion patterns of the target object, is used to predict the filling motion features of the target object in each missing frame of the broken track segment. These filling motion features are then used to fill in the broken track interval, thus completing the broken track segment. For track jump segments, based on the normal track nodes of multiple frames before and after the jump segment, a weighted average smoothing process is applied to the position of the jump segment using a graph attention mechanism. The completed broken track segment and the smoothed jump segment are then integrated into the first track data to obtain the second track data.

[0143] In addition, this application embodiment also provides a trajectory fusion device, which includes a processor and a memory.

[0144] The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the track fusion method described in any of the above method embodiments according to the instructions in the program code.

[0145] Furthermore, embodiments of this application also provide a computer-readable storage medium for storing program code for executing the steps of the track fusion method described in any of the above method embodiments.

[0146] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and equipment embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0147] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A trajectory fusion method, characterized in that, include: Collect multi-source target data in the target scene; The multi-source target data includes multi-source observation data of multiple target objects within the target scene; Based on the multi-source target data, the appearance features and motion features of each target object are obtained; the appearance features characterize the visual appearance of the target object, and the motion features characterize the position and motion state of the target object in the current frame. Based on the appearance and motion characteristics of each target object, target matching and association are performed from the historical track database to obtain the associated track information of each target object; the historical track database stores established historical track data, and each historical track data includes the appearance characteristics, motion characteristics, and unique target ID corresponding to the historical track data; the associated track information includes the unique target ID matched and associated with each target object and the first track data; Abnormal track data in the first track data is identified using a graph convolutional neural network; The abnormal track data in the first track data is optimized to obtain the second track data.

2. The method according to claim 1, characterized in that, The step involves matching and associating targets from a historical flight track database based on their appearance and motion characteristics to obtain associated flight track information for each target object, including: Based on the target fusion weight, the appearance features and motion features of each target object are weighted and fused to obtain the first fusion feature corresponding to each target object; The historical appearance features and historical motion features of the historical target objects corresponding to each historical track data are read from the historical track database, and the predicted motion features of the current frame are predicted based on the historical motion features of each historical target object. Based on the target fusion weight, the historical appearance features and predicted motion features of each historical target object are weighted and fused to obtain the second fusion feature corresponding to each historical target object; Based on the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object, target matching and association are performed to obtain the associated trajectory information of each target object.

3. The method according to claim 2, characterized in that, The step of performing target matching and association based on the first fusion feature corresponding to each target object and the second fusion feature corresponding to each historical target object to obtain the associated track information of each target object includes: Determine the similarity between the first fusion feature corresponding to each of the target objects and the second fusion feature corresponding to each of the historical target objects; Based on the similarity, the historical track data associated with each target object is determined, and the track association confidence level corresponding to each matching association is determined based on the similarity; the track association confidence level characterizes the accuracy of the matching association. Based on the historical track data associated with the matching, the associated track information of each target object is obtained.

4. The method according to claim 3, characterized in that, The target fusion weights are updated periodically with a first dynamic duration; the method for determining the first dynamic duration includes: Obtain the target environment features in the target scene; Based on the target environment features, a scene complexity score for the current frame is obtained; the scene complexity score for the current frame indicates the complexity of the target scene environment in the current frame. Based on the scene complexity score of the current frame and a preset mapping relationship, a first dynamic duration is determined; the preset mapping relationship includes: When the scene complexity score of the current frame is greater than the first preset threshold, the first dynamic duration is set to the first duration value; when the scene complexity score is less than or equal to the first preset threshold, the first dynamic duration is set to the second duration value; the first duration value is less than the second duration value.

5. The method according to claim 4, characterized in that, The target fusion weights are adjusted based on the track association confidence level in the previous update cycle, and the adjustment methods include: If the track association confidence level is lower than a preset confidence threshold and continues to exceed a preset number of frames in the previous update cycle, the allocation ratio of appearance feature weight and motion feature weight in the target fusion weight is adjusted according to the target environment characteristics.

6. The method according to claim 1, characterized in that, The abnormal track data includes track break segments and track jump segments; the step of identifying abnormal track data in the first track data using a graph convolutional neural network includes: The first track data is constructed into a track graph structure, and the track data corresponding to each frame in the first track data is used as a graph node. The node features of the graph node include the appearance features, motion features and unique target ID of the target object corresponding to the graph node. Based on the motion state of the target object represented by the motion features, the consistency of motion direction between each node is determined, and the consistency of motion direction is used as the edge weight between nodes. The spatiotemporal dependencies of each node in the trajectory graph structure are learned by graph convolutional neural network, and the normal motion pattern of the target object is determined by combining the motion features in the node features. Based on the normal motion pattern, track break segments and track jump segments are identified in the first track data. The track break segment is a range of multiple consecutive frames without valid track data under the same unique target ID, and valid track data exists in the frames before and after. The track jump segment is track frame data where the position of the target object represented by the motion feature in the node feature deviates from the normal motion pattern.

7. The method according to claim 6, characterized in that, The optimization processing of the abnormal flight path data to obtain the second flight path data includes: For the broken track segment, extract the effective track data of the preceding consecutive frames of the broken track segment, and predict the filling motion features of the target object in each missing frame of the broken track segment by combining the normal motion law of the target object with the graph convolutional neural network. Fill the broken track segment with the filling motion features and complete the broken track segment. For the aforementioned track jump segment, based on the normal track nodes of multiple frames before and after the track jump segment, the position of the track jump segment is processed by weighted average smoothing through a graph attention mechanism; The completed track break segments and the smoothed track jump segments are integrated into the first track data to obtain the second track data.

8. A trajectory fusion device, characterized in that, include: The data acquisition module is used to collect multi-source target data in the target scene; The multi-source target data includes multi-source observation data of multiple target objects within the target scene; The feature extraction module is used to obtain the appearance features and motion features of each target object based on the multi-source target data; The appearance feature characterizes the visual appearance of the target object, and the motion feature characterizes the position and motion state of the target object in the current frame; The track association module is used to perform target matching and association from the historical track database based on the appearance and motion characteristics of each target object to obtain the associated track information of each target object; the historical track database stores established historical track data, and each historical track data includes the appearance characteristics, motion characteristics, and unique target ID corresponding to the historical track data; the associated track information includes the unique target ID matched and associated with each target object and the first track data; An anomaly detection module is used to identify abnormal track data in the first track data using a graph convolutional neural network; The trajectory optimization module is used to optimize the abnormal trajectory data in the first trajectory data to obtain the second trajectory data.

9. A trajectory fusion device, characterized in that, include: Processor and memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the steps of the track fusion method according to any one of claims 1 to 7, based on the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for performing the steps of the track fusion method according to any one of claims 1 to 7.

Images