A method and system for dynamic visual display of event information

Abstract

The application discloses a kind of event information dynamic visual display method and system, belong to data processing technical field, including: by multimodal data fusion generates initial feature label, obtains unique identification code by weighted fusion and is bound with target;Based on the target of binding identification code, by three types of sensors cooperative collection and calibration motion vector, generate real-time motion vector data in combination with update frequency and synchronization mechanism;If target is out of sight, call fusion prediction model, input real-time motion vector data, prestore racecourse environment and preset target physical parameter, obtain the predicted trajectory with confidence degree;If target appears cross occlusion, construct individual motion prediction model to generate virtual trajectory, and match measured trajectory to find back identification code after occlusion is removed;Finally, fit multiple data into dynamic image, realize event information dynamic visual display, improve event tracking continuity and information display intuitiveness.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically a method and system for dynamically visualizing event information. Background Technology

[0002] In various competitive events, real-time monitoring of the movement status of participating targets and their visualization are crucial requirements for event management, referee decisions, and enhancing the spectator experience. Currently, existing event information display methods largely rely on single sensors for target tracking and data collection, which has significant limitations: First, when a participating target moves out of the sensor's field of view due to track obstruction or excessive distance, tracking is easily interrupted, making it impossible to continuously acquire target movement information. Second, when multiple participating targets move in parallel, cross-occlusion can easily occur, causing the sensor to be unable to distinguish individual targets, thus losing target identification information and affecting subsequent data association and display. Third, existing methods lack precise calibration and fusion of target movement data and do not incorporate track environment and target physical characteristics for trajectory prediction, resulting in insufficient continuity and accuracy in visualization, failing to meet the event's demand for real-time, complete, and accurate information presentation. Therefore, there is an urgent need for a method that can achieve stable target tracking, address the problems of field-of-view loss and cross-occlusion, and efficiently complete the dynamic visualization of event information. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes a method and system for dynamic visualization of event information. It generates initial feature labels through multimodal data fusion, obtains a unique identification code through weighted fusion, and binds it to the target. Based on the target with the bound identification code, three types of sensors collaboratively collect and calibrate motion vectors, generating real-time motion vector data by combining update frequency and synchronization mechanisms. If the target leaves the field of view, a fusion prediction model is invoked, inputting real-time motion vector data, pre-stored track environment, and preset target physical parameters to obtain a predicted trajectory with confidence. If the target is obstructed, an individual motion prediction model is constructed to generate a virtual trajectory; after the obstruction is removed, the actual measured trajectory is matched to retrieve the identification code. Finally, multiple types of data are fitted into a dynamic image, achieving dynamic visualization of event information and improving the continuity of event tracking and the intuitiveness of information display.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A method for dynamically visualizing event information includes:

[0006] S1: When the target enters the sensor's monitoring range for the first time, an initial feature label is generated through multimodal data fusion. The initial feature label is then weighted and fused to generate a unique identification code, which is then bound to the target to obtain the target with the bound identification code.

[0007] S2: Based on the binding identification code, the competition target collects and calibrates motion vectors through three types of sensors, and generates real-time motion vector data by combining update frequency and synchronization mechanism;

[0008] S3: If the target of the competition bound with the identification code leaves the sensor's field of view, the fusion prediction model is called. The real-time motion vector data, the pre-stored track environment, and the preset target physical parameters are input into the fusion prediction model to obtain the predicted trajectory with confidence.

[0009] S4: If the participating targets bound to the identification code are cross-occluded, the distance between the participating targets is monitored in real time by the visual camera to make a pre-judgment of occlusion. The motion data and appearance features before occlusion are cached in time to establish a snapshot library before occlusion. Then, the overall outline of the occluded group is obtained by LiDAR and the overall motion vector of the group is measured by millimeter-wave radar to obtain the overall trajectory of the occluded group. Based on the individual motion features of the participating targets in the snapshot library before occlusion, an individual motion prediction model is constructed. The overall trajectory is decomposed into a group trajectory to generate a virtual trajectory. After the occlusion is removed, the virtual trajectory is matched with the measured trajectory collected by the sensor to find the unique identification code bound to the participating target.

[0010] S5: Fit real-time motion vector data, predicted trajectories with confidence, and target data after retrieving the identification code into a dynamic image to complete the dynamic visualization of event information.

[0011] Specifically, when the target enters the sensor's monitoring range for the first time, generating initial feature labels through multimodal data fusion includes:

[0012] S1.1: When the target enters the sensor monitoring range for the first time, the three types of sensors will work together; the sensors include a visual camera, a lidar, and a millimeter-wave radar;

[0013] The visual camera acquires appearance image data of the participating target and extracts appearance feature parameters; the appearance feature parameters include the color features, shape features and texture features of the participating target.

[0014] The lidar acquires the three-dimensional contour data of the participating target and extracts contour feature parameters; the contour feature parameters include the length, width, and height parameters of the participating target.

[0015] The millimeter-wave radar acquires the initial velocity data of the participating target and extracts velocity feature parameters; the velocity feature parameters include the instantaneous velocity magnitude and velocity direction of the participating target;

[0016] S1.2: Input the appearance feature parameters, contour feature parameters and velocity feature parameters into a preset data fusion model. The data fusion model performs fusion processing on each feature parameter based on a weighted average algorithm and outputs initial feature labels.

[0017] Specifically, the steps of S2 include:

[0018] S2.1: Based on the target of the competition bound by the identification code, the three types of sensors, namely visual camera, LiDAR and millimeter-wave radar, are activated to enter the real-time monitoring state;

[0019] The visual camera captures motion images of the participating targets bound with identification codes in real time according to a preset acquisition frame rate. Based on the image recognition algorithm, the position coordinates of the participating targets in the image coordinate system are extracted, and the position change of the participating targets in adjacent frames is calculated to obtain the direction and magnitude of the preliminary motion vector.

[0020] The lidar scans the three-dimensional position of the participating target in real time, obtains the three-dimensional coordinates of the participating target in the three-dimensional spatial coordinate system, calculates the change of the three-dimensional coordinates per unit time, and obtains the three-dimensional motion vector.

[0021] The millimeter-wave radar monitors the radial and lateral velocities of the participating targets in real time and generates velocity vector data.

[0022] S2.2: Coordinate the preliminary motion vector, three-dimensional motion vector and velocity vector data to establish the timestamp correspondence of the data collected by each sensor. At the same time, call the sensor calibration database to obtain the measurement error parameters of each sensor.

[0023] S2.3: Based on the measurement error parameters, error compensation is performed on the obtained preliminary motion vector, three-dimensional motion vector and velocity vector data respectively. The compensation method is to superimpose the preliminary motion vector, three-dimensional motion vector and velocity vector data with the corresponding error correction values ​​respectively.

[0024] Specifically, the steps of S2 further include:

[0025] S2.4: Input the compensated preliminary motion vector, three-dimensional motion vector, and velocity vector data into the motion vector fusion model; the motion vector fusion model uses the Kalman filter algorithm to fuse the preliminary motion vector, three-dimensional motion vector, and velocity vector data, and outputs the calibrated motion vector;

[0026] S2.5: Preset the update frequency of motion vector data, and establish a time synchronization protocol between sensors. Based on the time synchronization protocol, calibrate the clocks of each sensor. The update frequency is determined according to the type of competition and the speed of the target.

[0027] S2.6: Periodically collect the calibrated motion vector according to the update frequency, and record the collection timestamp after each collection;

[0028] S2.7: Based on the time synchronization protocol between sensors, the time synchronization verification is performed on the calibrated motion vector and the corresponding acquisition timestamp for each acquisition. If the time deviation exceeds the preset deviation threshold, the acquisition data is corrected for time offset.

[0029] S2.8: Arrange the calibrated motion vectors, which have undergone time synchronization verification and correction, in the order of the acquisition timestamps to generate real-time motion vector data.

[0030] Specifically, the steps of S3 include:

[0031] S3.1: Real-time monitoring of the acquisition status of the three types of sensors on the competition targets with bound identification codes;

[0032] If the visual camera fails to capture the appearance image data of the target for a consecutive preset number of frames, the lidar fails to scan the three-dimensional contour data of the target for a consecutive preset time, and the millimeter-wave radar fails to detect the speed data of the target for a consecutive preset period, then the target is determined to be out of the sensor's field of view.

[0033] If any sensor can still collect data on the target, then continue with the step of generating real-time motion vector data;

[0034] S3.2: Pre-collect 3D terrain data, track boundary data, track obstacle distribution data, and track marker data of the race track to form a track environment model and store it in the track environment database. When the participating target leaves the sensor's field of view, retrieve the track environment model from the track environment database as the track environment data.

[0035] S3.3: Pre-set the target physical parameters and store them in the target physical parameter database. When the participating target leaves the sensor's field of view, retrieve the preset target physical parameters corresponding to the participating target from the target physical parameter database.

[0036] S3.4: The generated real-time motion vector data, the retrieved track environment data, and the retrieved preset target physical parameters are preprocessed according to the input format requirements of the fusion prediction model to form target fusion data and input into the fusion prediction model.

[0037] S3.5: The fusion prediction model first inputs the target fusion data into the LSTM network. The LSTM network learns the motion patterns of the participating target based on historical motion data and outputs a preliminary predicted trajectory. Then, the preliminary predicted trajectory is input into the particle filter algorithm module. The particle filter algorithm module optimizes and adjusts the preliminary predicted trajectory by combining the obstacle constraints in the track environment and the motion restrictions in the target's physical parameters. At the same time, the fusion prediction model calculates the confidence level of the predicted trajectory based on the completeness and accuracy of the input target fusion data and outputs the optimized predicted trajectory with confidence.

[0038] Specifically, the steps of S4 include:

[0039] S4.1: The visual camera acquires image data of multiple participating targets with bound identification codes in real time, and segments each participating target into independent target regions in the image based on the image segmentation algorithm, and calculates the distance between the target regions corresponding to adjacent participating targets in the image coordinate system; the distance includes horizontal distance and vertical distance;

[0040] S4.2: Compare the calculated distance with the preset occlusion determination threshold;

[0041] If the distance between all adjacent participating targets is greater than the occlusion detection threshold, it is determined that there is no risk of cross occlusion, and the step of generating real-time motion vector data continues.

[0042] If the distance between any adjacent participating targets is less than or equal to the occlusion detection threshold, it is determined that there is a risk of cross occlusion and an occlusion warning signal is issued.

[0043] S4.3: Upon receiving the occlusion warning signal, the data caching module is activated; the data caching module captures the latest motion data of the corresponding target from the generated real-time motion vector data, and simultaneously captures the latest appearance feature data of the target collected by the visual camera, i.e., the appearance feature parameters, and associates and stores the captured latest motion data and the latest appearance feature data according to the timestamp to form the snapshot data of the target before occlusion; the latest motion data includes motion speed, motion direction and motion position;

[0044] S4.4: Perform data capture and associated storage operations on all participating targets that have a risk of cross-occlusion, summarize the snapshot data of each participating target before occlusion, and establish a snapshot library before occlusion; each snapshot data in the snapshot library before occlusion contains the unique identification code of the participating target, the corresponding relationship between the motion data before occlusion and the appearance feature data before occlusion.

[0045] Specifically, the steps of S4 further include:

[0046] S4.5: The lidar performs a panoramic scan of the area where there is cross-occlusion, collects the three-dimensional point cloud data of the occlusion group composed of all participating targets in the area, and preprocesses the three-dimensional point cloud data. Based on the point cloud clustering algorithm, the preprocessed three-dimensional point cloud data is clustered to obtain the three-dimensional overall outline of the occlusion group; the three-dimensional overall outline includes the outer cuboid size of the occlusion group and the overall center position.

[0047] S4.6: The millimeter-wave radar continuously monitors the obstructing group and collects the overall speed data of the obstructing group; the overall speed data includes the magnitude of the overall movement speed and the overall movement direction;

[0048] S4.7: Based on the overall velocity data and the overall center position obtained by the lidar, calculate the change in the overall center position of the occlusion group per unit time to obtain the overall motion vector of the group;

[0049] S4.8: Based on the overall motion vector and time series, generate the overall trajectory of the occluded group in the track environment; the overall trajectory includes the overall position coordinates of the occluded group at different time points;

[0050] S4.9: Retrieve the motion data of each participating target before occlusion from the snapshot library before occlusion, and extract the individual motion characteristics of each participating target based on the motion data before occlusion; the motion data before occlusion includes the trend of motion speed change, the law of motion direction change, and the trajectory of motion position change within a preset time period before occlusion; the individual motion characteristics include motion acceleration characteristics, turning frequency characteristics, and linear motion preference characteristics;

[0051] S4.10: Combine the individual motion characteristics of each participating target with the preset basic motion model to construct an individual motion prediction model for each participating target; the basic motion model adopts a variable acceleration motion model.

[0052] Specifically, the steps of S4 further include:

[0053] S4.11: The overall trajectory of the occluded group is divided into multiple trajectory segments according to the timestamp. Each trajectory segment corresponds to a time interval. For each trajectory segment, based on the constructed individual motion prediction model of each participating target, the suspected motion trajectory of each participating target within the time interval is predicted. At the same time, combined with the overall contour constraint of the occluded group, the suspected motion trajectory of each participating target is adjusted. The overall contour constraint of the occluded group means that the predicted motion trajectory of each participating target cannot exceed the spatial range of the overall contour.

[0054] S4.12: The trajectory segmentation algorithm is used to match the adjusted suspected motion trajectories of each participating target with the overall trajectory segment to determine the specific motion path of each participating target in the overall trajectory segment. The specific motion paths of the participating targets determined in each time interval are connected in chronological order to generate the virtual trajectory of each participating target. The virtual trajectory contains the predicted position coordinates of the participating target at each time point during the occlusion period.

[0055] S4.13: The visual camera monitors the image data of the occluded area in real time. If the independent appearance image data of each participating target is re-captured, and the lidar re-scans the independent three-dimensional contour data of each participating target, and the millimeter-wave radar re-monitors the independent velocity data of each participating target, then the occlusion is determined to be lifted.

[0056] S4.14: After the occlusion is removed, the motion vectors of each participating target are collected and calibrated through the collaborative acquisition of three types of sensors, and the measured motion vector data of each participating target is generated. The measured trajectory is generated based on the measured motion vector data.

[0057] S4.15: Calculate the similarity between the virtual trajectory and the measured trajectory, and match the virtual trajectory with the highest similarity with the measured trajectory to retrieve the unique identification code bound to the competition target; the similarity is determined by calculating the average Euclidean distance between the position coordinates of the corresponding time points of the two trajectories; the unique identification code is contained in the snapshot data before occlusion associated with the individual motion prediction model of the competition target corresponding to the virtual trajectory.

[0058] Specifically, fitting the real-time motion vector data, the predicted trajectory with confidence, and the target data after retrieving the identification code into a dynamic image includes:

[0059] S5.1: Convert the generated real-time motion vector data into motion trajectory point data. Each motion trajectory point contains a timestamp, position coordinates, and motion speed information.

[0060] S5.2: Decompose the obtained predicted trajectory with confidence into predicted trajectory point data according to the timestamp. Each predicted trajectory point contains a timestamp, predicted location coordinates and confidence value.

[0061] S5.3: Integrate the target data after retrieving the identification code to obtain integrated target data; the target data includes the unique identification code of the participating target, the measured trajectory data, and the appearance feature data before occlusion;

[0062] S5.4: Input the motion trajectory point data, predicted trajectory point data, and integrated target data into the dynamic image generation model. The dynamic image generation model synchronously integrates the data based on the time axis, converts the trajectory point data into continuous trajectory lines, and generates a dynamic image of the competition target by combining the appearance feature data of the competition target.

[0063] A dynamic visualization system for sports event information includes: a target binding module, a vector generation module, a trajectory prediction module, an occlusion recognition module, and a dynamic visualization module;

[0064] The target binding module is used to extract target features, generate identification codes, and bind the target when the participating target first enters the sensor monitoring range.

[0065] The vector generation module is used to generate real-time motion vector data based on the competition target with the bound identification code through sensor collaborative acquisition, calibration and data fusion.

[0066] The trajectory prediction module is used to generate a predicted trajectory with confidence by fusing a prediction model when the target of the competition leaves the sensor's field of view.

[0067] The occlusion recognition module is used to perform occlusion pre-judgment, data caching, group trajectory analysis, virtual trajectory generation and identification code retrieval when the participating targets are cross-occluded;

[0068] The dynamic visualization module is used to integrate the output data, fit it into a dynamic image, and complete the visualization display of the event information.

[0069] Compared with the prior art, the beneficial effects of the present invention are:

[0070] 1. This invention proposes a dynamic visualization method for displaying event information. This method can effectively ensure the continuity and accuracy of tracking participating targets, and solve the tracking interruption problems caused by targets going out of sight or cross-occlusion, which are common in events. It achieves target binding by generating a unique identification code through multimodal data fusion, ensuring accurate initial identification. When a target goes out of sight, it generates a predicted trajectory with confidence based on a fusion prediction model combined with real-time data, track environment, and physical parameters to avoid tracking gaps. When cross-occlusion occurs, it efficiently retrieves the target identification code through pre-judgment, snapshot library establishment, group trajectory analysis, and virtual trajectory matching, maintaining stable tracking of participating targets throughout the entire process.

[0071] 2. This invention proposes a method for dynamically visualizing event information. This method improves the comprehensiveness and intuitiveness of the dynamic visualization of event information, optimizes the event participation experience, and generates real-time motion vector data through a collaborative acquisition and synchronization mechanism of three types of sensors. Combined with the predicted trajectory and target data after the identification code is retrieved, the method fits a dynamic image, which can completely present the real-time motion status of the participating target, the predicted trajectory when it is out of sight, and the accurate information after the occlusion is removed. This allows event staff to accurately grasp the event progress, and spectators can also clearly obtain the dynamics of the participating target. Attached Figure Description

[0072] Figure 1This is a schematic diagram of a method for dynamically visualizing event information according to the present invention;

[0073] Figure 2 This is a flowchart illustrating the principle of a dynamic visualization method for displaying event information according to the present invention.

[0074] Figure 3 This is an architecture diagram of a dynamic visualization display system for event information according to the present invention. Detailed Implementation

[0075] Example 1:

[0076] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a method for dynamically visualizing event information, comprising the following steps:

[0077] S1: When the target enters the sensor's monitoring range for the first time, an initial feature label is generated through multimodal data fusion. The initial feature label is then weighted and fused to generate a unique identification code, which is then bound to the target to obtain the target with the bound identification code.

[0078] S2: Based on the binding identification code, the competition target collects and calibrates motion vectors through three types of sensors, and generates real-time motion vector data by combining update frequency and synchronization mechanism;

[0079] S3: If the target of the competition bound with the identification code leaves the sensor's field of view, the fusion prediction model is called. The real-time motion vector data, the pre-stored track environment, and the preset target physical parameters are input into the fusion prediction model to obtain the predicted trajectory with confidence.

[0080] S4: If the participating targets bound to the identification code are cross-occluded, the distance between the participating targets is monitored in real time by the visual camera to make a pre-judgment of occlusion. The motion data and appearance features before occlusion are cached in time to establish a snapshot library before occlusion. Then, the overall outline of the occluded group is obtained by LiDAR and the overall motion vector of the group is measured by millimeter-wave radar to obtain the overall trajectory of the occluded group. Based on the individual motion features of the participating targets in the snapshot library before occlusion, an individual motion prediction model is constructed. The overall trajectory is decomposed into a group trajectory to generate a virtual trajectory. After the occlusion is removed, the virtual trajectory is matched with the measured trajectory collected by the sensor to find the unique identification code bound to the participating target.

[0081] S5: Fit real-time motion vector data, predicted trajectories with confidence, and target data after retrieving the identification code into a dynamic image to complete the dynamic visualization of event information.

[0082] When the target enters the sensor's monitoring range for the first time, initial feature labels are generated through multimodal data fusion, including:

[0083] S1.1: When the target enters the sensor monitoring range for the first time, the three types of sensors will work together; the sensors include a visual camera, a lidar, and a millimeter-wave radar.

[0084] The visual camera acquires appearance image data of the participating target and extracts appearance feature parameters; the appearance feature parameters include the color features, shape features and texture features of the participating target.

[0085] In this invention, an image segmentation algorithm is used to extract appearance feature parameters. However, the image segmentation algorithm is a prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0086] The lidar acquires the three-dimensional contour data of the participating target and extracts contour feature parameters; the contour feature parameters include the length, width, and height parameters of the participating target.

[0087] The millimeter-wave radar acquires the initial velocity data of the participating target and extracts velocity feature parameters; the velocity feature parameters include the instantaneous velocity magnitude and velocity direction of the participating target;

[0088] S1.2: Input the appearance feature parameters, contour feature parameters and velocity feature parameters into a preset data fusion model. The data fusion model performs fusion processing on each feature parameter based on a weighted average algorithm and outputs initial feature labels.

[0089] Furthermore, the specific steps in S1.2 include:

[0090] (1) Unify the dimensions of appearance feature parameters, contour feature parameters and velocity feature parameters to ensure that the three types of parameters have the same data structure and order of magnitude;

[0091] (2) Based on the discriminative power and stability of appearance feature parameters, contour feature parameters, and velocity feature parameters in target recognition, basic weights are assigned to each type of feature parameter. Among them, appearance feature parameters have the highest discriminative power in the initial target recognition due to their rich visual details, and are assigned a higher basic weight; contour feature parameters reflect the three-dimensional structure of the target, have strong stability but fewer details, and are assigned a medium basic weight; velocity feature parameters are easily affected by motion and have low stability, and are assigned a lower basic weight. The specific values ​​of the basic weights are determined through training with historical data. For example, by analyzing a large number of known target recognition cases, the recognition accuracy under different weight combinations is calculated, and the weight combination that achieves the highest accuracy is selected as the initial basic weight, such as 0.5 for appearance features, 0.3 for contour features, and 0.2 for velocity features.

[0092] (3) Multiply the standardized appearance feature parameters, contour feature parameters and velocity feature parameters with their corresponding weights to obtain the weighted feature vector of each type of feature. Then, add the three weighted feature vectors element by element to obtain the fused feature vector. Each element of the fused feature vector is the weighted sum of the corresponding elements of the three types of features.

[0093] (4) The principal component analysis algorithm is used to reduce the dimensionality of the fused feature vector, calculate the covariance matrix of the fused feature vector, solve the eigenvalues ​​and eigenvectors of the covariance matrix, select the principal components with eigenvalues ​​greater than the preset threshold as the new feature dimensions, and obtain the compressed feature vector.

[0094] (5) Convert the compressed feature vector into a structured initial feature label. The label contains metadata such as feature type identifier, feature value range, and feature confidence. The feature type identifier is used to distinguish the original feature categories corresponding to different dimensions in the label. The feature value range marks the normal fluctuation range of each feature dimension, which is used for fault tolerance judgment during subsequent feature matching. The feature confidence reflects the reliability of the label and is calculated based on the acquisition quality of the three types of original features.

[0095] (6) After generating the label, the uniqueness and distinguishability of the label are verified by comparing it with the existing target feature labels in the database. If the similarity between the newly generated label and any existing label exceeds the preset threshold, it is determined that there is a risk of feature confusion. The weight is readjusted or feature data is collected again until the generated initial feature label has sufficient uniqueness and reliability to effectively distinguish different participating targets.

[0096] (7) The performance of the data fusion model is evaluated regularly. Newly collected competition target data is used as the test set. The accuracy, recall and F1 score of the initial feature labels generated by the data fusion model in the target recognition task are calculated. Through performance evaluation and parameter optimization, the data fusion model is ensured to adapt to different competition environments and target types and to stably output high-quality initial feature labels. The calculation formulas of accuracy, recall and F1 score are existing technical content in this field and are not the inventive solution of this application. They will not be elaborated here.

[0097] The specific steps of S2 include:

[0098] S2.1: Based on the target of the competition bound by the identification code, the three types of sensors, namely visual camera, LiDAR and millimeter-wave radar, are activated to enter the real-time monitoring state;

[0099] The visual camera captures motion images of the participating targets bound with identification codes in real time according to a preset acquisition frame rate. Based on the image recognition algorithm, the position coordinates of the participating targets in the image coordinate system are extracted, and the position change of the participating targets in adjacent frames is calculated to obtain the direction and magnitude of the preliminary motion vector. The image recognition algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0100] The lidar scans the three-dimensional position of the participating target in real time, obtains the three-dimensional coordinates of the participating target in the three-dimensional spatial coordinate system, calculates the change of the three-dimensional coordinates per unit time, and obtains the three-dimensional motion vector.

[0101] The millimeter-wave radar monitors the radial and lateral velocities of the participating targets in real time and generates velocity vector data.

[0102] S2.2: Coordinate the preliminary motion vector, three-dimensional motion vector and velocity vector data to establish the timestamp correspondence of the data collected by each sensor. At the same time, call the sensor calibration database to obtain the measurement error parameters of each sensor.

[0103] Furthermore, the specific steps for establishing the timestamp correspondence between the data collected by each sensor include:

[0104] (1) Extract the acquisition timestamps from the preliminary motion vector, three-dimensional motion vector and velocity vector data, clarify the precise acquisition time corresponding to each data, and convert the timestamps of different sensors into the system standard time format;

[0105] (2) Based on the system standard time, construct a timestamp matching window, and determine the window size according to the sensor acquisition frequency;

[0106] (3) Using the timestamp of the initial motion vector as the reference point, search within the matching window for the record with the closest timestamp between the three-dimensional motion vector and the velocity vector to form a time association group of the initial motion vector-three-dimensional motion vector-velocity vector. For data with missing timestamps, based on the association group data of adjacent moments, generate the virtual three-dimensional motion vector at that moment through linear interpolation to ensure the continuity of time association. At the same time, establish dynamic association rules: when the sensor acquisition frequency changes, such as when the target moves quickly and triggers high-frequency acquisition by the vision camera, automatically adjust the size of the matching window; when the timestamp deviation exceeds the preset threshold, mark it as an abnormal association and start sensor clock synchronization correction.

[0107] Furthermore, the sensor calibration database is accessed to obtain the measurement error parameters for each sensor, including:

[0108] (1) Retrieve the corresponding calibration record based on the unique identifier of the sensor. The measurement error parameters stored in the calibration database are classified by type, including systematic errors and random errors. Systematic errors include the distance offset of the lidar and the velocity measurement deviation of the millimeter-wave radar. Random errors include the pixel jitter standard deviation of the vision camera and the time synchronization error range of each sensor. For the vision camera corresponding to the initial motion vector, extract its lens distortion correction parameters and the conversion error from pixel coordinates to physical coordinates. For the lidar corresponding to the three-dimensional motion vector, extract its angle measurement error and distance resolution error. For the millimeter-wave radar corresponding to the velocity vector, extract its Doppler frequency offset error and velocity measurement accuracy level. At the same time, obtain the correction coefficient of each sensor's error as the environment changes.

[0109] (2) Verify whether the extracted measurement error parameters are within the validity period. Each record in the sensor calibration database contains the calibration time and validity period. If the error parameter has expired, the system will automatically issue a calibration reminder and call the most recent calibration record as a temporary replacement parameter, while marking the reliability level of the parameter as low. If the sensor has undergone maintenance or severe vibration during use, the parameter will be forcibly recalibrated. The error parameters will be remeasured and updated to the calibration database through a standard test environment, such as a calibration plate with a known distance or a test target with uniform motion. For sensors that cannot be recalibrated in time, the current error parameters will be estimated through an error prediction model based on their historical error data and current environmental parameters, such as temperature and humidity, to ensure that the error parameters used for data processing always reflect the actual measurement status of the sensor.

[0110] (3) The multi-source motion vector data with established timestamp correspondence is associated and stored with the extracted sensor error parameters to form a complete data record containing data content, timestamp, sensor identifier, error parameters and associated status.

[0111] S2.3: Based on the measurement error parameters, error compensation is performed on the obtained preliminary motion vector, three-dimensional motion vector and velocity vector data respectively. The compensation method is to superimpose the preliminary motion vector, three-dimensional motion vector and velocity vector data with the corresponding error correction values ​​respectively.

[0112] S2.4: Input the compensated preliminary motion vector, three-dimensional motion vector, and velocity vector data into the motion vector fusion model; the motion vector fusion model uses the Kalman filter algorithm to fuse the preliminary motion vector, three-dimensional motion vector, and velocity vector data, and outputs the calibrated motion vector. The Kalman filter algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0113] S2.5: Preset the update frequency of motion vector data, and establish a time synchronization protocol between sensors. Based on the time synchronization protocol, calibrate the clock of each sensor. The update frequency is determined according to the type of competition and the speed of the target. The calibration process of the time synchronization protocol is existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0114] S2.6: Periodically collect the calibrated motion vector according to the update frequency, and record the collection timestamp after each collection;

[0115] S2.7: Based on the time synchronization protocol between sensors, the time synchronization verification is performed on the calibrated motion vector and the corresponding acquisition timestamp for each acquisition. If the time deviation exceeds the preset deviation threshold, the acquisition data is corrected for time offset.

[0116] Furthermore, the specific steps in S2.7 include:

[0117] (1) Configure the core parameters of the time synchronization protocol according to the type and acquisition frequency of the sensor, including: determining the unified time reference of the system, such as using the standard time provided by the network time protocol server, setting the time synchronization period of each sensor, such as performing time calibration every 100 milliseconds, specifying the format of the timestamp, such as UTC time accurate to microseconds, and assigning a unique time synchronization identifier to each sensor. Pre-set the master-slave synchronization relationship in the protocol, such as using the lidar with the highest acquisition frequency as the master sensor, and the vision camera and millimeter-wave radar as slave sensors. The slave sensors need to send time synchronization requests to the master sensor periodically.

[0118] (2) Extract the corresponding acquisition timestamps from the calibrated motion vector data acquired each time, and organize them into time series according to sensor type. For example, extract the timestamp sequence from the three-dimensional motion vector of the lidar, extract another time series from the initial motion vector of the vision camera, and extract a third time series from the velocity vector of the millimeter-wave radar. Check whether the format of each timestamp conforms to the protocol. If there is a format error, it is filled in based on the historical correspondence between the internal clock of the sensor and the system time.

[0119] (3) Using the timestamp of the main sensor as a reference, the timestamp of the slave sensor is synchronized and verified, including: in each time synchronization cycle, the timestamp of any moment of the main sensor is selected as a reference point, the closest timestamp is found in the time series of the slave sensor, and the difference between the two is the time deviation. The time deviation of multiple consecutive time synchronization cycles is statistically analyzed, and the average deviation and deviation fluctuation range are calculated. If the average deviation is stable within the preset threshold, such as within 3 milliseconds, and the fluctuation is small, such as the standard deviation is less than 1 millisecond, the time synchronization status is judged to be good; if the single deviation or average deviation exceeds the preset threshold, or the fluctuation range is too large, such as the standard deviation is greater than 2 milliseconds, the time synchronization is judged to be abnormal.

[0120] (4) For sensor data with time synchronization anomalies, a correction model is used for correction, including:

[0121] If the time deviation is a fixed value, a constant offset correction model is used, which directly adds a fixed deviation value to all timestamps of the sensor. If the deviation changes linearly with time, a linear offset correction model is used, which fits the linear relationship between the deviation and time using the least squares method to obtain the deviation calculation formula. If the deviation exhibits non-linear fluctuations, a sliding window correction model is used, which predicts the current deviation based on the deviation values ​​of the most recent N synchronization cycles. The window size is determined according to the fluctuation frequency, such as 5 cycles when fluctuations are frequent and 10 cycles when the fluctuation is stable. The parameters of the correction model are obtained by training with historical deviation data. The least squares method is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0122] (5) Correcting the timestamps of abnormal sensors: For constant offset, directly add / subtract a fixed value to each timestamp of the slave sensor; for linear offset, substitute the timestamp into the deviation calculation formula to obtain the correction amount, and then adjust the original timestamp; for sliding window correction, dynamically correct the timestamp according to the predicted current deviation; after correction, recalculate the deviation from the timestamp of the master sensor.

[0123] (6) Detailed information on time offset correction, including the deviation value before correction, the correction model used, the deviation value after correction, and the correction time, are associated with the corresponding motion vector data and stored to form a complete time synchronization log, which is convenient for traceability and troubleshooting.

[0124] S2.8: Arrange the calibrated motion vectors, which have undergone time synchronization verification and correction, in the order of the acquisition timestamps to generate real-time motion vector data.

[0125] The specific steps of S3 include:

[0126] S3.1: Real-time monitoring of the acquisition status of the three types of sensors on the competition targets with bound identification codes;

[0127] If the visual camera fails to capture the appearance image data of the participating target for a consecutive preset number of frames, the lidar fails to scan the three-dimensional contour data of the participating target for a consecutive preset time, and the millimeter-wave radar fails to detect the speed data of the participating target for a consecutive preset period, then the corresponding participating target is determined to be out of the sensor's field of view.

[0128] If any sensor can still collect data on the target, then continue with the step of generating real-time motion vector data;

[0129] S3.2: Collect 3D terrain data, track boundary data, track obstacle distribution data, and track marker data of the race track in advance to form a track environment model and store it in the track environment database;

[0130] Once the target object leaves the sensor's field of view, the track environment model is retrieved from the track environment database as the track environment data.

[0131] S3.3: Pre-set the target physical parameters and store them in the target physical parameter database; the target physical parameters include the mass parameters, inertial parameters, maximum speed limit parameters, and acceleration limit parameters of the participating target;

[0132] Once the target moves out of the sensor's field of view, the preset target physical parameters corresponding to the target are retrieved from the target physical parameter database.

[0133] S3.4: The generated real-time motion vector data, the retrieved track environment data, and the retrieved preset target physical parameters are preprocessed according to the input format requirements of the fusion prediction model to form target fusion data and input into the fusion prediction model.

[0134] S3.5: The fusion prediction model first inputs the target fusion data into the LSTM network. The LSTM network learns the motion patterns of the participating target based on historical motion data and outputs a preliminary predicted trajectory. Then, the preliminary predicted trajectory is input into the particle filter algorithm module. The particle filter algorithm module optimizes and adjusts the preliminary predicted trajectory by combining the obstacle constraints in the track environment and the motion restrictions in the target's physical parameters. At the same time, the fusion prediction model calculates the confidence level of the predicted trajectory based on the completeness and accuracy of the input target fusion data and outputs the optimized predicted trajectory with confidence.

[0135] Furthermore, the specific steps of S3.5 include:

[0136] (1) Preprocess the target fusion data of the input fusion prediction model, convert the preprocessed target fusion data into the input format required by the LSTM network, divide the data into segments according to a fixed time window, and each segment contains features such as position coordinates, velocity components, and acceleration components within the window to form a three-dimensional input tensor.

[0137] (2) Configure the hierarchical structure of the LSTM network, match the input layer dimension with the feature dimension of the target fusion data, and after training, input the preprocessed target fusion data into the LSTM network. The LSTM network calculates and outputs the position and velocity prediction values ​​for a preset time period through forward propagation to form a preliminary prediction trajectory. The LSTM network is the existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0138] (3) The particle set of the particle filtering algorithm is initialized based on the preliminary predicted trajectory. Each particle represents a trajectory hypothesis. The number of particles is set according to the accuracy requirements of trajectory prediction. For example, 500 particles are set for high-precision scenarios and 200 particles are set for conventional scenarios. The initial state of each particle, such as position and velocity, is generated by superimposing random perturbation on the corresponding time value of the preliminary predicted trajectory. The perturbation amplitude is proportional to the prediction error of the LSTM network. The particle filtering algorithm is the prior art in this field and is not an inventive solution of this application. It will not be described in detail here.

[0139] (4) Construct an obstacle constraint model for the track environment, convert fixed obstacles in the track map, such as guardrails, roadblocks, and curve markers, into spatial coordinate boundaries, and update dynamic obstacles in real time, such as the position and movement trend of the remaining participating targets, predict the spatial range they may occupy, and perform collision detection on each particle trajectory in the particle set at each time step: determine whether the position point on the trajectory falls within the spatial boundary of the obstacle. If the position overlaps with a fixed obstacle or the predicted range of a dynamic obstacle at any time, the particle trajectory is determined to violate the obstacle constraint and its weight is reduced; if the trajectory always stays in the safe passage area, its weight is maintained or increased, and reasonable trajectories that meet the track environment constraints are selected through weight adjustment.

[0140] (5) Call the mass, inertia, maximum velocity, and acceleration limit parameters in the target physical parameter database to apply motion constraint on the particle trajectory. For the velocity parameter of each particle, if the velocity value exceeds the maximum velocity limit at any time, it is proportionally reduced to 90% of the maximum velocity. For the acceleration parameter, calculate the velocity change rate between adjacent time moments. If it exceeds the acceleration limit, adjust the velocity change to meet the limit and simultaneously correct the position prediction value. Combine the inertia parameter to adjust the trajectory turning smoothness by increasing the time step during the turning process.

[0141] (6) After completing the obstacle constraint and physical restriction correction, calculate the weight of each particle. The weight is determined by three parts: the similarity with the initial LSTM predicted trajectory, the degree to which the obstacle constraint is satisfied, and the degree to which the physical parameter restriction is met. The particle weights are then normalized to ensure that the sum of all weights is one.

[0142] (7) A resampling algorithm is used to filter particles, retain high-weight particles and copy them, and remove low-weight particles, so that the particle set focuses on a more likely trajectory region and avoids particle degradation. The resampling algorithm is the prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0143] (8) The resampled particle trajectories are fused, and the weighted average of the positions of all particles at each prediction time is calculated to obtain the optimized trajectory;

[0144] (9) The fusion prediction model calculates the confidence score from two dimensions: completeness and accuracy of the target fusion data. In terms of completeness, the proportion of valid records in the input data is counted. The more missing data, the lower the completeness score and the lower the confidence score. In terms of accuracy, the deviation between the sensor measurement values ​​and calibration values ​​in the input data is compared. The smaller the deviation, the higher the accuracy score. The completeness score and accuracy score are weighted and summed at a ratio of 7:3 to obtain the initial confidence score. Then, the confidence score is adjusted according to the dispersion of the optimized trajectory and particle set: the dispersion is small, the confidence score is high, and the dispersion is large, the confidence score is low. The final output is a predicted trajectory with confidence, which includes the position coordinates, velocity vector and corresponding confidence score value at each time step. The dispersion is measured by the standard deviation of the particle position.

[0145] The specific steps of S4 include:

[0146] S4.1: The visual camera acquires image data of multiple participating targets with bound identification codes in real time, and segments each participating target into independent target regions in the image based on the image segmentation algorithm, and calculates the distance between the target regions corresponding to adjacent participating targets in the image coordinate system; the distance includes horizontal distance and vertical distance;

[0147] Furthermore, the horizontal distance is the absolute value of the difference between the center coordinates of the two competing targets on the horizontal axis of the image, and the vertical distance is the absolute value of the difference on the vertical axis.

[0148] S4.2: Compare the calculated distance with the preset occlusion determination threshold;

[0149] If the distance between all adjacent participating targets is greater than the occlusion detection threshold, it is determined that there is no risk of cross occlusion, and the step of generating real-time motion vector data continues.

[0150] If the distance between any adjacent participating targets is less than or equal to the occlusion detection threshold, it is determined that there is a risk of cross occlusion and an occlusion warning signal is issued.

[0151] S4.3: Upon receiving the occlusion warning signal, the data caching module is activated; the data caching module captures the latest motion data of the corresponding competition target from the generated real-time motion vector data in real time, and simultaneously captures the latest appearance feature data of the competition target collected by the visual camera, i.e., the appearance feature parameters, and associates and stores the captured latest motion data and the latest appearance feature data according to the timestamp to form the snapshot data of the competition target before occlusion; the latest motion data includes motion speed, motion direction and motion position;

[0152] S4.4: Perform data capture and associated storage operations on all participating targets that have a risk of cross occlusion, summarize the snapshot data of each participating target before occlusion, and establish a snapshot database before occlusion; each snapshot data in the snapshot database before occlusion contains the unique identification code of the participating target, the corresponding relationship between the motion data before occlusion and the appearance feature data before occlusion;

[0153] S4.5: The lidar performs a panoramic scan of the area where there is cross-occlusion, collects three-dimensional point cloud data of the occlusion group composed of all participating targets in the area, and preprocesses the three-dimensional point cloud data. Based on the point cloud clustering algorithm, the preprocessed three-dimensional point cloud data is clustered to obtain the three-dimensional overall outline of the occlusion group. The three-dimensional overall outline includes the outer cuboid size of the occlusion group and the overall center position. The point cloud clustering algorithm is the prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0154] S4.6: The millimeter-wave radar continuously monitors the obstructing group and collects the overall speed data of the obstructing group; the overall speed data includes the magnitude of the overall movement speed and the overall movement direction;

[0155] S4.7: Based on the overall velocity data and the overall center position obtained by the lidar, calculate the change in the overall center position of the occlusion group per unit time to obtain the overall motion vector of the group;

[0156] Furthermore, the specific steps in S4.7 include:

[0157] (1) Collect the overall velocity data of the obstructing group acquired by millimeter-wave radar, arrange them in the order of timestamps to form a continuous velocity time series, and each velocity data contains information on velocity magnitude and direction of motion;

[0158] (2) Extract the overall center position coordinates of each frame from the three-dimensional overall contour data of the occluded group obtained from the lidar. The center position coordinates are usually the geometric center of the contour, i.e. the average value of the coordinates of all contour points. Transform the center position coordinates to a unified spatial coordinate system, such as the global coordinate system of the competition venue, to eliminate the coordinate deviation caused by the lidar installation position and angle. The specific coordinate transformation process is the existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0159] (3) Based on the sensor acquisition frequency and the motion characteristics of the occluded group, the unit time interval for calculating the change is dynamically determined. If the group moves quickly, such as more than 5 m / s, a smaller unit time, such as 100 ms, is selected to capture rapid position changes. If the movement is slow, such as less than 1 m / s, the unit time is increased, such as 200 ms, to reduce the amount of calculation and avoid introducing too much noise. Based on the determined unit time, the aligned center position time series is divided into continuous time windows. Each window contains center position data at two adjacent times, such as time t and time t+unit time. Each window corresponds to the position change calculation within a unit time interval.

[0160] (4) Calculate the spatial difference between the coordinates of the two center positions within each time window to obtain the change in position per unit time. At the same time, calculate the absolute value of the change in position, where the absolute value reflects the distance the group moves per unit time.

[0161] (5) Combine the horizontal and vertical changes per unit time into the overall motion vector of the group. The magnitude of the overall motion vector is the square root of the sum of the squares of the changes in each direction, that is, the distance of the spatial displacement. The direction of the overall motion vector is determined by the ratio of the changes in each direction. For example, the ratio of the horizontal change to the vertical change corresponds to the angle between the motion direction and the x-axis.

[0162] (6) Add a corresponding timestamp to each calculated overall motion vector, wherein the timestamp is the timestamp of the end of the time window, and the time period during which the motion represented by the vector occurs is clearly defined;

[0163] (7) The overall motion vector is associated with the corresponding unit time, overall velocity data, center position change and verification results to form a complete motion vector record. The records are stored in chronological order. At the same time, the occlusion group identification information is added to the record, such as the identification code of which competition targets are included, to ensure that the overall motion vector is accurately bound to the specific occlusion group and to avoid data confusion between different groups.

[0164] S4.8: Based on the overall motion vector and time series, generate the overall trajectory of the occluded group in the track environment; the overall trajectory includes the overall position coordinates of the occluded group at different time points;

[0165] Furthermore, the specific steps of S4.8 include:

[0166] (1) Deeply correlate the overall motion vector with the corresponding time series to ensure that each motion vector accurately corresponds to the time point in time when it occurs;

[0167] (2) Establish a global coordinate system for the track environment, with the starting point of the track as the origin, the direction of movement along the track as the X-axis, the direction perpendicular to the track as the Y-axis, and the direction perpendicular to the ground as the Z-axis. Convert the position data collected by the visual camera, lidar, and millimeter-wave radar sensors to this global coordinate system and eliminate coordinate deviations through sensor extrinsic parameters.

[0168] (3) Determine the initial overall position coordinates of the occlusion group at the start of the time series. The initial overall position coordinates are usually taken from the overall center position when the lidar first detects the occlusion group.

[0169] (4) Starting from the initial overall position coordinates, the overall position coordinates of the occlusion group are calculated recursively according to each overall motion vector in the time series. For the nth time point in the time series, its position coordinates are equal to the position coordinates of the (n-1)th time point plus the displacement component of the (n-1)th motion vector in a unit time, thus obtaining continuous position coordinates.

[0170] (5) Perform continuity verification on the continuous position coordinates obtained by recursion calculation, and check whether the position changes of adjacent time points conform to the physical laws of the overall motion vector, including: calculating the distance difference between adjacent coordinates. The distance difference should be basically matched with the magnitude and time interval of the corresponding motion vector. If the deviation exceeds 20%, it is judged as a trajectory abnormality. Analyze the cause of the abnormality: if it is due to a sudden change in the motion vector, use the moving average method to correct the vector so that it is consistent with the trend of the vector before and after; if it is due to the cumulative error of coordinate recursion, select a known reliable position point in the trajectory, such as the coordinates when the group passes through a certain track mark as the anchor point, and recalibrate the position coordinates before and after the anchor point to eliminate the cumulative deviation.

[0171] (6) Compare the calculated overall trajectory with the track environment constraints to ensure that the trajectory is always within the feasible area of ​​the track. This includes: calling the boundary data in the track environment map and checking whether the trajectory coordinates exceed the constraint range at each time step. If the coordinates fall within the guardrail range at any time step, the position is corrected according to the track width and movement direction: the coordinates are adjusted towards the center of the track, and the adjustment distance is the distance beyond the boundary plus safety redundancy. If the trajectory crosses the prohibited area, the corrected trajectory is generated by curve fitting based on the reasonable position at the previous and next times.

[0172] (7) Label the corresponding time point for each position coordinate in the overall trajectory to form a time-position correlation data pair, and label the additional information of each time point, including the magnitude, direction and trajectory reliability of the overall motion vector at that moment;

[0173] (8) Generate a visual preview of the overall trajectory, draw continuous position coordinates on the track environment map and connect them into a line to intuitively show the movement path of the occluded group, including: checking whether there are obvious unreasonable jumps or distortions in the trajectory through the preview. If a problem is found, backtrack to the calculation process of the corresponding time point for correction; after the verification is completed, optimize the storage of trajectory data: use a time-series database to store time-location related data pairs, and support fast query by time range; compress the trajectory to retain detailed data of key turning points, such as turning points and speed changes, and use interval sampling storage for uniform linear motion segments, such as storing 1 time point every 5 time points, so as to reduce storage space occupation while ensuring trajectory accuracy; the final stored overall trajectory must contain a complete time series and corresponding position coordinates.

[0174] S4.9: Retrieve the motion data of each participating target before occlusion from the snapshot library before occlusion, and extract the individual motion characteristics of each participating target based on the motion data before occlusion; the motion data before occlusion includes the trend of motion speed change, the law of motion direction change, and the trajectory of motion position change within a preset time period before occlusion; the individual motion characteristics include motion acceleration characteristics, turning frequency characteristics, and linear motion preference characteristics;

[0175] S4.10: Combine the individual motion characteristics of each participating target with the preset basic motion model to construct an individual motion prediction model for each participating target; the basic motion model adopts a variable acceleration motion model, wherein the variable acceleration motion model is the prior art in this field and is not an inventive solution of this application, and will not be described in detail here.

[0176] Furthermore, the specific steps of S4.10 include:

[0177] (1) Extract individual motion features of each participating target from the snapshot database before occlusion, covering motion state parameters and motion pattern rules. Among them, motion state parameters include the position coordinate sequence, instantaneous velocity value, acceleration value and corresponding timestamp in the continuous time period before occlusion. The statistical characteristics of these parameters are calculated to achieve quantification. The motion pattern rules are obtained by analyzing the geometric characteristics of historical trajectories. Each feature parameter is standardized and the values ​​are mapped to the range of 0 to 1 to form an individual motion feature vector containing velocity characteristics, acceleration characteristics and turning characteristics.

[0178] (2) Clarify the core parameters and physical meaning of the variable acceleration motion model. The variable acceleration motion model should be able to describe the motion law of acceleration changing with time, including four key parameters: initial acceleration, rate of change of acceleration, maximum acceleration threshold, and minimum acceleration threshold. The initial acceleration is the acceleration value at the moment the model starts, reflecting the initial dynamic characteristics of the target motion; the rate of change of acceleration describes the trend of acceleration changing with time, with a positive value indicating increased acceleration capability and a negative value indicating decreased acceleration capability; the maximum and minimum acceleration thresholds are set according to the physical parameters of the participating target to limit the reasonable range of acceleration and ensure that the model meets the physical constraints.

[0179] (3) Establish the mapping relationship between individual motion feature vectors and variable acceleration motion model parameters so that the basic model can adapt to the motion characteristics of different participating targets. For example, for speed characteristics, if the historical average speed of the participating target is high and the fluctuation is small, the acceleration change rate is reduced so that the speed change predicted by the model is smoother; if the speed fluctuation is large, the acceleration change rate is increased; for steering characteristics, the direction of the acceleration change rate is adjusted by analyzing the lateral acceleration during steering. When turning left, the lateral acceleration is positive, and the acceleration change rate of the model in the corresponding time period can be reduced to avoid excessive speed during steering.

[0180] (4) The initial model parameters obtained by mapping are optimized using the historical motion data before occlusion to obtain the individual motion prediction model of each participating target. The least squares method is used to minimize the error between the predicted trajectory output by the model and the historical actual trajectory. The least squares method is the existing technology in this field and is not an inventive solution of this application. It will not be elaborated here.

[0181] S4.11: The obtained overall trajectory of the occluded group is divided into multiple trajectory segments according to the timestamp. Each trajectory segment corresponds to a time interval. For each trajectory segment, based on the constructed individual motion prediction model of each participating target, the suspected motion trajectory of each participating target within the time interval is predicted. At the same time, combined with the overall contour constraint of the occluded group, the suspected motion trajectory of each participating target is adjusted. The overall contour constraint of the occluded group means that the predicted motion trajectory of each participating target cannot exceed the spatial range of the overall contour.

[0182] Furthermore, the specific steps of S4.11 include:

[0183] (1) Analyze all timestamps contained in the overall trajectory, determine the time span and sampling interval of the trajectory, and divide the overall trajectory into continuous and non-overlapping trajectory segments according to the uniformity of the sampling interval. Each trajectory segment contains the start position, end position and corresponding timestamp information within the time interval.

[0184] (2) For each time interval corresponding to each trajectory segment, adjust the time window parameters of the individual motion prediction model of each participating target so that the prediction step size of the individual motion prediction model matches the time length of the trajectory segment.

[0185] (3) Retrieve the variable acceleration motion model parameters of each target from the model parameter library, and initialize the model input by combining the overall motion state of the group at the beginning of the trajectory segment;

[0186] (4) Call the individual motion prediction model of each participating target to predict the motion trajectory within each trajectory segment. The model calculates the position change of the target within the segment based on the initialized parameters and time interval: starting from the suspected position at the beginning of the trajectory segment, the acceleration and velocity changes of each tiny time step are calculated by combining the variable acceleration motion model, and the position at the end time is obtained step by step to form a continuous suspected motion trajectory. At the same time, in order to cover possible motion uncertainties, multiple suspected motion trajectories are generated for each target. Each trajectory corresponds to the fine adjustment of the model parameters, reflecting the possible motion deviation of the target within the group, such as different motion states near the center or edge of the group. Each suspected motion trajectory must contain detailed information such as timestamp, position coordinates, and velocity vector, and the trajectory length is consistent with the time interval of the trajectory segment.

[0187] (5) Extract the overall contour parameters of the group corresponding to each trajectory segment from the lidar data, transform the overall contour parameters of the group to the same global coordinate system as the individual trajectory, and establish a contour constraint model: the outer rectangle of the contour defines the spatial boundary of the group motion, and the suspected motion trajectory of any individual must not exceed the range of the rectangle; the contour area is used to estimate the distribution density of targets within the group. The smaller the area, the more concentrated the target distribution and the smaller the allowable offset of the individual trajectory; the contour boundary point set is used for accurate collision detection to ensure that the individual trajectory does not conflict with the contour boundary.

[0188] (6) Perform point-by-point spatial conflict detection on each suspected motion trajectory to determine whether each position point on the trajectory conforms to the overall contour constraint. First, check whether the position point is within the bounding rectangle of the contour. If it is outside the range, mark it as a boundary conflict. Second, calculate the distance between the position point and the contour boundary. If the distance is less than the safety threshold, such as 0.2 meters, mark it as a boundary proximity conflict. Finally, combine the suspected motion trajectories of other targets in the group to check whether there is positional overlap between individuals. If the distance between the trajectory points of two targets is less than 1 / 2 of the target's own size, mark it as an internal conflict. For trajectories with conflicts, record the time point of the conflict, the type of conflict, and the degree of deviation.

[0189] (7) Based on the conflict detection results, targeted adjustment strategies are adopted for different types of conflict trajectories. For trajectories with boundary conflicts, the trajectory is shifted towards the center of the group by a distance exceeding the boundary plus a safety redundancy, such as 0.1 meters. At the same time, the direction of movement is adjusted so that the trajectory gradually returns to the outline range at subsequent time points. If the shift causes conflict with other target trajectories, the trajectory that is more consistent with the overall movement direction of the group is retained first, and the speed of the other trajectory is adjusted. For trajectories with conflicts near the boundary, the acceleration parameters are finely adjusted so that the trajectory is slightly shifted towards the outline center to avoid touching the boundary. For trajectories with internal conflicts, the turning angle of one of the trajectories is adjusted according to the movement priority of the target, such as increasing the turning angle by 5 degrees, so that the two trajectories are misaligned in time and space.

[0190] (8) Select the optimal motion trajectory from the multiple adjusted suspected motion trajectories. The selection criteria include: the degree of matching with the overall motion vector of the group, the smoothness of the trajectory, and the duration of no conflict. Assign a comprehensive score to each trajectory, which is the weighted sum of the various selection criteria. Select the trajectory with the highest score as the final suspected motion trajectory of the target in the current trajectory segment. Piece together the final suspected motion trajectories of all targets in each trajectory segment in the order of timestamps to form a complete sequence of individual suspected motion trajectories during the occlusion period. Each individual suspected motion trajectory sequence contains the position, velocity, and acceleration information of continuous time points and marks the adjustment record of each trajectory segment.

[0191] S4.12: A trajectory segmentation algorithm is used to match the adjusted suspected motion trajectories of each participating target with the overall trajectory segment to determine the specific motion path of each participating target in the overall trajectory segment. The specific motion paths of the participating targets determined in each time interval are connected in chronological order to generate a virtual trajectory for each participating target. The virtual trajectory includes the predicted position coordinates of the participating target at each time point during the occlusion period. The trajectory allocation algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0192] S4.13: The visual camera monitors the image data of the occluded area in real time. If the independent appearance image data of each participating target is re-captured, and the lidar re-scans the independent three-dimensional contour data of each participating target, and the millimeter-wave radar re-monitors the independent velocity data of each participating target, then the occlusion is determined to be lifted.

[0193] S4.14: After the occlusion is removed, the motion vectors of each participating target are collected and calibrated through the collaborative acquisition of three types of sensors, and the measured motion vector data of each participating target is generated. The measured trajectory is generated based on the measured motion vector data.

[0194] Furthermore, the specific steps in S4.14 include:

[0195] (1) The occlusion removal event is detected in real time through image analysis of the visual camera. When the originally overlapping target areas in multiple consecutive frames of images are clearly separated and the identification codes of each target can be clearly identified, the occlusion state is determined to be removed. The sensor collaborative acquisition mechanism is immediately triggered to wake up the lidar and millimeter-wave radar in low power state, and the precise timestamp of the occlusion removal is recorded at the same time.

[0196] (2) The visual camera acquires image sequences containing each participating target and extracts the pixel position and motion direction of the target in each frame; the lidar outputs the three-dimensional point cloud data of the target, including spatial coordinates and reflection intensity; the millimeter-wave radar provides the radial velocity, distance and azimuth of the target. The raw data of the three types of sensors are processed for time synchronization. Based on a high-precision clock, the acquisition timestamps of different sensors are unified to the same time axis.

[0197] (3) Target detection and tracking are performed on the images captured by the vision camera. The identity of each participating target is determined by identification code matching. The bounding box of the participating target is extracted by the image segmentation algorithm. The pixel coordinates of the center of the bounding box are calculated. Combined with the camera intrinsic parameters, such as focal length and principal point coordinates, the pixel coordinates are converted into the position in the image coordinate system. Then, the extrinsic parameters are used to convert it to the global coordinate system to obtain the spatial position of the target. Based on the position change of two consecutive frames, the displacement within the time interval is calculated. The initial value of the motion vector is obtained by combining the timestamp difference.

[0198] (4) The point cloud data of the lidar is preprocessed to remove ground points and noise points. The DBSCAN clustering algorithm is used to cluster the remaining point cloud according to spatial distance, and each cluster corresponds to a competition target. The geometric center of each cluster is calculated as the three-dimensional spatial coordinates of the competition target and compared with the position obtained by the visual camera. If the deviation exceeds 0.3 meters, it is checked whether the point cloud is missing due to partial occlusion of the competition target. Morphological completion is used for the missing area. Based on the change of the center coordinates at continuous time, the motion vector of the target is calculated, and the longitudinal and lateral velocity components are extracted. The velocity direction is corrected by using the distribution range of the point cloud. The DBSCAN clustering algorithm is the prior art in this field and is not an inventive solution of this application. It will not be described in detail here.

[0199] (5) Extract the radial velocity and azimuth of each target in the millimeter-wave radar data, and convert the azimuth into the direction angle in the global coordinate system in combination with the radar installation position. The conversion process is achieved by trigonometric functions, which are existing technologies in this field and are not the inventive solution of this application. They will not be described in detail here.

[0200] (6) Establish a collaborative calibration model for sensor data, assign weights according to the accuracy characteristics of the sensors in different scenarios, and perform weighted fusion of position and velocity data of the same target to obtain the calibrated motion vector;

[0201] (7) Arrange the calibrated motion vectors in the order of timestamps to form a continuous time series, namely the calibrated motion vector time series, wherein each time series contains target identification code, timestamp, position coordinates, velocity magnitude, velocity direction and acceleration information;

[0202] (8) Starting from the first effective time point after the occlusion is removed, connect the calibrated motion vector time series in chronological order to generate the measured trajectory.

[0203] S4.15: Calculate the similarity between the virtual trajectory and the measured trajectory, and match the virtual trajectory with the highest similarity with the measured trajectory to retrieve the unique identification code bound to the competition target; the similarity is determined by calculating the average Euclidean distance between the position coordinates of the corresponding time points of the two trajectories; the unique identification code is contained in the snapshot data before occlusion associated with the individual motion prediction model of the competition target corresponding to the virtual trajectory.

[0204] The process of fitting real-time motion vector data, predicted trajectories with confidence levels, and target data after retrieving the identification code into a dynamic image includes:

[0205] S5.1: Convert the generated real-time motion vector data into motion trajectory point data. Each motion trajectory point contains a timestamp, position coordinates, and motion speed information.

[0206] S5.2: Decompose the obtained predicted trajectory with confidence into predicted trajectory point data according to the timestamp. Each predicted trajectory point contains a timestamp, predicted location coordinates and confidence value.

[0207] S5.3: Integrate the target data after retrieving the identification code to obtain integrated target data; the target data includes the unique identification code of the participating target, the measured trajectory data, and the appearance feature data before occlusion;

[0208] S5.4: Input the motion trajectory point data, predicted trajectory point data, and integrated target data into the dynamic image generation model. The dynamic image generation model synchronously integrates the data based on the time axis, converts the trajectory point data into continuous trajectory lines, and generates a dynamic image of the competition target by combining the appearance feature data of the competition target.

[0209] Furthermore, the specific steps in S5.4 include:

[0210] (1) Analyze the motion trajectory point data, predicted trajectory point data and integrated target data, extract core fields, and form a standardized dataset with consistent format;

[0211] (2) Based on the timestamp range and sampling frequency of the standardized dataset, configure the model output parameters, including resolution, frame rate, and viewpoint, and construct a timeline that matches the data sampling frequency. Insert uniform intermediate time points between consecutive timestamps according to the frame rate requirements to ensure that the timeline covers all data periods and that each time point corresponds to a unique image frame, thus forming a dynamic image generation model;

[0212] (3) Based on the constructed time axis, the motion trajectory points, predicted trajectory points and target appearance data in the standardized dataset are matched by timestamps, and the known data points on the time axis are directly associated with the corresponding information; for intermediate time points, linear interpolation is used to generate position coordinates and velocity vectors; if there are both measured and predicted trajectories in any time period, they are fused according to time weight, the confidence of the predicted trajectory is marked, and a time-continuous integrated dataset is formed.

[0213] (4) Based on the trajectory point sequence of the integrated dataset, continuous trajectory lines are generated by Bézier curve fitting, and visual attributes are defined according to trajectory type: the measured trajectory is a solid line, the predicted trajectory is a dashed line, and arrows are added to indicate the direction of movement, forming trajectory line data with visual features;

[0214] (5) Based on the target appearance feature parameters in the integrated dataset, including size, shape, color and logo, construct a three-dimensional basic model, combine the velocity direction and acceleration value in the trajectory line data, adjust the model posture in real time, so that the posture of the three-dimensional basic model is consistent with the motion state, and generate three-dimensional model data with posture parameters.

[0215] (6) Import the 3D environment model of the competition venue as the base layer, overlay the trajectory line data and 3D model data onto the base layer according to the spatial coordinates, add auxiliary information layers, sort the layers according to spatial depth, and form layered image data. Among them, the auxiliary information layers include timestamps, competition target identification code labels, and speed values. The layer sorting order is scene layer - trajectory layer - model layer - information layer.

[0216] (7) Generate dynamic images frame by frame based on layered image data in chronological order, calculate the position difference and attitude angle difference between adjacent frames, insert a gradual transition for changes exceeding the visual threshold, so as to smooth the target movement and attitude adjustment, and generate a continuous dynamic image sequence.

[0217] Example 2:

[0218] Please see Figure 3 Another embodiment of the present invention provides: a dynamic visualization display system for event information, comprising:

[0219] Target binding module, vector generation module, trajectory prediction module, occlusion recognition module, and dynamic visualization module;

[0220] The target binding module is used to extract target features, generate identification codes, and bind targets when they first enter the sensor's monitoring range.

[0221] The vector generation module is used to generate continuous and accurate real-time motion vector data based on the competition target with the bound identification code through sensor collaborative acquisition, calibration and data fusion.

[0222] The trajectory prediction module is used to generate a predicted trajectory with confidence by fusing prediction models when the target leaves the sensor's field of view, thus avoiding interruption of target tracking.

[0223] The occlusion recognition module is used to perform occlusion pre-judgment, data caching, group trajectory analysis, virtual trajectory generation and identification code retrieval when the participating targets are cross-occluded, so as to ensure that target tracking is not interrupted after the occlusion is removed;

[0224] The dynamic visualization module integrates the data output from each module, fits it into a dynamic image, and completes the visualization display of event information, forming a closed loop in the system.

[0225] The vector generation module includes: an acquisition unit, a correlation and calibration unit, a motion vector fusion unit, and a vector integration unit;

[0226] The acquisition unit is used to control the vision camera to acquire motion images and calculate preliminary motion vectors at a preset frame rate for the competition target with the bound identification code, the lidar to scan the three-dimensional position and calculate the three-dimensional motion vector, the millimeter-wave radar to monitor the speed and generate speed vector data, and output three types of raw motion vector data.

[0227] The correlation and calibration unit is used to receive three types of raw motion vector data from the acquisition unit, perform error compensation on the three types of raw motion vector data, and output the compensated motion vector data.

[0228] The motion vector fusion unit is used to input the compensated motion vector data output by the correlation and calibration unit into the motion vector fusion model and output the calibrated motion vector.

[0229] The vector integration unit is used to periodically acquire the calibrated motion vectors output by the motion vector fusion unit at a set frequency, record the acquisition timestamp, and perform time synchronization verification on the acquired data based on the time synchronization protocol. The calibrated motion vectors after verification and correction are sorted by timestamp to generate and output real-time motion vector data.

[0230] The occlusion recognition module includes: an occlusion pre-judgment unit, a snapshot library construction unit, a trajectory acquisition unit, a model construction unit, a virtual trajectory generation unit, and an occlusion removal judgment unit;

[0231] The occlusion pre-judgment unit is used to control the visual camera to acquire image data of multiple competition targets bound with identification codes, segment the target region through the image segmentation algorithm and calculate the horizontal / vertical distance of adjacent targets, compare the distance with the preset occlusion judgment threshold, and issue an occlusion warning signal if it is less than or equal to the occlusion judgment threshold; otherwise, it triggers the vector generation module to continue running.

[0232] The snapshot library building unit is used to activate the data caching function after receiving the occlusion warning signal, capture the latest motion data of the participating target in the real-time motion vector data and the latest appearance feature parameters collected by the visual camera, and store them as snapshot data before occlusion according to the timestamp to establish the snapshot library before occlusion.

[0233] The trajectory acquisition unit is used to control the lidar to scan the occluded area to obtain the overall three-dimensional outline of the group, control the millimeter-wave radar to monitor the overall velocity data of the group, calculate the overall motion vector of the group, and generate the overall trajectory of the occluded group based on the overall motion vector and time series.

[0234] The model building unit is used to retrieve the motion data of each participating target before occlusion from the snapshot library before occlusion, extract individual motion features, and combine the individual motion features with the preset basic motion model to build an individual motion prediction model for each target.

[0235] The virtual trajectory generation unit is used to divide the overall trajectory of the occluded group into trajectory segments according to the timestamp, predict the suspected motion trajectory of each participating target in each trajectory segment based on the individual motion prediction model, and make adjustments in combination with the overall contour constraints of the group to generate the virtual trajectory of each participating target.

[0236] The occlusion removal determination unit is used to monitor the data collected by the visual camera, lidar, and millimeter-wave radar in real time. If all three types of sensors re-collect independent data of each participating target, the occlusion is determined to be removed, triggering the actual trajectory acquisition process.

[0237] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

Claims

1. A method for dynamically visualizing event information, characterized in that, include: S1: When the target enters the sensor's monitoring range for the first time, an initial feature label is generated through multimodal data fusion. The initial feature label is then weighted and fused to generate a unique identification code, which is then bound to the target to obtain the target with the bound identification code. S2: Based on the binding identification code, the competition target collects and calibrates motion vectors through three types of sensors, and generates real-time motion vector data by combining update frequency and synchronization mechanism; S3: If the target of the competition bound with the identification code leaves the sensor's field of view, the fusion prediction model is called. The real-time motion vector data, the pre-stored track environment, and the preset target physical parameters are input into the fusion prediction model to obtain the predicted trajectory with confidence. S4: If the participating targets bound to the identification code are cross-occluded, the distance between the participating targets is monitored in real time by the visual camera to make a pre-judgment of occlusion. The motion data and appearance features before occlusion are cached in time to establish a snapshot library before occlusion. Then, the overall outline of the occluded group is obtained by LiDAR and the overall motion vector of the group is measured by millimeter-wave radar to obtain the overall trajectory of the occluded group. Based on the individual motion features of the participating targets in the snapshot library before occlusion, an individual motion prediction model is constructed. The overall trajectory is decomposed into a group trajectory to generate a virtual trajectory. After the occlusion is removed, the virtual trajectory is matched with the measured trajectory collected by the sensor to find the unique identification code bound to the participating target. S5: Fit real-time motion vector data, predicted trajectories with confidence, and target data after retrieving the identification code into a dynamic image to complete the dynamic visualization of event information; The specific steps of S3 include: S3.1: Real-time monitoring of the acquisition status of the three types of sensors on the competition targets with bound identification codes; If the visual camera fails to capture the appearance image data of the target for a consecutive preset number of frames, the lidar fails to scan the three-dimensional contour data of the target for a consecutive preset time, and the millimeter-wave radar fails to detect the speed data of the target for a consecutive preset period, then the target is determined to be out of the sensor's field of view. If any sensor can still collect data on the target, then continue with the step of generating real-time motion vector data; S3.2: Pre-collect 3D terrain data, track boundary data, track obstacle distribution data, and track marker data of the race track to form a track environment model and store it in the track environment database. When the participating target leaves the sensor's field of view, retrieve the track environment model from the track environment database as the track environment data. S3.3: Pre-set the target physical parameters and store them in the target physical parameter database. When the participating target leaves the sensor's field of view, retrieve the preset target physical parameters corresponding to the participating target from the target physical parameter database. S3.4: The generated real-time motion vector data, the retrieved track environment data, and the retrieved preset target physical parameters are preprocessed according to the input format requirements of the fusion prediction model to form target fusion data and input into the fusion prediction model. S3.5: The fusion prediction model first inputs the target fusion data into the LSTM network. The LSTM network learns the motion patterns of the participating target based on historical motion data and outputs a preliminary predicted trajectory. Then, the preliminary predicted trajectory is input into the particle filter algorithm module. The particle filter algorithm module optimizes and adjusts the preliminary predicted trajectory by combining the obstacle constraints in the track environment and the motion restrictions in the target's physical parameters. At the same time, the fusion prediction model calculates the confidence level of the predicted trajectory based on the completeness and accuracy of the input target fusion data and outputs the optimized predicted trajectory with confidence.

2. The method for dynamically visualizing event information as described in claim 1, characterized in that, When the target enters the sensor's monitoring range for the first time, initial feature labels are generated through multimodal data fusion, including: S1.1: When the target enters the sensor monitoring range for the first time, the three types of sensors will work together; the sensors include a visual camera, a lidar, and a millimeter-wave radar. The visual camera acquires appearance image data of the participating target and extracts appearance feature parameters; the appearance feature parameters include the color features, shape features and texture features of the participating target. The lidar acquires the three-dimensional contour data of the participating target and extracts contour feature parameters; the contour feature parameters include the length, width, and height parameters of the participating target. The millimeter-wave radar acquires the initial velocity data of the participating target and extracts velocity characteristic parameters; the velocity characteristic parameters include the instantaneous velocity magnitude and velocity direction of the participating target; S1.2: Input the appearance feature parameters, contour feature parameters and velocity feature parameters into a preset data fusion model. The data fusion model performs fusion processing on each feature parameter based on a weighted average algorithm and outputs initial feature labels.

3. The method for dynamically visualizing event information as described in claim 2, characterized in that, The specific steps of S2 include: S2.1: Based on the competition target bound by the identification code, activate the three types of sensors—visual camera, LiDAR, and millimeter-wave radar—to enter real-time monitoring mode; The visual camera captures motion images of the participating targets bound with identification codes in real time according to a preset acquisition frame rate. Based on the image recognition algorithm, the position coordinates of the participating targets in the image coordinate system are extracted, and the position change of the participating targets in adjacent frames is calculated to obtain the direction and magnitude of the preliminary motion vector. The lidar scans the three-dimensional position of the participating target in real time, obtains the three-dimensional coordinates of the participating target in the three-dimensional spatial coordinate system, calculates the change of the three-dimensional coordinates per unit time, and obtains the three-dimensional motion vector. The millimeter-wave radar monitors the radial and lateral velocities of the participating targets in real time and generates velocity vector data. S2.2: Coordinate the preliminary motion vector, three-dimensional motion vector and velocity vector data to establish the timestamp correspondence of the data collected by each sensor. At the same time, call the sensor calibration database to obtain the measurement error parameters of each sensor. S2.3: Based on the measurement error parameters, error compensation is performed on the obtained preliminary motion vector, three-dimensional motion vector and velocity vector data respectively. The compensation method is to superimpose the preliminary motion vector, three-dimensional motion vector and velocity vector data with the corresponding error correction values ​​respectively.

4. The method for dynamically visualizing event information as described in claim 3, characterized in that, The specific steps of S2 also include: S2.4: Input the compensated preliminary motion vector, three-dimensional motion vector, and velocity vector data into the motion vector fusion model; the motion vector fusion model uses the Kalman filter algorithm to fuse the preliminary motion vector, three-dimensional motion vector, and velocity vector data, and outputs the calibrated motion vector; S2.5: Preset the update frequency of motion vector data, and establish a time synchronization protocol between sensors. Based on the time synchronization protocol, calibrate the clocks of each sensor. The update frequency is determined according to the type of competition and the speed of the target. S2.6: Periodically collect the calibrated motion vector according to the update frequency, and record the collection timestamp after each collection; S2.7: Based on the time synchronization protocol between sensors, the time synchronization verification is performed on the calibrated motion vector and the corresponding acquisition timestamp for each acquisition. If the time deviation exceeds the preset deviation threshold, the acquisition data is corrected for time offset. S2.8: Arrange the calibrated motion vectors, which have undergone time synchronization verification and correction, in the order of the acquisition timestamps to generate real-time motion vector data.

5. The method for dynamically visualizing event information as described in claim 4, characterized in that, The specific steps of S4 include: S4.1: The visual camera acquires image data of multiple participating targets with bound identification codes in real time, and segments each participating target into independent target regions in the image based on the image segmentation algorithm, and calculates the distance between the target regions corresponding to adjacent participating targets in the image coordinate system; the distance includes horizontal distance and vertical distance; S4.2: Compare the calculated distance with the preset occlusion determination threshold; If the distance between all adjacent participating targets is greater than the occlusion detection threshold, it is determined that there is no risk of cross occlusion, and the step of generating real-time motion vector data continues. If the distance between any adjacent participating targets is less than or equal to the occlusion detection threshold, it is determined that there is a risk of cross occlusion and an occlusion warning signal is issued. S4.3: Upon receiving the occlusion warning signal, the data caching module is activated; the data caching module captures the latest motion data of the corresponding target from the generated real-time motion vector data, and simultaneously captures the latest appearance feature data of the target collected by the visual camera, i.e., the appearance feature parameters, and associates and stores the captured latest motion data and the latest appearance feature data according to the timestamp to form the snapshot data of the target before occlusion; the latest motion data includes motion speed, motion direction and motion position; S4.4: Perform data capture and associated storage operations on all participating targets that have a risk of cross-occlusion, summarize the snapshot data of each participating target before occlusion, and establish a snapshot library before occlusion; each snapshot data in the snapshot library before occlusion contains the unique identification code of the participating target, the corresponding relationship between the motion data before occlusion and the appearance feature data before occlusion.

6. The method for dynamically visualizing event information as described in claim 5, characterized in that, The specific steps of S4 also include: S4.5: The lidar performs a panoramic scan of the area where there is cross-occlusion, collects the three-dimensional point cloud data of the occlusion group composed of all participating targets in the area, and preprocesses the three-dimensional point cloud data. Based on the point cloud clustering algorithm, the preprocessed three-dimensional point cloud data is clustered to obtain the three-dimensional overall outline of the occlusion group; the three-dimensional overall outline includes the outer cuboid size of the occlusion group and the overall center position. S4.6: The millimeter-wave radar continuously monitors the obstructing group and collects the overall speed data of the obstructing group; the overall speed data includes the magnitude of the overall movement speed and the overall movement direction; S4.7: Based on the overall velocity data and the overall center position obtained by the lidar, calculate the change in the overall center position of the occlusion group per unit time to obtain the overall motion vector of the group; S4.8: Based on the overall motion vector and time series, generate the overall trajectory of the occluded group in the track environment; the overall trajectory includes the overall position coordinates of the occluded group at different time points; S4.9: Retrieve the motion data of each participating target before occlusion from the snapshot library before occlusion, and extract the individual motion characteristics of each participating target based on the motion data before occlusion; the motion data before occlusion includes the trend of motion speed change, the law of motion direction change, and the trajectory of motion position change within a preset time period before occlusion; the individual motion characteristics include motion acceleration characteristics, turning frequency characteristics, and linear motion preference characteristics; S4.10: Combine the individual motion characteristics of each participating target with the preset basic motion model to construct an individual motion prediction model for each participating target; the basic motion model adopts a variable acceleration motion model.

7. The method for dynamically visualizing event information as described in claim 6, characterized in that, The specific steps of S4 also include: S4.11: The overall trajectory of the occluded group is divided into multiple trajectory segments according to the timestamp. Each trajectory segment corresponds to a time interval. For each trajectory segment, based on the constructed individual motion prediction model of each participating target, the suspected motion trajectory of each participating target within the time interval is predicted. At the same time, combined with the overall contour constraint of the occluded group, the suspected motion trajectory of each participating target is adjusted. The overall contour constraint of the occluded group means that the predicted motion trajectory of each participating target cannot exceed the spatial range of the overall contour. S4.12: The trajectory segmentation algorithm is used to match the adjusted suspected motion trajectories of each participating target with the overall trajectory segment to determine the specific motion path of each participating target in the overall trajectory segment. The specific motion paths of the participating targets determined in each time interval are connected in chronological order to generate the virtual trajectory of each participating target. The virtual trajectory contains the predicted position coordinates of the participating target at each time point during the occlusion period. S4.13: The visual camera monitors the image data of the occluded area in real time. If the independent appearance image data of each participating target is re-captured, and the lidar re-scans the independent three-dimensional contour data of each participating target, and the millimeter-wave radar re-monitors the independent velocity data of each participating target, then the occlusion is determined to be lifted. S4.14: After the occlusion is removed, the motion vectors of each participating target are collected and calibrated through the collaborative acquisition of three types of sensors, and the measured motion vector data of each participating target is generated. The measured trajectory is generated based on the measured motion vector data. S4.15: Calculate the similarity between the virtual trajectory and the measured trajectory, and match the virtual trajectory with the highest similarity with the measured trajectory to retrieve the unique identification code bound to the competition target; the similarity is determined by calculating the average Euclidean distance between the position coordinates of the corresponding time points of the two trajectories; the unique identification code is contained in the snapshot data before occlusion associated with the individual motion prediction model of the competition target corresponding to the virtual trajectory.

8. The method for dynamically visualizing event information as described in claim 7, characterized in that, The process of fitting real-time motion vector data, predicted trajectories with confidence levels, and target data after retrieving the identification code into a dynamic image includes: S5.1: Convert the generated real-time motion vector data into motion trajectory point data. Each motion trajectory point contains a timestamp, position coordinates, and motion speed information. S5.2: Decompose the obtained predicted trajectory with confidence into predicted trajectory point data according to the timestamp. Each predicted trajectory point contains a timestamp, predicted location coordinates and confidence value. S5.3: Integrate the target data after retrieving the identification code to obtain integrated target data; the target data includes the unique identification code of the participating target, the measured trajectory data, and the appearance feature data before occlusion; S5.4: Input the motion trajectory point data, predicted trajectory point data, and integrated target data into the dynamic image generation model. The dynamic image generation model synchronously integrates the data based on the time axis, converts the trajectory point data into continuous trajectory lines, and generates a dynamic image of the competition target by combining the appearance feature data of the competition target.

9. A dynamic visualization display system for sports event information, used to implement the dynamic visualization display method for sports event information as described in any one of claims 1-8, characterized in that, include: Target binding module, vector generation module, trajectory prediction module, occlusion recognition module, and dynamic visualization module; The target binding module is used to extract target features, generate identification codes, and bind the target when the participating target first enters the sensor monitoring range. The vector generation module is used to generate real-time motion vector data based on the competition target with the bound identification code through sensor collaborative acquisition, calibration and data fusion. The trajectory prediction module is used to generate a predicted trajectory with confidence by fusing a prediction model when the target of the competition leaves the sensor's field of view. The occlusion recognition module is used to perform occlusion pre-judgment, data caching, group trajectory analysis, virtual trajectory generation and identification code retrieval when the participating targets are cross-occluded; The dynamic visualization module is used to integrate the output data, fit it into a dynamic image, and complete the visualization display of the event information.

Images