Motion track construction method based on visual large model

Abstract

The invention discloses a motion track construction method based on a visual large model, and relates to the technical field of computer vision, and the method comprises the steps: employing a target tracking algorithm based on Kalman filtering, and generating a smooth time sequence synchronization track from an original laser radar point cloud and an original video stream; dividing the trajectory into a series of candidate kinematic segments which are complete in kinematics and semantics by applying a hybrid method combining a minimum description length principle and visual language model semantic verification; performing preliminary semantic annotation on the segments by utilizing a visual language model to generate an initial annotation track; performing logic consistency refining on the initial labeling track until the initial labeling track is converged into a refined labeling track; and formatting the refined trajectory into a standard structured semantic trajectory representation character string. According to the method, a semantic gap between low-dimensional physical observation and high-dimensional driving intention is bridged, and ideal input is provided for understanding and prediction tasks of downstream complex scenes.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more particularly to a motion trajectory construction method based on a large visual model. Background Art

[0002] Reliable perception and understanding of dynamic and complex traffic environments is a fundamental challenge in the development and deployment of autonomous driving systems. Traditional perception pipelines rely on fusing multimodal data from sensors such as lidar and cameras to construct a geometric model of the world, based on which they perform object detection and state estimation. However, this paradigm suffers from a fundamental semantic gap: a significant discrepancy between the low-level, quantitative data generated by sensors and the high-level, intent-driven scene understanding possessed by human drivers. While existing systems can accurately detect objects and estimate their kinematic states, they face difficulties in inferring the underlying driving intent or contextual logic behind these kinematic states. This bottleneck is particularly prominent when dealing with "long-tail scenarios" or complex interactive scenarios that are not fully reflected in the training data.

[0003] Current approaches to vehicle trajectory understanding and prediction based on large visual models primarily rely on representing motion trajectories as sequences of purely geometric coordinate points. This low-level representation lacks rich semantic information, forcing large visual models to learn the complex physical laws underlying motion and high-level driving intent from scratch. Research has shown that this semantic gap between representation and reasoning is a core bottleneck that limits the reasoning capabilities of existing models in complex traffic scenarios, often leading to models that merely reproduce textual cues rather than truly understand the visual content.

[0004] In view of this, the present invention proposes a motion trajectory construction method based on a large visual model to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention provides the following technical solution: a motion trajectory construction method based on a large visual model, comprising:

[0006] Step F1, based on the collected original lidar point cloud and original video stream, as well as the preset Kalman filter state transition matrix, Kalman filter process noise covariance matrix, and Kalman filter measurement noise covariance matrix, apply a Kalman filter-based target tracking algorithm to extract the target vehicle's motion trajectory and align it with the video timestamp to obtain a time-synchronized trajectory and visual context video;

[0007] Step F2: Based on pre-collected high-definition map data, a preset minimum description length model primitive set, and a minimum description length encoding accuracy, a hybrid segmentation method combining the minimum description length principle with semantic verification of a visual language model is applied to process the time-synchronized trajectory and the visual context video to generate candidate kinematic segments, wherein segmentation points are determined according to the minimum description length principle and verified by the visual language model to ensure continuity of driving intention.

[0008] Step F3, traverse the candidate kinematic segments and use the visual language model to assign initial semantic labels to each segment in combination with the visual context video, high-definition map data and the predefined semantic behavior ontology library to obtain the initial annotated trajectory;

[0009] Step F4: Based on the visual context video and HD map data, as well as the preset judge visual language model confidence threshold, maximum number of refinement iterations, and collaborative teaching reasoning path set, an iterative cycle including judge visual language model evaluation and collaborative teaching correction mechanism is used to process the initial labeled trajectory until the logical consistency of the label sequence converges, and the refined labeled trajectory is output;

[0010] Step F5, traverse each segment in the refined annotated trajectory, combine the semantic label, start time and end time of the segment into a standardized text representation, and concatenate the text representations of all segments into a string to obtain a structured semantic trajectory representation.

[0011] Furthermore, the implementation method of step F2 includes:

[0012] Steps F2 and F1 apply a hybrid segmentation method based on the minimum description length principle, process the time-series synchronization trajectory according to the preset minimum description length model primitive set and minimum description length encoding accuracy, and generate preliminary candidate segmentation points that are optimal in terms of information compression;

[0013] Step F2F2, initialize an empty final segmentation point list;

[0014] In step F2F3, for each candidate point in the preliminary candidate segmentation points, steps F2F3F1 to F2F3F3 are executed in a loop: in step F2F3F1, two adjacent motion segments with the candidate point as the segmentation point are extracted, and information required for semantic verification is extracted from the visual context video and the high-definition map data to construct a visual language model semantic verification context; in step F2F3F2, prompt words are constructed to guide the visual language model to perform step-by-step reasoning, and the prompt words and the visual language model semantic verification context are used to query the visual language model to determine whether the two adjacent motion segments together constitute a semantically continuous driving intention, thereby obtaining a verification result; in step F2F3F3, if the verification result indicates that the two adjacent motion segments together constitute a semantically continuous driving intention, the candidate point is added to the final segmentation point list;

[0015] In steps F2 and F4, the final segmentation point list is used to segment the time-series synchronization trajectory to obtain candidate kinematic segments.

[0016] Furthermore, the implementation method of step F4 includes:

[0017] Step F4F1, copy the initial annotation trajectory as the current iteration trajectory;

[0018] Step F4F2, initialize the Boolean convergence flag to false and the integer iteration counter to 0;

[0019] Step F4F3, under the condition that the convergence flag is false and the iteration counter is less than the maximum number of refinement iterations, loop through steps F4F3F1 to F4F3F4: Step F4F3F1, mark the current iteration trajectory as the previous round trajectory, and set the convergence flag to true; Step F4F3F2, for each pair of adjacent semantic labels in the current iteration trajectory, guide a visual language model as a referee by constructing a prompt word, combine the visual context video and high-definition map data corresponding to the pair of adjacent semantic labels, judge the logical consistency of the pair of adjacent semantic labels and output a confidence score, and then identify the noisy semantic labels whose logical consistency is judged as false or whose confidence score is lower than the preset referee visual language model confidence threshold, and obtain a noisy label index set; Step F4F3F3, if the noisy label index set is not empty, execute steps F4F3F3F1 and F4F3F3F2: Step F4F3F3F1, set the convergence flag to false; Step F4F3F3F2, for the noisy labels For each noise index in the signature index set, steps F4F3F3F2F1 to F4F3F3F2F4 are executed in a loop: Step F4F3F3F2F1, for the noisy semantic label corresponding to the noise index, respectively use the first reasoning configuration and the second reasoning configuration contained in the collaborative teaching reasoning path set to query the visual language model to obtain candidate revised semantic label 1 and its confidence score 1, as well as candidate revised semantic label 2 and its confidence score 2; Step F4F3F3F2F2, if the candidate revised semantic label 1 is the same as the candidate revised semantic label 2, then the same candidate revised semantic label is selected as the final revised semantic label; Step F4F3F3F2F3, otherwise, select the candidate revised semantic label with a higher confidence score between the candidate revised semantic label 1 and the candidate revised semantic label 2 as the final revised semantic label; Step F4F3F3F2F4, use the final revised label to update the label at the noise index in the current iteration trajectory; Step F4F3F4, increment the iteration counter by one;

[0020] In step F4F4, the final current iteration trajectory is named as the refined labeled trajectory.

[0021] Furthermore, the implementation method of steps F2 and F1 includes:

[0022] Steps F2F1F1 define and calculate the total description length of any subsequence of the time-synchronized trajectory, the total description length being calculated by adding the number of bits required to describe the model parameters of a geometric model selected from a preset set of model primitives of minimum description length used to fit the subsequence to the number of bits required to encode the fitting errors of all trajectory points in the subsequence to the geometric model;

[0023] Step F2F1F2, initialize the cost array and the split point index array;

[0024] Steps F2, F1, and F3 apply a dynamic programming algorithm to determine and record the optimal segmentation point required to achieve the minimum total description length through an iterative solution process. The solution process includes: for each current trajectory point in the time-series synchronization trajectory, traverse all trajectory points before the current trajectory point as candidate segmentation points, calculate the cumulative total description length when the candidate segmentation point is used as the last segmentation point, select the candidate segmentation point that minimizes the cumulative total description length as the optimal predecessor segmentation point of the current trajectory point, and store the selected minimum cumulative total description length and the index of the optimal predecessor segmentation point in the cost array and the segmentation point index array, respectively;

[0025] Steps F2, F1, and F4 are to extract all the optimal segmentation points by backtracking the segmentation point index array to obtain preliminary candidate segmentation points. The backtracking method is as follows: starting from the end point of the timing synchronization trajectory, reversely search for the optimal predecessor segmentation point corresponding to the end point using the segmentation point index array, and repeatedly perform reverse search using the found optimal predecessor segmentation point as the search starting point until the starting point of the trajectory is traced back, and record all the optimal segmentation points found during the entire backtracking process.

[0026] Furthermore, the implementation method of steps F2F3F1 includes:

[0027] Steps F2F3F1F1: find the previous and next segmentation points of the candidate point in the preliminary candidate segmentation point list, and define two adjacent motion segments centered on the candidate point;

[0028] Steps F2F3F1F2: extracting corresponding video slices from the visual context video according to the time range of the two adjacent motion segments;

[0029] Steps F2F3F1F3: extracting corresponding road network information from the high-definition map data based on the spatial range of the two adjacent motion segments;

[0030] In steps F2, F3, F1, and F4, the extracted video slices and road network information are combined to construct a semantic verification context for the visual language model.

[0031] Furthermore, the implementation method of steps F4, F3, and F2 includes:

[0032] Steps F4F3F2F1, initialize an empty noisy label index set;

[0033] Step F4F3F2F2, traverse all semantic labels from the first to the second to last in the current iteration trajectory, and for each pair of adjacent semantic labels, loop through steps F4F3F2F2F1 to F4F3F2F2F4: Step F4F3F2F2F1, extract the visual context video slices and high-definition map data corresponding to the pair of adjacent semantic labels as multimodal context; Step F4F3F2F2F2, construct multimodal thinking chain prompt words, which are used to guide the visual language model as the referee to perform step-by-step reasoning, which includes: The next steps are: identifying the previous semantic label, identifying the next semantic label, and judging whether the conversion from the previous semantic label to the next semantic label conforms to the driving logic based on the multimodal context; step F4F3F2F2F3, using the constructed prompt words and the multimodal context to query the visual language model serving as the referee to obtain a logical consistency judgment and a confidence score; step F4F3F2F2F4, if the logical consistency judgment is no or the confidence score is lower than the confidence threshold of the referee visual language model, then adding the index of the pair of adjacent semantic labels to the noisy label index set.

[0034] Furthermore, the implementation method of steps F4F3F3F2F1 includes:

[0035] Step F4F3F3F2F1F1, extracting the noisy semantic label corresponding to the noise index and its multimodal context;

[0036] Step F4F3F3F2F1F2, adopting the first reasoning configuration, using the multimodal context to query the visual language model, and obtaining a candidate revised semantic label 1 and a confidence score 1;

[0037] Step F4F3F3F2F1F3, adopting the second reasoning configuration, using the multimodal context to query the visual language model, and obtaining the candidate revised semantic label 2 and the confidence score 2;

[0038] Among them, the first reasoning configuration is different from the second reasoning configuration, and the difference lies in that the multimodal thinking chain prompt word template used to query the visual language model is different, or the queried visual language model is a model of different version or different architecture.

[0039] Beneficial effects of the motion trajectory construction method based on a visual large model of the present invention:

[0040] By employing a hybrid segmentation approach based on the minimum description length principle and semantic validation using the visual language model, the present invention is able to generate kinematically efficient and semantically complete motion segments. Specifically, the MDL principle ensures that segmentation is based on optimal compression of the data's inherent structure, avoiding the arbitrariness of traditional heuristic methods; while the semantic validation of the VLM further ensures that segmentation points conform to the transformation of driving intent, bridging the gap between pure kinematic segmentation and high-level semantic understanding. This feature addresses the difficulty of traditional methods in generating meaningful behavioral units and provides a high-quality foundation for subsequent precise semantic annotation.

[0041] By introducing an iterative loop involving referee visual language model evaluation and a collaborative teaching and correction mechanism, the present invention significantly improves the logical consistency and reliability of the final semantic label sequence. To address the logical contradictions that may arise in a single VLM annotation, the collaborative teaching mechanism of the present invention generates mutually verified candidate labels through two different reasoning paths and iteratively corrects them, effectively filtering out noisy labels. This feature directly overcomes the lack of logical consistency in semantic output in existing methods, ensuring the inherent interpretability and credibility of trajectory representation.

[0042] Ultimately, this invention formats the refined trajectory into a structured semantic trajectory representation, providing high-quality, purified input for downstream autonomous driving planning and decision-making systems. Downstream modules no longer need to process a noisy sequence of raw coordinate points, but can instead operate directly on a symbolic sequence composed of clear, logically verified action primitives. This not only fundamentally reduces the cognitive burden and reasoning complexity of the decision-making system, but also effectively decouples the perception and understanding modules from the planning and decision-making modules, forming a verifiable, stable semantic log that is crucial for system safety verification and failure analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a method for constructing a motion trajectory based on a large visual model according to the present invention;

[0044] Figure 2 This is a schematic diagram of system modules of a motion trajectory construction system based on a large visual model according to the present invention;

[0045] Figure 3 Schematic diagram of an application scenario of a motion trajectory construction method based on a visual large model of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] Example 1

[0048] See Figure 1 The method for constructing a motion trajectory based on a large visual model described in this embodiment includes steps F1 to F5.

[0049] This section details the complete implementation process for converting raw multimodal sensor data streams into high-level structured semantic trajectory representations through a series of processing and analysis stages. The process begins with a basic state estimation phase, which converts the raw, noisy sensor readings into a smooth and physically plausible time-synchronized trajectory by applying the Kalman filter algorithm. The process then enters a core segmentation and annotation phase, which first performs data-driven kinematic segmentation using the minimum description length principle (MDL), followed by semantic verification and preliminary annotation using the visual language model (VLM). Finally, to ensure the logical rigor of the final output, the process enters an iterative refinement phase, which uses a co-teaching mechanism to self-correct and verify the consistency of the annotation sequence until the final, high-quality refined annotated trajectory is generated.

[0050] Step F1, based on the collected original lidar point cloud and original video stream, and the preset Kalman filter state transfer matrix , Kalman filter process noise covariance matrix and the Kalman filter measurement noise covariance matrix ,The Kalman filter-based target tracking algorithm is applied to extract the target vehicle’s motion trajectory and align it with the video timestamp to obtain the time-synchronized trajectory and visual context video.

[0051] The raw LiDAR point cloud comes from a multi-line rotating LiDAR sensor on a vehicle that meets the standard specifications of the autonomous driving industry. The sensor determines distance by emitting laser pulses and measuring the flight time after they are reflected from an object. Its internal rotation mechanism ensures periodic scanning of the surrounding environment with a scanning frequency of at least 10Hz. The raw data generated by each scan is a dense set of points, where each point is quantized into a three-dimensional spatial coordinate containing its coordinate relative to the sensor coordinate system. and high-precision time stamps synchronized with the vehicle master clock These time-stamped point cloud sequences together constitute the raw lidar point cloud.

[0052] The raw video stream is captured by an onboard, forward-looking, automotive-standard digital camera system. This system continuously records video at a minimum frame rate of 25 frames per second to ensure smooth representation of motion. The video data is encoded and encapsulated in a standard digital container format that contains not only the sequence of image frames but also the precise timestamp of each frame's presentation. This encapsulation ensures precise temporal alignment of the visual information with data from other sensors, such as the raw LiDAR point cloud.

[0053] Kalman filter state transition matrix It is not an input obtained through measurement, but a matrix derived based on a certain kinematic model. Specifically, the Kalman filter state transfer matrix It is based on a two-dimensional constant velocity (CV) model, which assumes that the velocity of an object remains constant within a discrete time step. The state vector is formally defined to represent and Direction, position and speed, 、 Respectively and Direction of location, 、 Respectively and Direction speed, Kalman filter state transfer matrix Through mathematical construction, it is used to linearly project the current state to the processing time step The next moment defined by The structure in the above example directly implements the kinematic equations and , where the subscript Represents the current moment, subscript Represents the next moment, 、 Represents the current time k and Direction of location, 、 Represents the current time k and The two kinematic equations are the theoretical core of the Kalman filter's "prediction" step.

[0054] Kalman filter process noise covariance matrix It is used to quantify the uncertainty of the motion model itself. It recognizes that the assumption of constant velocity is an ideal state. In reality, the target vehicle will have small random accelerations that are not modeled. This matrix is ​​determined based on the "discrete Wiener process acceleration model". This model assumes that the processing time step The acceleration is a random walk process with a mean of zero. The magnitude of this random acceleration is determined by the adjustable parameter process noise standard deviation Control. Kalman filter process noise covariance matrix By formula Calculated, where is the noise gain matrix, which maps the noise in the acceleration dimension to the state space of position and velocity, and T is the transpose of the matrix. This method provides a way to inject uncertainty into the model during the filter’s prediction step, thereby preventing the filter from becoming overconfident in its own model and enabling it to better adapt to real-world maneuvers.

[0055] Kalman filter measurement noise covariance matrix represents the uncertainty inherent in the sensor measurement process. Its value is determined based on the manufacturer's technical specifications of the LiDAR sensor used to generate the original LiDAR point cloud. This matrix is ​​usually a diagonal Matrix whose diagonal elements Represents the sensor and The variance of the measurement error in the direction is the square of the standard deviation. These values ​​directly quantify the measurement accuracy of the sensor and are assumed to obey independent Gaussian distributions. Kalman filter measurement noise covariance matrix It is crucial in the "update" step of the Kalman filter because it determines how much the system should weigh its trust in new measurements versus its trust in its own predictions when fusing information.

[0056] The Kalman filter is a recursive Bayesian filtering algorithm for state estimation in dynamic systems containing uncertainty. It effectively combines predictions based on motion models with noisy sensor measurements through a "predict-update" cycle, making it a standard and fundamental method in target tracking.

[0057] This step employs a Kalman filter, whose theoretical foundations are linear dynamic systems and Gaussian noise assumptions. It provides an optimal linear estimator for extracting smooth and physically plausible motion trajectories from noisy sensor data, such as LiDAR point clouds, by minimizing the mean squared error of the state estimate. The constant velocity model was chosen as the basis because it strikes a good balance between computational efficiency and accuracy in describing most driving scenarios.

[0058] The implementation method of step F1 includes: step F1F1, identifying the initial position of the target vehicle from the first frame of the original lidar point cloud Steps F1 and F2, use the initial position and zero initial velocity to initialize the state vector of the two-dimensional constant velocity Kalman filter Step F1F3, for each subsequent frame in the original lidar point cloud, loop through steps F1F3F1 to F1F3F4: Step F1F3F1, using the Kalman filter state transfer matrix The state at the previous moment is predicted to obtain the predicted state at the current moment. Steps F1, F3, and F2 extract the measured position of the target vehicle from the current lidar frame. Steps F1, F3, and F3 combine the predicted state, measured position, and Kalman filter process noise covariance matrix. and the Kalman filter measurement noise covariance matrix , execute the Kalman filter update step to obtain the corrected state at the current moment. Steps F1, F3, and F4 append the position and timestamp information in the corrected state to the trajectory sequence. Step F1, F4 aggregates the position and timestamp information in the corrected state at all moments to obtain a time-synchronized trajectory. Step F1, F5 crops and synchronizes the original video stream based on the timestamps in the time-synchronized trajectory to obtain a visual context video.

[0059] The time-series synchronization trajectory generated in this step is a time series data structure, in which each element contains a timestamp and the two-dimensional position coordinates of the target vehicle at that timestamp after smoothing by the Kalman filter. Logically, the time-synchronized trajectory represents the optimal linear estimate of the target vehicle's true motion trajectory, eliminating high-frequency noise in the raw sensor data and filling in possible short measurement gaps. The time-synchronized trajectory serves as the core input for the subsequent kinematic segmentation step, providing it with a clean, continuous, and physically plausible coordinate sequence.

[0060] The visual context video generated in this step is a processed video file stream whose time base is fully aligned with the time-synchronized trajectory. Logically, the visual context video provides corresponding, realistic visual scene information for each time point in the time-synchronized trajectory, such as road conditions, traffic signals, and pedestrian postures. The visual context video serves as the key multimodal input for all subsequent steps requiring semantic understanding, such as semantic verification, initial annotation, and consistency refinement, providing the necessary visual evidence for the reasoning of the Visual Language Model (VLM).

[0061] After obtaining a smooth, time-synchronized trajectory through Kalman filtering, the next core task is to extract meaningful, discrete motion units from this continuous sequence of coordinate points. Directly using a fixed time window or simple thresholds based on velocity or acceleration for segmentation can easily produce semantically incomplete or meaningless segments. To identify key "change points" in the trajectory in an objective and data-driven manner, subsequent steps will introduce the minimum description length principle (MDL) based on information theory. The logical intention of this move is to restructure the trajectory segmentation problem as a data compression problem, thereby finding a set of segmentation points that can describe the entire trajectory in the most concise way, ensuring the mathematical optimality of the segmentation.

[0062] In step F2, based on the pre-collected high-definition map data, as well as the preset minimum description length model primitive set and minimum description length encoding accuracy, a hybrid segmentation method combining the minimum description length principle with the semantic verification of the visual language model is applied to process the time-synchronized trajectory and the visual context video to generate candidate kinematic segments whose segmentation points are determined according to the minimum description length principle and verified by the visual language model to ensure the continuity of driving intention.

[0063] HD map data is derived from professional surveying and mapping, encoded and stored in a vectorized format. This data is not generated in real time but rather input as prior knowledge. Its quantification method involves accurately representing physical road elements such as lane centerlines, stop lines, road boundaries, and traffic sign locations as a series of geometric primitives and their topological relationships. HD map data must include centimeter-level static road network information to provide precise geospatial constraints for subsequent scene understanding and semantic analysis.

[0064] The minimum description length model primitive set is a list of strings determined by design that defines the library of geometric models that the MDL segmentation algorithm can use when fitting trajectories. For example, ['line', 'arc'] indicates that the algorithm can only use straight lines and circular arc segments to describe trajectories, where 'line' represents a straight line segment and 'arc' represents a circular arc segment. This set is determined based on prior knowledge of typical vehicle motion patterns in target scenarios, such as urban driving, which assumes that most driving behaviors can be decomposed into a combination of straight lines and circular arcs. The minimum description length model primitive set is a key design parameter that directly constrains the model's expressiveness and complexity.

[0065] The minimum description length encoding precision is a positive floating point number, which is a design parameter used to set the minimum description length in the calculation model. The numerical precision used when encoding the parameters of each primitive in the minimum description length model primitive set is in bits. Refers to the geometric model selected from the minimum description length model primitive set. The determination of the minimum description length encoding accuracy is a trade-off between model simplicity and fitting accuracy. A higher minimum description length encoding accuracy allows for more accurate model parameters, but increases The specific value is set based on the analysis of the trajectory data coordinate range and the required fitting accuracy.

[0066] The rationality and feasibility of the hybrid segmentation method adopted in this step are based on two mature theoretical foundations. First, the minimum description length principle is a core concept in information theory and statistical inference. This principle formalizes model selection as data compression, and believes that the model that can describe the data with the shortest coding length is the best model. Applying this principle to trajectory segmentation is equivalent to finding a segmentation representation that is mathematically the most concise and can best capture the intrinsic structure of the data, thereby avoiding the use of heuristics or arbitrary thresholds. Second, the candidate points generated by MDL are semantically verified, and its feasibility stems from the powerful zero-sample reasoning and scene understanding capabilities demonstrated by modern visual language models. VLM can integrate multimodal information, including vision, maps, and text, and perform human-like common sense reasoning to determine whether a kinematic breakpoint also corresponds to a transition point of driving intention, thereby ensuring that the final segmentation is complete in both kinematic and semantic dimensions.

[0067] The core innovation of this step lies in the formal integration of bottom-up information-theoretic optimality and top-down semantic rationality. Traditional trajectory segmentation methods rely solely on kinematic thresholds, which can produce meaningless segmentations, or rely entirely on models, which can ignore the structure of the data itself. This method utilizes the MDL principle to first find the most mathematically concise segmentation points from a data compression perspective, ensuring the kinematic efficiency of the segmentation. Then, leveraging the powerful scene understanding capabilities of the VLM, these candidate points are semantically pruned to ensure that the segmentation boundaries align with the start and end points of the high-level driving intent.

[0068] The principle of minimum description length originates from information theory, whose core idea is that the best explanation of data is the one that maximizes its compression. Viewing trajectory segmentation as a data compression problem allows for the identification of intrinsic, non-arbitrary "change points." The introduction of the VLM is based on its demonstrated ability to connect vision, text, and high-level concepts in emerging multimodal reasoning tasks, making it an ideal tool for verifying the semantic validity of segmentation points. This combination ensures that the final segmentation is optimal in both kinematic and semantic dimensions.

[0069] The implementation method of step F2 includes: Step F2F1: Applying a hybrid segmentation method based on the minimum description length principle, processing the time-synchronized trajectory according to a preset minimum description length model primitive set and minimum description length encoding accuracy, and generating preliminary candidate segmentation points that are optimal in terms of information compression. Step F2F2: Initializing an empty final segmentation point list. Step F2F3: For each preliminary candidate segmentation point, looping through steps F2F3F1 to F2F3F3: Step F2F3F1: Extracting the two adjacent motion segments preceding and following the candidate point as the segmentation point, and extracting information required for semantic verification from the visual context video and the high-definition map data to construct a visual language model semantic verification context. Step F2F3F2: Querying the visual language model using multimodal thought chain prompts to determine whether the candidate point violates the integrity of the semantically continuous driving intent due to minor kinematic fluctuations, thereby obtaining a verification result. Step F2F3F3: If the verification result indicates "intent integrity is not violated," the candidate point is added to the final segmentation point list. In steps F2 and F4, the final segmentation point list is used to segment the time-series synchronization trajectory to obtain candidate kinematic segments.

[0070] The implementation method of step F2F1 includes: step F2F1F1, defining and calculating the total description length of any subsequence of the time-synchronized trajectory, wherein the total description length is calculated by adding the number of bits required to describe the model parameters used to fit the subsequence and the number of bits required to encode the fitting errors of all trajectory points in the subsequence to the model. The total description length is calculated by the formula Indicates; where, Indicates the total description length of the subsequence in bits. Indicates the trajectory point Arrive Subsequence data of Representing the geometric model The encoding length, Indicates the encoding length of the fitting error, that is, in a known geometric model The number of bits required to encode data D under the premise of . Step F2F1F2, initialize the cost array Cost and the split point index array Split. Step F2F1F3, apply the dynamic programming algorithm, determine and record the optimal split point required to achieve the minimum total description length through an iterative solution process, the process includes: for each current trajectory point in the time-series synchronization trajectory, traverse all trajectory points before the current trajectory point as candidate split points, calculate the cumulative total description length when the candidate split point is used as the last split point, select the candidate split point that minimizes the cumulative total description length as the optimal predecessor split point of the current point, and store the selected minimum cumulative total description length and the index of the optimal predecessor split point in the cost array and split point index array respectively. The iterative solution is achieved through the recursive formula Implementation; where, Indicates splitting the trajectory from the starting point to the point The minimum cumulative total description length, Indicates splitting the trajectory from the starting point to the point The minimum cumulative total description length, To describe the point Arrive The total description length of the subsequence, and Indicates a minimum value operation. Steps F2, F1, and F4 extract all optimal segmentation points by backtracking the segmentation point index array to obtain preliminary candidate segmentation points. The backtracking method is as follows: starting from the end point of the time-series synchronization trajectory, reversely search the optimal predecessor segmentation point corresponding to the end point using the segmentation point index array, and repeatedly reverse search using the found optimal predecessor segmentation point as the search starting point until the trajectory is traced back to the starting point, recording all optimal segmentation points found during the entire backtracking process.

[0071] This step formalizes the trajectory segmentation problem as a data compression problem, aiming to find the most concise trajectory representation. Dynamic programming is a standard and efficient algorithm for solving such sequence segmentation problems with optimal substructure properties.

[0072] The preliminary candidate segmentation points generated in this step are an ordered list of timestamps, each of which represents an optimal segmentation point in the time-synchronized trajectory calculated based on the principle of minimum description length. Logically, these preliminary candidate segmentation points represent critical points where the vehicle's motion pattern undergoes significant changes, based solely on kinematics and information compression. These preliminary candidate segmentation points serve as input to the subsequent semantic verification step, which uses the reasoning capabilities of the VLM to filter these purely mathematical segmentation points and eliminate redundant points that do not semantically constitute a shift in driving intent.

[0073] The implementation method of step F2F3F1 includes: step F2F3F1F1, finding the previous and next segmentation points of the candidate point in the preliminary candidate segmentation point list, and defining two adjacent motion segments centered on the candidate point. Step F2F3F1F2, extracting corresponding video slices from the visual context video based on the time range of the two adjacent motion segments. Step F2F3F1F3, extracting corresponding road network information from the high-definition map data based on the spatial range of the two adjacent motion segments. Step F2F3F1F4, combining the extracted video slices and road network information to construct a visual language model semantic verification context.

[0074] The visual language model semantic verification context generated in this step is a temporary data structure that contains multimodal information around a specific candidate point. In terms of data form, it aggregates a video slice and a piece of vectorized map data. Its logical function is to provide the visual language model with a centralized spatiotemporal context closely related to the segmentation point to be verified. The transfer purpose of the visual language model semantic verification context is to serve as direct input for the next VLM query, enabling the VLM to determine whether the segmentation point is semantically reasonable based on sufficient visual and geospatial evidence.

[0075] The candidate kinematic segments generated in this step are a list of trajectory segments, each of which is a subsequence of a time-synchronized trajectory, and whose segmentation boundaries have been verified by both MDL optimality calculations and VLM semantic plausibility. Logically, the candidate kinematic segments represent a collection of kinematically stable and semantically complete behavioral units, laying a solid foundation for subsequent explicit semantic labeling. These candidate kinematic segments serve as input to step F3, which assigns each segment an explicit label from a semantic behavior ontology.

[0076] After generating a set of kinematically and semantically complete candidate kinematic segments, the next step is to assign explicit, human-understandable semantic labels to these anonymous behavioral units. The logical intent of this step is to leverage the powerful zero-shot classification capabilities of the visual language model (VLM) to map the visual and map contextual information of each segment into a predefined driving behavior vocabulary. By constructing a cue word containing multimodal information and constraining the VLM to select from a semantic behavior ontology, a preliminary high-level semantic label can be generated for each segment, completing the initial conversion from continuous trajectory to discrete semantic sequence.

[0077] In step F3, the candidate kinematic segments are traversed and the visual language model is used to assign initial semantic labels to each segment in combination with the visual context video, high-definition map data and the predefined semantic behavior ontology library to obtain the initial annotated trajectory.

[0078] The semantic behavior ontology is a closed list of strings predefined by domain experts. This list, constructed following the principles of driving behavior taxonomy, aims to exhaustively enumerate all possible basic driving actions or intentions in the target scenario, such as <driving straight at a constant speed>, <accelerating to overtake>, <slowing down to yield>, and <waiting at an intersection>. Each label is an atomic semantic primitive. The quantitative form of this ontology is a fixed enumeration set, which provides a constrained and standardized output space for the subsequent VLM labeling task, ensuring label consistency and interpretability.

[0079] This step leverages the powerful capabilities of visual language models in zero-shot or few-shot visual question answering (VQA) and description generation. By providing rich multimodal context, such as videos and maps, and constraining it to choose from a predefined semantic behavior ontology, a preliminary, high-level semantic label is generated for each kinematically and semantically complete segment.

[0080] The implementation method of step F3 includes: step F3F1, initializing an empty initial annotation trajectory. Step F3F2, for each motion segment in the candidate kinematic segmentation, looping through steps F3F2F1 to F3F2F5: step F3F2F1, extracting the time slice corresponding to the motion segment in the visual context video to obtain a segmented video slice. Step F3F2F2, extracting the road network information corresponding to the motion segment in the high-definition map data to obtain a segmented map context. Step F3F2F3, constructing a prompt word, which includes the segmented video slice, the segmented map context, and an instruction to select the most appropriate label from the semantic behavior ontology library. Step F3F2F4, using the constructed prompt word to query a visual language model and parse its output to obtain a segmented semantic label. Step F3F2F5, appending the motion segment and its corresponding segmented semantic label as an entry to the initial annotation trajectory.

[0081] The initial annotated trajectory generated by this step is a structured list, in which each entry contains a motion segment, that is, a sequence of spatiotemporal coordinates, and its corresponding segmented semantic label initially assigned by VLM. Logically, the initial annotated trajectory is the first data product in the entire process to fully link continuous kinematic data with discrete semantic concepts. However, since VLM's annotation is performed independently in segments, the trajectory may have logical inconsistencies, for example, an action is followed by an unreasonable one. Therefore, the purpose of the initial annotated trajectory is to serve as the input of the next iterative refinement process to correct possible noise and logical errors.

[0082] Although the initial annotated trajectory generated in step F3 assigns semantic meaning to each motion segment, since the VLM annotates each segment independently, the output label sequence may lack global logical consistency. For example, the model may identify "acceleration" in one segment and "stop" in the next segment, ignoring that a "deceleration" transition is usually required between these two behaviors in real driving situations. To address this issue and ensure that the final output representation is logically self-consistent, the subsequent steps will introduce an iterative self-correction loop. The logical intention of this move is to explicitly treat the initial labels as hypotheses that may contain noise and use a robust learning framework to systematically detect and correct these logical inconsistencies.

[0083] In step F4, based on the visual context video and high-definition map data, as well as the preset referee visual language model confidence threshold, the maximum number of refinement iterations, and the collaborative teaching reasoning path set, an iterative loop including the referee visual language model evaluation and collaborative teaching correction mechanism is used to process the initial labeled trajectory until the logical consistency of the label sequence converges, and the refined labeled trajectory is output.

[0084] The referee visual language model confidence threshold is a floating-point number between 0 and 1, and it is a key design threshold. The logical basis for setting this threshold is that during the iterative consistency refinement process, a quantitative standard is needed to determine whether the evaluation result of a label transformation by the "referee" VLM is sufficiently reliable. When the confidence score output by the VLM is lower than this threshold, even if it is judged to be "logically consistent", the system will regard it as a low-confidence, potentially problematic "noise" sample and trigger the correction process. The specific setting of this value is usually achieved through experiments on the validation set to find a point that achieves the best balance between recall and precision, that is, to find the best balance between identifying truly inconsistent transformations and avoiding misjudging consistent transformations as inconsistent.

[0085] The maximum number of refinement iterations is a positive integer that serves as one of the termination criteria for the iterative process. The logic behind this setting is to prevent the system from failing to reach logically consistent convergence conditions in certain special cases, such as oscillating between two conflicting correction schemes and falling into an infinite loop. Determining this value is an engineering trade-off: it needs to be large enough to ensure that the algorithm has an adequate chance of converging to a stable state in most cases, yet not too large to avoid unnecessary computational overhead. Its specific value is typically set based on empirical observations of convergence speed in typical scenarios.

[0086] The co-teaching reasoning path set is a list of two different reasoning configurations. The logical basis for its setting is a core requirement of the Co-Teaching paradigm: the two models or reasoning processes involved in co-teaching must be diverse to avoid them falling into the same mindset and reinforcing each other's errors. To achieve this, each configuration in the list aims to create an independent and different reasoning path from the other. This difference can be quantified and achieved in a variety of ways, such as: (1) using two different multimodal thought chain MCoT prompt word templates to guide the VLM to reason from different perspectives or in different steps; (2) calling two different VLM API endpoints, which may point to models of different versions or different architectures. The co-teaching reasoning path set ensures the diversity of the two candidate correction labels generated, which is the key to effectively identifying and correcting systematic biases.

[0087] The iterative correction mechanism adopted in this step is theoretically based on the co-teaching paradigm in the field of robust learning. Co-teaching has been shown to be very effective in processing training data with noisy labels. The core idea is to train two or more models simultaneously and let them "teach" each other samples that they each consider to be "clean", that is, small-loss samples, thereby effectively filtering out noise and preventing the model from memorizing and overfitting incorrect labels. This scheme innovatively applies this paradigm to correct the semantic label sequence generated by VLM. By ensuring the diversity of the two "teacher" VLMs through the co-teaching reasoning path set, it is possible to effectively break the self-reinforcing confirmation bias that may be generated by a single model during iteration, thereby significantly improving the logical consistency and robustness of the final label sequence.

[0088] The core innovation of this step is that it explicitly models the initial output of the VLM as potentially noisy data and, for the first time, applies the co-teaching concept from robust learning to the VLM's iterative self-correction process. Traditional methods typically use a one-time feedforward annotation, but the self-consistency of the VLM can lead to iterative reinforcement of its initial biases or errors.

[0089] Co-teaching is an effective paradigm for robust training in the presence of label noise. Its core mechanism is to train two models simultaneously and have them "teach" each other what are considered clean examples—that is, labels with high confidence and logical consistency. By introducing two diverse reasoning paths to generate correction suggestions, which are guaranteed by a set of co-teaching reasoning paths, this step proactively breaks the self-reinforcing cycle of a single model and effectively filters out label errors that appear reasonable in isolated fragments but are illogical in the context of a continuous narrative, significantly improving the logical consistency and reliability of the final representation.

[0090] The implementation method of step F4 includes: step F4F1, copying the initial labeled trajectory as the current iterative trajectory. Step F4F2, initializing the Boolean convergence flag to false and the integer iteration counter to 0. Step F4F3, under the condition that the convergence flag is false and the iteration counter is less than the maximum number of refinement iterations, looping steps F4F3F1 to F4F3F4: step F4F3F1, marking the current iterative trajectory as the previous round trajectory, and setting the convergence flag to true. Step F4F3F2, using the "referee" VLM, combining the visual context video and the pre-collected high-definition map data, and based on the preset referee visual language model confidence threshold, evaluate the logical consistency of all adjacent label pairs in the current iterative trajectory, and identify all "noisy" labels and their indexes that are judged to be inconsistent or have a confidence level lower than the threshold, and obtain a noisy label index set. In step F4F3F3, if the noisy label index set is not empty, execute steps F4F3F3F1 and F4F3F3F2: In step F4F3F3F1, set the convergence flag to false. In step F4F3F3F2, for each noise index in the noisy label index set, loop through steps F4F3F3F2F1 to F4F3F3F2F4: In step F4F3F3F2F1, query the visual language model for the noisy semantic label corresponding to the noise index using the first and second reasoning configurations contained in the collaborative teaching reasoning path set, respectively, to obtain candidate revised semantic label 1 and its confidence score 1, as well as candidate revised semantic label 2 and its confidence score 2. In step F4F3F3F2F2, if the candidate revised semantic label 1 is the same as the candidate revised semantic label 2, select the same candidate revised semantic label as the final revised semantic label. Otherwise, in step F4F3F3F2F3, the candidate revised semantic label with the higher confidence score between candidate revised semantic label 1 and candidate revised semantic label 2 is selected as the final revised semantic label. In step F4F3F3F2F4, the final revised label is used to update the label at the noise index in the current iteration trajectory. In step F4F3F4, the iteration counter is incremented by one. In step F4F4, the final current iteration trajectory is named the refined labeled trajectory.

[0091] The implementation method of step F4F3F2 includes: step F4F3F2F1, initializing an empty noisy label index set. Step F4F3F2F2, traversing all semantic labels from the first to the second to last in the current iteration trajectory, and for each pair of adjacent semantic labels, looping through steps F4F3F2F2F1 to F4F3F2F2F4: step F4F3F2F2F1, extracting the visual context video slices and high-definition map data corresponding to the pair of adjacent semantic labels as multimodal context. Step F4F3F2F2F2, constructing a multimodal thinking chain MCoT prompt word to guide the "referee" VLM to perform step-by-step reasoning, for example: "Step 1: Identify the previous behavior as Step 2: Identify the next action Step 3: Check the video and map to determine arrive Whether the conversion conforms to the driving logic in the current scenario. Step 4: Output the judgment of 'yes' or 'no' and give a confidence score from 0 to 1. "Step F4F3F2F2F3, use the constructed prompt words and multimodal context to query the "referee" VLM to obtain the logical consistency judgment and confidence score. Step F4F3F2F2F4, if the logical consistency judgment is "no" or the confidence score is lower than the referee visual language model confidence threshold, then the index and Add to the noisy label index set.

[0092] Multimodal Chain-of-Thought (MCoT) prompting technology can significantly improve the model's performance on complex multimodal tasks by guiding it to generate a series of intermediate reasoning steps. This step uses MCoT to ensure that the VLM's judgments are not based on simple pattern matching, but on a structured and logical understanding of the scenario.

[0093] The implementation method of step F4F3F3F2F1 includes: step F4F3F3F2F1F1, extracting the noisy semantic label corresponding to the noise index and its multimodal context. Step F4F3F3F2F1F2, using the first reasoning configuration and multimodal context in the collaborative teaching reasoning path set, the first reasoning configuration can be, for example, the first MCoT prompt word template, and the multimodal context can be, for example, video slices, map information, querying the visual language model, and obtaining candidate revised label 1 and its confidence. Step F4F3F3F2F1F3, using the second reasoning configuration and multimodal context in the collaborative teaching reasoning path set, the second reasoning configuration can be, for example, the second MCoT prompt word template or a different VLM model, querying the visual language model, and obtaining candidate revised label 2 and its confidence.

[0094] In ensemble learning and robust learning, model diversity is key to improving performance. This step enforces diversity by using two different reasoning paths, ensuring that the two candidate correction labels generated do not come from the same mindset, which is crucial for effectively identifying and correcting systematic biases.

[0095] The refined annotated trajectory generated in this step is a structured list. Its data format is identical to the initial annotated trajectory, but its core logical value lies in the semantic label sequence it contains, which has been iteratively verified and corrected for logical consistency. The refined annotated trajectory represents a verified, semantically coherent narrative of driving behavior. The refined annotated trajectory serves as input to the final formatting step, generating the final structured semantic trajectory representation for downstream tasks.

[0096] Step F5, traverse each segment in the refined annotated trajectory, combine the semantic label, start time and end time of the segment into a standardized text representation, and concatenate the text representations of all segments into a string to obtain a structured semantic trajectory representation.

[0097] The structured semantic trajectory representation generated in this step is a single, formatted string. In data form, it consists of a series of substrings in the format of labels (t_start = start time, t_end = end time) concatenated by spaces. Logically, the structured semantic trajectory representation is the final deliverable of the entire technical solution. It completely and unambiguously abstracts a complex, continuous vehicle motion process into a discrete, high-level, human- and machine-readable sequence of semantic symbols. The purpose of the structured semantic trajectory representation is to serve as an ideal input for downstream advanced intelligent tasks such as behavior prediction, intent reasoning, and risk assessment. By providing preprocessed and logically verified structured information, it greatly simplifies the complexity of these tasks.

[0098] This embodiment details the complete process of converting raw multimodal sensor data streams into structured semantic representations. The process begins by extracting smooth, time-synchronized trajectories from the raw lidar point cloud and raw video stream through Kalman filtering. The trajectory is then processed by a segmentation algorithm based on the minimum description length principle to generate candidate kinematic segments. These segments are initially labeled by the visual language model to form initial labeled trajectories. Finally, through an iterative refinement cycle based on co-teaching, the initial labeled trajectory is corrected for logical consistency to produce a refined labeled trajectory, which is finally formatted into a structured semantic trajectory representation.

[0099] This process successfully transforms a continuous, low-dimensional kinematic problem into a discrete, high-dimensional semantic reasoning problem by systematically combining optimal state estimation, information theory, multimodal AI reasoning, and robust learning theory. Figure 3 As shown in Figure 3, the final product of this process is a structured semantic trajectory representation, which provides an unprecedented, highly abstract, logically verified, and machine-readable input for the downstream autonomous driving planning and decision-making system, thereby fundamentally solving the semantic gap from raw observations to high-level intent understanding.

[0100] Example 2

[0101] See Figure 2 As shown, this embodiment provides a motion trajectory construction system based on a visual large model, the system comprising:

[0102] The extraction module applies a Kalman filter-based target tracking algorithm based on the collected raw lidar point cloud and raw video stream, as well as the preset Kalman filter state transition matrix, Kalman filter process noise covariance matrix, and Kalman filter measurement noise covariance matrix. It extracts the target vehicle's motion trajectory and aligns it with the video timestamp to obtain a time-synchronized trajectory and visual context video.

[0103] A segmentation module processes the time-synchronized trajectory and visual context video using a hybrid segmentation method that combines the minimum description length principle with semantic verification of a visual language model based on pre-collected high-definition map data, a preset minimum description length model primitive set, and minimum description length encoding accuracy. The method generates candidate kinematic segments whose segmentation points are determined according to the minimum description length principle and verified by the visual language model to ensure the continuity of driving intention.

[0104] The initial annotation module traverses the candidate kinematic segments and uses the visual language model to assign initial semantic labels to each segment, combining visual context video, high-definition map data, and a predefined semantic behavior ontology library to obtain the initial annotated trajectory;

[0105] The annotation refinement module, based on the visual context video and HD map data, as well as the preset judge visual language model confidence threshold, maximum number of refinement iterations, and collaborative teaching reasoning path set, uses an iterative cycle including judge visual language model evaluation and collaborative teaching correction mechanism to process the initial annotation trajectory until the logical consistency of the label sequence converges, and outputs the refined annotation trajectory;

[0106] The semantic trajectory module traverses each segment in the refined and labeled trajectory, combines the semantic label, start time and end time of the segment into a standardized text representation, and concatenates the text representations of all segments into a string to obtain a structured semantic trajectory representation.

[0107] Anything not described in this application can be achieved by adopting or drawing on existing technologies.

[0108] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0109] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A motion trajectory construction method based on a large visual model, characterized in that: include: Step F1, based on the collected original lidar point cloud and original video stream, as well as the preset Kalman filter state transition matrix, Kalman filter process noise covariance matrix, and Kalman filter measurement noise covariance matrix, apply a Kalman filter-based target tracking algorithm to extract the target vehicle's motion trajectory and align it with the video timestamp to obtain a time-synchronized trajectory and visual context video; Step F2: Based on pre-collected high-definition map data, a preset minimum description length model primitive set, and a minimum description length encoding accuracy, a hybrid segmentation method combining the minimum description length principle with semantic verification of a visual language model is applied to process the time-synchronized trajectory and the visual context video to generate candidate kinematic segments, wherein segmentation points are determined according to the minimum description length principle and verified by the visual language model to ensure continuity of driving intention. Step F3, traverse the candidate kinematic segments and use the visual language model to assign initial semantic labels to each segment in combination with the visual context video, high-definition map data and the predefined semantic behavior ontology library to obtain the initial annotated trajectory; Step F4: Based on the visual context video and HD map data, as well as the preset judge visual language model confidence threshold, maximum number of refinement iterations, and collaborative teaching reasoning path set, an iterative cycle including judge visual language model evaluation and collaborative teaching correction mechanism is used to process the initial labeled trajectory until the logical consistency of the label sequence converges, and the refined labeled trajectory is output; Step F5, traverse each segment in the refined annotated trajectory, combine the semantic label, start time and end time of the segment into a standardized text representation, and concatenate the text representations of all segments into a string to obtain a structured semantic trajectory representation.

2. The method for constructing a motion trajectory based on a visual large model according to claim 1, characterized in that: The implementation method of step F2 includes: Steps F2 and F1 apply a hybrid segmentation method based on the minimum description length principle, process the time-series synchronization trajectory according to the preset minimum description length model primitive set and minimum description length encoding accuracy, and generate preliminary candidate segmentation points that are optimal in terms of information compression; Step F2F2, initialize an empty final segmentation point list; In step F2F3, for each candidate point in the preliminary candidate segmentation points, steps F2F3F1 to F2F3F3 are executed in a loop: in step F2F3F1, two adjacent motion segments with the candidate point as the segmentation point are extracted, and information required for semantic verification is extracted from the visual context video and the high-definition map data to construct a visual language model semantic verification context; in step F2F3F2, prompt words are constructed to guide the visual language model to perform step-by-step reasoning, and the prompt words and the visual language model semantic verification context are used to query the visual language model to determine whether the two adjacent motion segments together constitute a semantically continuous driving intention, thereby obtaining a verification result; in step F2F3F3, if the verification result indicates that the two adjacent motion segments together constitute a semantically continuous driving intention, the candidate point is added to the final segmentation point list; In steps F2 and F4, the final segmentation point list is used to segment the time-series synchronization trajectory to obtain candidate kinematic segments.

3. The method for constructing a motion trajectory based on a visual large model according to claim 1, characterized in that: The implementation method of step F4 includes: Step F4F1, copy the initial annotation trajectory as the current iteration trajectory; Step F4F2, initialize the Boolean convergence flag to false and the integer iteration counter to 0; Step F4F3, under the condition that the convergence flag is false and the iteration counter is less than the maximum number of refinement iterations, loop through steps F4F3F1 to F4F3F4: Step F4F3F1, mark the current iteration trajectory as the previous round trajectory, and set the convergence flag to true; Step F4F3F2, for each pair of adjacent semantic labels in the current iteration trajectory, guide a visual language model as a referee by constructing a prompt word, combine the visual context video and high-definition map data corresponding to the pair of adjacent semantic labels, judge the logical consistency of the pair of adjacent semantic labels and output a confidence score, and then identify the noisy semantic labels whose logical consistency is judged as false or whose confidence score is lower than the preset referee visual language model confidence threshold, and obtain a noisy label index set; Step F4F3F3, if the noisy label index set is not empty, execute steps F4F3F3F1 and F4F3F3F2: Step F4F3F3F1, set the convergence flag to false; Step F4F3F3F2, for the noisy labels For each noise index in the signature index set, steps F4F3F3F2F1 to F4F3F3F2F4 are executed in a loop: Step F4F3F3F2F1, for the noisy semantic label corresponding to the noise index, respectively use the first reasoning configuration and the second reasoning configuration contained in the collaborative teaching reasoning path set to query the visual language model to obtain candidate revised semantic label 1 and its confidence score 1, as well as candidate revised semantic label 2 and its confidence score 2; Step F4F3F3F2F2, if the candidate revised semantic label 1 is the same as the candidate revised semantic label 2, then the same candidate revised semantic label is selected as the final revised semantic label; Step F4F3F3F2F3, otherwise, select the candidate revised semantic label with a higher confidence score between the candidate revised semantic label 1 and the candidate revised semantic label 2 as the final revised semantic label; Step F4F3F3F2F4, use the final revised label to update the label at the noise index in the current iteration trajectory; Step F4F3F4, increment the iteration counter by one; In step F4F4, the final current iteration trajectory is named as the refined labeled trajectory.

4. The method for constructing a motion trajectory based on a visual large model according to claim 2, characterized in that: The implementation method of steps F2 and F1 includes: Steps F2F1F1 define and calculate the total description length of any subsequence of the time-synchronized trajectory, the total description length being calculated by adding the number of bits required to describe the model parameters of a geometric model selected from a preset set of model primitives of minimum description length used to fit the subsequence to the number of bits required to encode the fitting errors of all trajectory points in the subsequence to the geometric model; Step F2F1F2, initialize the cost array and the split point index array; Steps F2, F1, and F3 apply a dynamic programming algorithm to determine and record the optimal segmentation point required to achieve the minimum total description length through an iterative solution process. The solution process includes: for each current trajectory point in the time-series synchronization trajectory, traverse all trajectory points before the current trajectory point as candidate segmentation points, calculate the cumulative total description length when the candidate segmentation point is used as the last segmentation point, select the candidate segmentation point that minimizes the cumulative total description length as the optimal predecessor segmentation point of the current trajectory point, and store the selected minimum cumulative total description length and the index of the optimal predecessor segmentation point in the cost array and the segmentation point index array, respectively; Steps F2, F1, and F4 are to extract all the optimal segmentation points by backtracking the segmentation point index array to obtain preliminary candidate segmentation points. The backtracking method is as follows: starting from the end point of the timing synchronization trajectory, reversely search for the optimal predecessor segmentation point corresponding to the end point using the segmentation point index array, and repeatedly perform reverse search using the found optimal predecessor segmentation point as the search starting point until the starting point of the trajectory is traced back, and record all the optimal segmentation points found during the entire backtracking process.

5. The method for constructing a motion trajectory based on a visual large model according to claim 2, characterized in that: The implementation method of steps F2, F3, and F1 includes: Steps F2F3F1F1: find the previous and next segmentation points of the candidate point in the preliminary candidate segmentation point list, and define two adjacent motion segments centered on the candidate point; Steps F2F3F1F2: extracting corresponding video slices from the visual context video according to the time range of the two adjacent motion segments; Steps F2F3F1F3: extracting corresponding road network information from the high-definition map data based on the spatial range of the two adjacent motion segments; In steps F2, F3, F1, and F4, the extracted video slices and road network information are combined to construct a semantic verification context for the visual language model.

6. The method for constructing a motion trajectory based on a visual large model according to claim 3, characterized in that: The implementation method of steps F4, F3, and F2 includes: Steps F4F3F2F1, initialize an empty noisy label index set; Step F4F3F2F2, traverse all semantic labels from the first to the second to last in the current iteration trajectory, and for each pair of adjacent semantic labels, loop through steps F4F3F2F2F1 to F4F3F2F2F4: Step F4F3F2F2F1, extract the visual context video slices and high-definition map data corresponding to the pair of adjacent semantic labels as multimodal context; Step F4F3F2F2F2, construct multimodal thinking chain prompt words, which are used to guide the visual language model as the referee to perform step-by-step reasoning, which includes: The next steps are: identifying the previous semantic label, identifying the next semantic label, and judging whether the conversion from the previous semantic label to the next semantic label conforms to the driving logic based on the multimodal context; step F4F3F2F2F3, using the constructed prompt words and the multimodal context to query the visual language model serving as the referee to obtain a logical consistency judgment and a confidence score; step F4F3F2F2F4, if the logical consistency judgment is no or the confidence score is lower than the confidence threshold of the referee visual language model, then adding the index of the pair of adjacent semantic labels to the noisy label index set.

7. The method for constructing a motion trajectory based on a visual large model according to claim 3, characterized in that: The implementation method of steps F4F3F3F2F1 includes: Step F4F3F3F2F1F1, extracting the noisy semantic label corresponding to the noise index and its multimodal context; Step F4F3F3F2F1F2, adopting the first reasoning configuration, using the multimodal context to query the visual language model, and obtaining a candidate revised semantic label 1 and a confidence score 1; Step F4F3F3F2F1F3, adopting the second reasoning configuration, using the multimodal context to query the visual language model, and obtaining the candidate revised semantic label 2 and the confidence score 2; Among them, the first reasoning configuration is different from the second reasoning configuration, and the difference lies in that the multimodal thinking chain prompt word template used to query the visual language model is different, or the queried visual language model is a model of different version or different architecture.

Images