An assisted driving method and system for predicting object trajectories on a motorcycle intelligent ar helmet

Abstract

The application discloses an auxiliary driving method and system for predicting object trajectory on a motorcycle intelligent AR helmet, belongs to the technical field of motion trajectory prediction, inputs an observation trajectory into a preset trajectory prediction model, processes the observation trajectory through the trajectory prediction model, and obtains a predicted trajectory, which comprises the following steps: after normalizing the observation trajectory, the observation trajectory is mapped to a frequency domain space through DCT coding to obtain observation trajectory frequency domain coefficients; a target is taken as a node, and an interaction relationship between targets is taken as an edge to obtain a target interaction graph; a feature of the observation trajectory frequency domain coefficients is taken as a node, and spatiotemporal interaction processing between targets is performed in combination with the target interaction graph to obtain preliminary prediction coefficients; based on a preset trajectory mode prototype of a future trajectory and the observation trajectory frequency domain coefficients, a trajectory mode representation is obtained; based on the preliminary prediction coefficients and the trajectory mode representation, correction prediction coefficients are obtained; after the correction prediction coefficients are subjected to DCT decoding and then are subjected to inverse normalization, the predicted trajectory is obtained; and driving assistance is performed based on the predicted trajectory.

Description

Technical Field

[0001] This application belongs to the field of motion trajectory prediction technology, specifically relating to an assisted driving method and system for predicting object trajectories on a motorcycle smart AR helmet. Background Technology

[0002] Motorcycle riders are very concerned about the safety threats posed by pedestrians when driving at low speeds, such as on rural roads or in residential areas. This objectively creates a need for certain safety assistance functions to help motorcycle riders make judgments. As a result, people have developed smart AR helmets for motorcycle riders that are equipped with external cameras and recorders as well as internal AR prompts. These smart AR helmets can also be connected to each other via the Internet.

[0003] While existing smart AR helmets already incorporate camera recorders, motorcycles are significantly less stable than cars during driving. Therefore, the video footage captured by these recorders contains more noise, and the motorcyclist's head is also constantly moving. Consequently, the trajectory analysis algorithms used in traditional cars are unsuitable for the realities of motorcycles, necessitating the development of new algorithms.

[0004] In low-speed environments such as rural roads and residential areas, where people are mixed together, traffic accidents are prone to occur, which is an urgent need for the application scenarios of this invention. However, under such low-speed conditions, drivers and computers also have a certain calculation and reaction time. Therefore, this engineering environment is suitable for using smart AR helmets for trajectory analysis and risk warning, and it is also suitable for implementing early warnings among convoy members using smart AR helmets.

[0005] Theoretically, in such a real open environment, predicting the trajectory of an object requires not only characterizing its continuous temporal evolution based on its historical motion state, but also considering the spatial interactions between individuals to reasonably infer multiple possible future trajectories. In this process, existing methods often face the following two important scientific problems:

[0006] First, there is the problem of insufficient utilization of the continuous and smooth features of the trajectory. In real-world scenarios, pedestrian trajectories typically exhibit continuity and smoothness. This temporal evolution pattern itself is important prior information for improving the accuracy and reasonableness of trajectory prediction. This is also a solution to the instability caused by motorcycle stability. If the model cannot effectively extract and utilize these features during the prediction process and only focuses on discrete position changes, the predicted trajectory may fail to reflect the continuous motion process, thus affecting the stability and realism of the results.

[0007] Second, there is a problem of insufficient multimodal behavior modeling. The same historical trajectory often corresponds to multiple possible future behaviors, such as the other party continuing to go straight, suddenly turning, slowing down and stopping, or detouring to avoid it. If the model only outputs a single prediction result when deployed at the edge, it is easy to miss potential high-risk movement patterns, making it difficult for downstream systems to make safety decisions in advance based on the prediction.

[0008] Therefore, how to fully explore the continuous and smooth evolution law in the trajectory in a complex and open environment, and effectively characterize the multimodal uncertainty of future behavior, so as to generate prediction results with accuracy, stability and security, is an important research problem for achieving high-quality trajectory prediction. Summary of the Invention

[0009] Purpose of the invention: This application develops an assisted driving method and system for predicting object trajectories on a motorcycle smart AR helmet, aiming to solve the technical problems in the prior art.

[0010] Technical Solution: In a first aspect, embodiments of this application provide an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, comprising inputting the observed trajectory of a target into a preset trajectory prediction model, processing the observed trajectory through the trajectory prediction model, and obtaining a predicted trajectory, including:

[0011] After normalizing the observation trajectory, it is mapped to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observation trajectory;

[0012] Using the target as nodes and the interaction relationships between the targets as edges, obtain the target interaction graph;

[0013] Using the frequency domain coefficients of the observed trajectory as the features of the node, and combining them with the target interaction graph, spatiotemporal interaction processing is performed between the targets to obtain preliminary prediction coefficients;

[0014] Based on the preset future trajectory pattern prototype and the frequency domain coefficients of the observed trajectory, a trajectory pattern representation is obtained;

[0015] Based on the preliminary prediction coefficients and the trajectory pattern representation, corrected prediction coefficients are obtained;

[0016] After DCT decoding of the modified prediction coefficients, inverse normalization is performed to obtain the predicted trajectory;

[0017] Driving assistance is provided based on the predicted trajectory.

[0018] In some embodiments, the step of obtaining the frequency domain coefficients of the observation trajectory includes:

[0019] Based on the position of the target at each time step in the observed trajectory, DCT encoding is performed to obtain orthogonal DCT basis functions. The characterization formulas of the orthogonal DCT basis functions include:

[0020] ;

[0021] in, The orthogonal DCT basis functions are... , This represents the maximum length of the observation time. For frequency index, used to characterize the first One DCT basis function; The number of DCT basis functions;

[0022] The orthogonal DCT basis functions are extended to the trajectory space based on the Kronecker product to obtain the first orthogonal basis matrix. The representation formula of the first orthogonal basis matrix includes:

[0023] ;

[0024] in, This is the first orthogonal basis matrix; It is a two-dimensional identity matrix;

[0025] The frequency domain coefficients of the observation trajectory are obtained based on the first orthogonal basis matrix and the observation trajectory. The characterization formula of the frequency domain coefficients of the observation trajectory includes:

[0026] ;

[0027] in, The frequency domain coefficients of the observed trajectory; The first orthogonal basis matrix transpose; For the target The observation trajectory.

[0028] In some embodiments, the step of using the frequency domain coefficients of the observed trajectory as features of the node, and combining them with the target interaction graph to perform spatiotemporal interaction processing between the targets to obtain preliminary prediction coefficients includes:

[0029] Using the frequency domain coefficients of the observed trajectory as node features, a spatiotemporal convolutional network is used to perform spatial convolution on the target interaction graph, and spatial interaction modeling is performed by combining convolution operations in the time dimension to obtain the spatial interaction output. The representation formula of the spatial interaction output includes:

[0030] ;

[0031] in, For the spatial interaction output of the first Layer feature representation; For the spatial interaction output of the first Layer feature representation; The adjacency matrix under different spatial relationships is used to characterize the topological structure of the target interaction graph; These are learnable parameters; It is a non-linear activation function;

[0032] The spatial interaction output is subjected to multi-layer one-dimensional convolution operations by a temporal prediction network to obtain the preliminary prediction coefficients.

[0033] In some embodiments, the step of presetting a trajectory pattern prototype for the future trajectory includes:

[0034] During the model training phase of the target trajectory model, the future actual trajectory of the target, which serves as the model input sample, is mapped to the frequency domain space based on the DCT orthogonal basis to obtain the frequency domain coefficients of the future actual trajectory.

[0035] K-meas clustering is performed on the frequency domain coefficients of the future actual trajectory, and trajectory pattern prototypes with cluster centers of the future trajectory are obtained. The characterization formula of the trajectory pattern prototype includes:

[0036] ;

[0037] in, For the first The cluster centers of the i-th cluster are used to characterize the i-th cluster. The aforementioned trajectory pattern prototype; The number of clusters is used to characterize the number of trajectory pattern prototypes; for A set of samples for each cluster; For the target of the sample The frequency domain coefficients of the future actual trajectory.

[0038] In some embodiments, the step of obtaining the trajectory pattern representation based on a preset future trajectory pattern prototype and the frequency domain coefficients of the observed trajectory includes:

[0039] A pattern weight generator based on a multilayer sensing mechanism processes the frequency domain coefficients of the observed trajectory to obtain the trajectory pattern combination weight. The representation formula of the trajectory pattern combination weight includes:

[0040] ;

[0041] in, For the pattern weight generator; The trajectory pattern is combined with weights. The Line number Column elements are denoted as , for the first The aforementioned objective is in the first For the candidate predicted trajectory, the first The response strength of a trajectory pattern prototype; For the target The frequency domain coefficients of the observed trajectory;

[0042] Based on the dimensional normalization of the trajectory pattern combination according to the trajectory pattern prototype, normalized weights are obtained. The representation formula of the normalized weights includes:

[0043] ;

[0044] in, For the target In the For the candidate predicted trajectory, the first Normalized weights for each trajectory pattern prototype; The summation index is used to iterate through all trajectory pattern prototypes; The number of the trajectory pattern prototypes;

[0045] The normalized weights and the trajectory pattern prototypes are weighted and combined to obtain a trajectory pattern representation, the representation formula of which includes:

[0046] ;

[0047] ;

[0048] in, This is a representation of the trajectory pattern; The sequence number of the trajectory pattern prototype; The number of the trajectory pattern prototypes; A set including the trajectory pattern prototypes; For the first The aforementioned trajectory pattern prototype. .

[0049] In some embodiments, the step of performing DCT decoding on the corrected prediction coefficients followed by inverse normalization to obtain the predicted trajectory includes:

[0050] DCT decoding is performed on the corrected prediction coefficients to obtain the second orthogonal basis matrix;

[0051] Mapping the second orthogonal basis matrix back to Euclidean space yields the Euclidean space representation, the representation formula of which includes:

[0052] ;

[0053] in, The Euclidean space representation of a one-dimensional vector; This is the second orthogonal basis matrix; The corrected prediction coefficients;

[0054] The Euclidean space representation is reconstructed into a two-dimensional time series and normalized to obtain the predicted trajectory. The formula representing the predicted trajectory includes:

[0055] ;

[0056] in, For the target The predicted trajectory; Scale factor; For the target A two-dimensional time series is used to characterize the predicted trajectory of the operation to be normalized; It is a rotation matrix; It is a translation vector.

[0057] In some embodiments, during the model training phase of the preset trajectory prediction model, the model generates multiple candidate predicted trajectories of the target as samples, compares them with the future actual trajectories of the target as samples, calculates the error using a total loss function, and updates the model parameters. The formula for the total loss function includes:

[0058] ;

[0059] in, The total loss function is... For frequency domain coefficient loss; This represents the average displacement loss in Euclidean space. This represents the displacement loss at the endpoint of Euclidean space.

[0060] ;

[0061] in, The number of targets; The sort number for the target; Candidate predicted trajectories; The total number of candidate predicted trajectories; For the first The first goal in The coefficient representation of each candidate predicted trajectory in the frequency domain; For the first The coefficient representation of the true future trajectory of an objective in the frequency domain;

[0062] ;

[0063] in, The sort number for the target; The maximum length of the future prediction time; For a specific moment; For the goal The Candidate predicted trajectories at future moments The predicted location; For the goal The future moment The actual location;

[0064] ;

[0065] in, For the goal The Candidate predicted trajectories at future moments The predicted location, where ; For the goal In the future The true location, among which .

[0066] In some embodiments, the method further includes:

[0067] Based on the target's speed of motion, the target is classified into a moving target or a stationary target;

[0068] The observed trajectory information of the moving target is input into a preset trajectory prediction model of the moving target to obtain the predicted trajectory of the moving target;

[0069] The observed trajectory information of the stationary target is input into a preset trajectory prediction model for the stationary target to obtain the predicted trajectory of the stationary target;

[0070] The predicted trajectories of the moving target and the stationary target are fused to obtain a comprehensive predicted trajectory.

[0071] In some embodiments, the step of classifying the target into moving targets and stationary targets based on the target's speed includes:

[0072] The average velocity of the target at the end of the observation period is obtained, and the formula for representing the average velocity includes:

[0073] ;

[0074] in, For the target The average speed; For the target At any moment The location, among which , This represents the maximum length of the observation time. For the target At any moment The location, among which ;

[0075] The modulus of the average velocity is obtained, and a motion / static mask is obtained by combining it with a preset modulus threshold. The representation formula of the motion / static mask includes:

[0076] ;

[0077] in, For the target The motion mask; The average speed The modulus length; The modulus threshold;

[0078] Based on the motion mask, the target is classified into the moving target or the stationary target;

[0079] The step of fusing the predicted trajectories of the moving target and the stationary target to obtain a comprehensive predicted trajectory includes:

[0080] Based on the dynamic and static mask, the predicted trajectory of the moving target and the predicted trajectory of the stationary target are fused to obtain a comprehensive predicted trajectory.

[0081] Secondly, embodiments of this application also provide an assisted driving system for predicting object trajectories on a motorcycle smart AR helmet, comprising:

[0082] A trajectory prediction module, wherein the trajectory prediction module is used to input the observed trajectory of the target into a preset trajectory prediction model, process the observed trajectory through the trajectory prediction model, and obtain a predicted trajectory, the trajectory prediction module comprising:

[0083] Frequency domain coefficient unit, which is used to normalize the observation trajectory and then map it to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observation trajectory;

[0084] The graph construction unit is used to obtain a target interaction graph with the target as nodes and the interaction relationships between the targets as edges.

[0085] A spatiotemporal interaction unit is used to perform spatiotemporal interaction processing between the targets by using the frequency domain coefficients of the observed trajectory as the features of the nodes and combining them with the target interaction graph to obtain preliminary prediction coefficients.

[0086] A trajectory pattern unit is used to obtain a trajectory pattern representation based on a preset trajectory pattern prototype of a future trajectory and the frequency domain coefficients of the observed trajectory;

[0087] A prediction correction unit is configured to obtain corrected prediction coefficients based on the initial prediction coefficients and the trajectory pattern representation;

[0088] A prediction output unit is used to perform DCT decoding on the modified prediction coefficients and then perform inverse normalization to obtain the prediction trajectory.

[0089] An assisted driving module, which is used to provide driving assistance based on the predicted trajectory.

[0090] Beneficial Effects: Compared with existing technologies, the present application provides an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet. The method inputs the observed trajectory of the target into a preset trajectory prediction model, processes the observed trajectory through the trajectory prediction model, and obtains the predicted trajectory. This includes: normalizing the observed trajectory and mapping it to the frequency domain space using DCT encoding to obtain the frequency domain coefficients of the observed trajectory; obtaining a target interaction graph with the target as nodes and the interaction relationships between targets as edges; using the frequency domain coefficients of the observed trajectory as features of the nodes and combining them with the target interaction graph to perform spatiotemporal interaction processing between targets to obtain preliminary prediction coefficients; obtaining a trajectory pattern representation based on a preset future trajectory pattern prototype and the frequency domain coefficients of the observed trajectory; obtaining corrected prediction coefficients based on the preliminary prediction coefficients and the trajectory pattern representation; performing DCT decoding on the corrected prediction coefficients and then inverse normalizing them to obtain the predicted trajectory; and providing driving assistance based on the predicted trajectory. This application introduces a trajectory representation method based on Discrete Cosine Transform (DCT). Considering that pedestrian trajectories typically have continuous and smooth evolution characteristics in the time dimension, their motion changes in the frequency domain are often dominated by low-frequency components. Based on this, this application maps time-series trajectories to an orthogonal frequency domain base space and performs frequency decomposition, enabling the model to characterize the overall trend of trajectory changes using a small number of low-frequency coefficients, thereby achieving a compact representation of continuous and smooth features. Compared to modeling directly based on discrete time-domain locations, this application not only helps reduce the oversensitivity of point-by-point time-domain modeling to local fluctuations, but also more fully mines the global structural information contained in the trajectory, enhancing the model's understanding of motion inertia, direction evolution, and overall trends, thereby improving the rationality, stability, and usability of the prediction results. Furthermore, this application constructs trajectory pattern prototypes for future trajectories in the frequency domain space for multimodal behavior modeling problems, forming a structured motion pattern representation space and explicitly characterizing the relationships between different potential motion patterns. Simultaneously, a lightweight weight allocation mechanism is employed to adaptively weight and combine different patterns in the pattern library according to the observed trajectory, achieving flexible expression of multiple potential motion patterns, thereby improving the expressive power and discriminative power of multimodal prediction while controlling model complexity. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 A flowchart illustrating the steps of an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application.

[0093] Figure 2 The flowchart illustrates the steps of obtaining the predicted trajectory by processing the observed trajectory through a trajectory prediction model in the assisted driving method for predicting the trajectory of an object on a motorcycle smart AR helmet provided in this application embodiment.

[0094] Figure 3 A flowchart illustrating the steps for obtaining the frequency domain coefficients of the observed trajectory in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in this application embodiment;

[0095] Figure 4 A flowchart illustrating the steps for obtaining preliminary prediction coefficients in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in this application embodiment;

[0096] Figure 5 A flowchart illustrating the steps of obtaining a trajectory pattern prototype in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application.

[0097] Figure 6 A flowchart illustrating the steps of obtaining a trajectory pattern representation in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application.

[0098] Figure 7 A flowchart illustrating the steps of obtaining a predicted trajectory in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application.

[0099] Figure 8 This is a flowchart illustrating the steps of separating a target into static and dynamic states in a method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application.

[0100] Figure 9 This is a module connection diagram of an assisted driving system for predicting object trajectories on a motorcycle smart AR helmet, provided in an embodiment of this application.

[0101] Figure 10 This is an algorithm structure diagram of the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in an embodiment of this application;

[0102] Figure 11 A schematic diagram of the DCT discrete cosine transform in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in the embodiments of this application;

[0103] Figure 12 This is a schematic diagram illustrating the acquisition of a trajectory pattern prototype and the construction of a trajectory pattern library in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in an embodiment of this application.

[0104] Figure 13A schematic diagram illustrating the principle of the future trajectory pattern library weight allocation mechanism in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment;

[0105] Figure 14 This is a schematic diagram of the structural connection of the driver assistance device for predicting object trajectories on a motorcycle smart AR helmet provided in an embodiment of this application;

[0106] Figure reference numerals: 10, trajectory prediction module; 11, frequency domain coefficient unit; 12, graph construction unit; 13, spatiotemporal interaction unit; 14, trajectory pattern unit; 15, prediction correction unit; 16, prediction output unit; 20, assisted driving module. Detailed Implementation

[0107] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0108] Motorcycle riders are very concerned about the safety threats posed by pedestrians when driving at low speeds, such as on rural roads or in residential areas. This objectively creates a need for certain safety assistance functions to help motorcycle riders make judgments. As a result, people have developed smart AR helmets for motorcycle riders that are equipped with external cameras and recorders as well as internal AR prompts. These smart AR helmets can also be connected to each other via the Internet.

[0109] While existing smart AR helmets already incorporate camera recorders, motorcycles are significantly less stable than cars during driving. Therefore, the video footage captured by these recorders contains more noise, and the motorcyclist's head is also constantly moving. Consequently, the trajectory analysis algorithms used in traditional cars are unsuitable for the realities of motorcycles, necessitating the development of new algorithms.

[0110] In low-speed environments such as rural roads and residential areas, where people are mixed together, traffic accidents are prone to occur, which is an urgent need for the application scenarios of this invention. However, under such low-speed conditions, drivers and computers also have a certain calculation and reaction time. Therefore, this engineering environment is suitable for using smart AR helmets for trajectory analysis and risk warning, and it is also suitable for implementing early warnings among convoy members using smart AR helmets.

[0111] Theoretically, in such a real open environment, predicting the trajectory of an object requires not only characterizing its continuous temporal evolution based on its historical motion state, but also considering the spatial interactions between individuals to reasonably infer multiple possible future trajectories. In this process, existing methods often face the following two important scientific problems:

[0112] First, there is the problem of insufficient utilization of the continuous and smooth features of the trajectory. In real-world scenarios, pedestrian trajectories typically exhibit continuity and smoothness. This temporal evolution pattern itself is important prior information for improving the accuracy and reasonableness of trajectory prediction. This is also a solution to the instability caused by motorcycle stability. If the model cannot effectively extract and utilize these features during the prediction process and only focuses on discrete position changes, the predicted trajectory may fail to reflect the continuous motion process, thus affecting the stability and realism of the results.

[0113] Second, there is a problem of insufficient multimodal behavior modeling. The same historical trajectory often corresponds to multiple possible future behaviors, such as the other party continuing to go straight, suddenly turning, slowing down and stopping, or detouring to avoid it. If the model only outputs a single prediction result when deployed at the edge, it is easy to miss potential high-risk movement patterns, making it difficult for downstream systems to make safety decisions in advance based on the prediction.

[0114] Therefore, how to fully explore the continuous and smooth evolution law in the trajectory in a complex and open environment, and effectively characterize the multimodal uncertainty of future behavior, so as to generate prediction results with accuracy, stability and security, is an important research problem for achieving high-quality trajectory prediction.

[0115] In view of this, embodiments of this application provide an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet. The method involves inputting the observed trajectory of the target into a preset trajectory prediction model, processing the observed trajectory through the trajectory prediction model to obtain a predicted trajectory, including: normalizing the observed trajectory and mapping it to the frequency domain space via DCT encoding to obtain the frequency domain coefficients of the observed trajectory; obtaining a target interaction graph with the target as nodes and the interaction relationships between targets as edges; using the frequency domain coefficients of the observed trajectory as features of the nodes and combining them with the target interaction graph to perform spatiotemporal interaction processing between targets to obtain preliminary prediction coefficients; obtaining a trajectory pattern representation based on a preset trajectory pattern prototype of the future trajectory and the frequency domain coefficients of the observed trajectory; obtaining corrected prediction coefficients based on the preliminary prediction coefficients and the trajectory pattern representation; performing DCT decoding on the corrected prediction coefficients and then inverse normalizing them to obtain the predicted trajectory; and providing driving assistance based on the predicted trajectory. This application introduces a trajectory representation method based on Discrete Cosine Transform (DCT). Considering that pedestrian trajectories typically have continuous and smooth evolution characteristics in the time dimension, their motion changes in the frequency domain are often dominated by low-frequency components. Based on this, this application maps time-series trajectories to an orthogonal frequency domain base space and performs frequency decomposition, enabling the model to characterize the overall trend of trajectory changes using a small number of low-frequency coefficients, thereby achieving a compact representation of continuous and smooth features. Compared to modeling directly based on discrete time-domain locations, this application not only helps reduce the oversensitivity of point-by-point time-domain modeling to local fluctuations, but also more fully mines the global structural information contained in the trajectory, enhancing the model's understanding of motion inertia, direction evolution, and overall trends, thereby improving the rationality, stability, and usability of the prediction results. Furthermore, this application constructs trajectory pattern prototypes for future trajectories in the frequency domain space for multimodal behavior modeling problems, forming a structured motion pattern representation space and explicitly characterizing the relationships between different potential motion patterns. Simultaneously, a lightweight weight allocation mechanism is employed to adaptively weight and combine different patterns in the pattern library according to the observed trajectory, achieving flexible expression of multiple potential motion patterns, thereby improving the expressive power and discriminative power of multimodal prediction while controlling model complexity.

[0116] Accordingly, embodiments of this application also provide an assisted driving system for predicting object trajectories on a motorcycle smart AR helmet; please refer to [link to relevant documentation]. Figure 9 , Figure 9 This is a module connection diagram of a driver assistance system for predicting object trajectories on a motorcycle smart AR helmet provided in an embodiment of this application. The driver assistance system for predicting object trajectories on a motorcycle smart AR helmet provided in an embodiment of this application includes:

[0117] The trajectory prediction module 10 is used to input the observed trajectory of the target into a preset trajectory prediction model, and to process the observed trajectory through the trajectory prediction model. The trajectory prediction module 10 includes:

[0118] Frequency domain coefficient unit 11 is used to normalize the observation trajectory and then map it to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observation trajectory.

[0119] Graph construction unit 12 is used to obtain a target interaction graph with the target as the node and the interaction relationship between the targets as the edge.

[0120] Spatiotemporal interaction unit 13 is used to perform spatiotemporal interaction processing between targets by taking the frequency domain coefficients of the observed trajectory as the feature and combining them with the target interaction graph to obtain preliminary prediction coefficients.

[0121] The trajectory pattern unit 14 is used to obtain the trajectory pattern representation based on the preset trajectory pattern prototype of the future trajectory and the frequency domain coefficients of the observed trajectory.

[0122] Prediction correction unit 15 is used to obtain corrected prediction coefficients based on the preliminary prediction coefficients and trajectory pattern representation;

[0123] Prediction output unit 16 is used to perform inverse normalization on the corrected prediction coefficients after DCT decoding to obtain the prediction trajectory.

[0124] The driver assistance module 20 is used to assist driving based on a predicted trajectory.

[0125] Accordingly, embodiments of this application also provide an assisted driving device for predicting object trajectories on a motorcycle smart AR helmet; please refer to [link to relevant documentation]. Figure 14 , Figure 14 This is a schematic diagram of the structural connection of the driver assistance device for predicting the trajectory of an object on a motorcycle smart AR helmet provided in this application embodiment. The driver assistance device for predicting the trajectory of an object on a motorcycle smart AR helmet provided in this application embodiment includes a smart AR helmet, a computing and communication module disposed on the motorcycle, and a mobile phone.

[0126] Specifically, the smart AR helmet includes an external camera module with a built-in SD card, an AR module, a pose sensor, a near-field communication module, and a helmet that meets national safety standards.

[0127] An external camera module with a built-in SD card is mounted on top of the helmet, facing directly forward to capture and record road conditions. It functions as both a dashcam and a data collector. The recorded video is stored on its SD card and simultaneously transmitted via a short-range communication module to the motorcycle's onboard computing and communication module.

[0128] The AR module is installed in front of the helmet. After receiving various information from the computing and communication module carried by the motorcycle, it projects the information onto the visor in front of the helmet in an AR manner, providing relevant information without affecting the driver's safe observation.

[0129] The pose sensor is installed inside the helmet, with its orientation aligned with the external camera module that has a built-in SD card. Its Z-axis points towards the distance from the camera, its X-axis runs from left to right, and its Y-axis runs from top to bottom. It is used to analyze the rider's head movements. The data it collects is also transmitted via a short-range communication module to the computing and communication module carried by the motorcycle.

[0130] The short-range communication module includes Type-C direct data connection, Bluetooth communication, and Wi-Fi connectivity with a mobile phone. It is installed on the helmet and is responsible for data communication.

[0131] A helmet that meets national safety standards is used to protect the driver's safety and integrates the above modules.

[0132] In a specific embodiment, the present invention uses a helmet designed independently by Nanjing Baituo Vision Technology Co., Ltd., with authorization announcement number CN117122120B and titled "Helmet Display Follow-up System and Method Based on Reinforcement Learning".

[0133] Specifically, the computing and communication module carried by the motorcycle mainly includes three devices: an onboard GPU, a short-range communication module, and a long-range communication module. It is powered by the motorcycle.

[0134] One of the vehicle-mounted GPUs is used to provide computing power, store the assisted driving system for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment, and run the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment. This application uses a small Nano Jetson GPU module.

[0135] A short-range communication module is used to communicate with modules on the helmet and mobile phones. It includes direct Type-C data connection, Bluetooth communication, and Wi-Fi connectivity with mobile phones. It is installed on the helmet and is responsible for data communication.

[0136] A long-range communication module is used to communicate with other teammates in the convoy and report relevant hazard information when mobile phones are unavailable. In this specific embodiment of the invention, a 5.8G wireless digital high-definition image transmission module—the AJ-TC5850 data radio—is used. This radio is chosen because it can transmit image information, allowing hazard information to be sent to teammates in image format.

[0137] Specifically, the mobile phone is carried on the driver's body or fixed on the steering wheel. It communicates with other devices, making it convenient for the driver to debug, observe, and operate the data information of various modules, thus realizing the human-computer interaction of this system.

[0138] In some embodiments, the targets in this application include, but are not limited to, pedestrians, vehicles, animals, airplanes, high-speed trains, trains, and other objects. The specific targets can be determined by actual engineering needs. For ease of description, this application uses pedestrians as an example for illustrative purposes.

[0139] In some embodiments, please refer to Figure 10 , Figure 10 This is an algorithm structure diagram of the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in an embodiment of this application. In the embodiment of this application, before inputting the trajectory information of the target into the trajectory prediction model, the target is divided into moving targets or stationary targets based on the target's speed. Then, the observed trajectory information of the moving target is input into a preset trajectory prediction model for moving targets to obtain the predicted trajectory of the moving target. Similarly, the observed trajectory information of the stationary target is input into a preset trajectory prediction model for stationary targets to obtain the predicted trajectory of the stationary target.

[0140] Understandably, in real-world scenarios, pedestrian motion states exhibit significant heterogeneity, with different individuals potentially in states such as stationary, low-speed, or continuous motion. Stationary or low-speed pedestrians show relatively minor trajectory changes, while moving pedestrians exhibit more pronounced directional and dynamic variations. However, existing methods typically employ a uniform model to model all trajectories, neglecting the distributional differences between different motion states. This can easily lead to the model learning compromise representations, thereby reducing its ability to characterize specific motion patterns. To address this, this application introduces a motion-stationary separation mechanism, classifying trajectories based on pedestrian motion states. During the input phase, observed trajectories are divided into moving and stationary / low-speed categories, and each is processed through two independent branches. This allows the model to learn more targeted representations for different types of trajectories, thereby improving prediction performance and stability.

[0141] In some embodiments, please refer to Figure 8 , Figure 8 This is a flowchart illustrating the steps of separating a target into static and dynamic states in a method for predicting object trajectories on a motorcycle smart AR helmet, as provided in this application embodiment. The method for separating a target into static and dynamic states in this application is specifically implemented through steps c to e:

[0142] Step c: Obtain the target's average velocity at the end of the observation period.

[0143] In some embodiments, for a time length of The observation trajectory, assuming pedestrian At any moment The two-dimensional position coordinates are Then, based on the average velocity at the end of the observation period... The formulas for estimating its motion state and representing average velocity include:

[0144] ;

[0145] in, For the goal The average speed; For the goal At any moment The location, among which , This represents the maximum length of the observation time. For the goal At any moment The location, among which .

[0146] Step d: Obtain the average velocity modulus and combine it with a preset modulus threshold to obtain the motion mask.

[0147] In some embodiments, the representation formula for the static / dynamic mask includes:

[0148] ;

[0149] in, For the goal The motion mask; average speed The modulus length; This is the modulus threshold.

[0150] Step e: Classify the target into moving targets or stationary targets based on the motion mask.

[0151] Understandably, based on this mask, all observed trajectories are divided into moving branches and stationary branches. The data from the two branches are then normalized, frequency-domain mapped, and trajectory predicted, respectively. At the same time, the model also constructs independent anchor prototype libraries for the two types of trajectories, namely, fixed pattern libraries containing trajectory pattern prototypes of future trajectories, to enhance the representation capability under different motion patterns.

[0152] In some embodiments, please refer to Figure 1 and Figure 10 , Figure 1 This is a flowchart illustrating the steps of an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. Figure 10This is a diagram illustrating the algorithm structure of the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment is specifically implemented through steps 100 to 200:

[0153] Step 100: Input the observed trajectory of the target into the preset trajectory prediction model, process the observed trajectory through the trajectory prediction model, and obtain the predicted trajectory.

[0154] In some embodiments, when determining the observation trajectory of a target, in a scenario with a total duration of T, there are N pedestrians, where the actual position of the ni-th pedestrian at time ti can be recorded as:

[0155] ;

[0156] in, Indicates the pedestrian's ID number; Indicates the time step index; and They represent the first A pedestrian at a moment The true x and y coordinates.

[0157] In the length of observation time Inside, the system collects the location data of all pedestrians, thus forming a collection of historical observation trajectories. .in, Indicates the first The historical trajectory of a pedestrian within the observation time period, and This indicates the maximum length of the observation period, i.e., the number of time steps contained in the observation trajectory.

[0158] The goal of this task is to use the observed trajectories to predict the future time period after the observation period ends. Set of motion trajectories within , of which The predicted trajectory of an individual pedestrian is represented as follows:

[0159] ;

[0160] in, Indicates the first A pedestrian at a moment The predicted location coordinates, This indicates the maximum length of the predicted time range, i.e., the number of time steps corresponding to the future trajectory.

[0161] Therefore, this task can be further represented as learning a mapping function f such that:

[0162]

[0163] in, This represents the set of parameters that need to be learned in the model.

[0164] Specifically, in order to improve the effectiveness and generalization ability of this application, the training data, validation data, and test data of the model are three publicly available datasets widely used in the field of pedestrian trajectory prediction:

[0165] 1. ETH, from S. Pellegrini, A. Ess, K. Schindler, et al., "You'll neverwalk alone: ​​Modeling social behavior for multi-target tracking," in Proc.IEEE 12th Int. Conf. Comput. Vis., 2009, pp. 261–268.

[0166] 2. UCY, from A. Lerner, Y. Chrysanthou, and D. Lischinski, “Crowds by example,” Comput. Graph. Forum, vol. 26, no. 3, pp. 655–664, 2007.

[0167] 3. SDD (Stanford Drone Dataset), from A. Robicquet, A. Sadeghian, A. Alahi, et al., "Learning social etiquette: Human trajectory understanding incrowded scenes," in Proc. Eur. Conf. Comput. Vis., Cham: Springer, 2016, pp.549–565.

[0168] These datasets cover different scene densities, viewpoints, and pedestrian interaction complexities, and can comprehensively reflect the model's performance in real-world environments.

[0169] The ETH and UCY datasets are benchmark datasets in the field of pedestrian trajectory prediction. They are collected from a fixed camera perspective with a sampling frequency of 2.5 Hz and contain five typical pedestrian scenes: Eth, Hotel, Univ, Zara1, and Zara2. During training, the domain-standard Leave-One-Scene-Out partitioning strategy (from A. Mohamed, K. Qian, M. Elhoseiny, et al., “Social-stgcnn: A social spatio-temporal graphconvolutional neural network for human trajectory prediction,” in Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 14424–14432) is employed. This means that one scene is used sequentially as the test set, and the remaining scenes are used as the training set to ensure fairness and comparability in the evaluation.

[0170] The SDD dataset is a top-down view of pedestrian trajectories captured by drones. Its sampling frequency is 2.5Hz, and it includes multiple complex scenes, covering the motion trajectories of various target categories within a campus setting. This application follows the same setup as the Social-Implicit dataset (from A. Mohamed, D. Zhu, W. Vu, et al., “Social-implicit: Rethinking trajectory prediction evaluation and the effectiveness of implicit maximum likelihood estimation,” in Proc. Eur. Conf. Comput. Vis., Cham:Springer, 2022, pp. 463–479.), dividing the dataset into training, validation, and test sets in a 7:1:2 ratio for model training, validation, and testing, respectively.

[0171] In practical use of this model for trajectory prediction, the visual depth map captured by the recorder on the AR smart helmet is first converted into a bird's-eye view, and then input into the model of this application for processing. For the specific conversion process, please refer to (BEVFusion: A Simple and Robust LiDAR Camera Fusion Framework, NeurIPS 2022).

[0172] In some embodiments, please refer to Figure 2 and Figure 10 , Figure 2 This is a flowchart illustrating the steps of obtaining a predicted trajectory by processing the observed trajectory through a trajectory prediction model in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. Figure 10 This is an algorithm structure diagram of the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment. The method for obtaining the predicted trajectory in this application embodiment is specifically implemented through steps 110 to 160:

[0173] Step 110: After normalizing the observation trajectory, map it to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observation trajectory.

[0174] Understandably, in real-world scenarios, pedestrians typically exhibit continuous and smooth displacement changes during movement, with fewer irregular and drastic abrupt changes. Therefore, from a signal representation perspective, pedestrian trajectories can be considered as a type of continuously evolving time-series signal, with its overall motion trend usually characterized primarily by low-frequency components, while local disturbances and short-term fluctuations are more represented by high-frequency components. Based on this observation, this application introduces Discrete Cosine Transform (DCT) (from N. Ahmed, T. Natarajan, and KR Rao, “Discrete cosine transform,” IEEE Trans. Comput., vol. 100, no. 1, pp. 90–93, 1974.) to orthogonally represent the observed trajectory. Compared to directly regressing discrete position sequences in the time domain, DCT can map the trajectory to mutually orthogonal frequency domain base spaces, reducing redundancy and correlation in the original time-series signal, enabling the model to model future motion patterns based on the overall trajectory morphology. In this model, low-frequency components primarily characterize the global motion trend of the trajectory, while high-frequency components reflect local detail changes. Based on this frequency domain decomposition method, the model can more effectively extract and utilize the continuous and smooth evolution patterns inherent in the trajectory, thereby enhancing its ability to represent motion inertia, direction changes, and overall structural information. Furthermore, since the DCT basis functions have global effects, changes in individual frequency domain coefficients collectively affect the entire trajectory, rather than just corresponding to position changes at a specific moment. Therefore, this representation method is more conducive to maintaining the overall consistency of the trajectory in the time dimension. Based on this, this application uses DCT to map the original observed trajectory to the frequency domain space for modeling and recovers the final predicted trajectory through inverse transformation.

[0175] In some embodiments, please refer to Figure 3 and Figure 11 , Figure 3This is a flowchart illustrating the steps for obtaining the frequency domain coefficients of the observed trajectory in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. Figure 11 This is a schematic diagram of the DCT (Discrete Cosine Transform) principle in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment. The method for obtaining the frequency domain coefficients of the observed trajectory in this application is specifically implemented through steps 111 to 113:

[0176] Step 111: Perform DCT encoding based on the target's position at each time step in the observed trajectory to obtain orthogonal DCT basis functions.

[0177] In some embodiments, the observed trajectory is mapped to the frequency domain via DCT encoding. Its continuous and smooth characteristics concentrate the main information in the low frequency range, which is then unfolded into one-dimensional components in chronological order. By retaining a small number of low-frequency coefficients, a low-dimensional and compact representation of the overall trend of the trajectory can be achieved, thereby effectively modeling its continuity and global trend. Let the... The two-dimensional trajectory of a pedestrian during the observation phase is , can be obtained Based on this, construct a structure with a length of One-dimensional orthogonal DCT basis functions The characterization formulas for orthogonal DCT basis functions include:

[0178] ;

[0179] in, These are orthogonal DCT basis functions; , This represents the maximum length of the observation time. For frequency index, used to characterize the first One DCT basis function; This refers to the number of DCT basis functions, i.e., the dimension of the frequency domain representation. Considering that the trajectory is mainly dominated by low-frequency components, usually only the first few are retained. A low-frequency component is used to achieve dimensionality reduction and smooth representation.

[0180] Step 112: Extend the orthogonal DCT basis functions to the trajectory space based on the Kronecker product to obtain the first orthogonal basis matrix.

[0181] In some embodiments, to adapt to the two-dimensional coordinate form, the time base is extended to the trajectory space using the Kronecker product to obtain the first orthogonal basis matrix for the observation phase. The characterization formulas for the first orthogonal basis matrix include:

[0182] ;

[0183] in, It is the first orthogonal basis matrix; It is a two-dimensional identity matrix;

[0184] Step 113: Obtain the frequency domain coefficients of the observation trajectory based on the first orthogonal basis matrix and the observation trajectory.

[0185] In some embodiments, the formula for representing the frequency domain coefficients of the observation trajectory includes:

[0186] ;

[0187] in, These are the frequency domain coefficients of the observed trajectory; The first orthogonal basis matrix transpose; For the goal The observation trajectory.

[0188] Understandably, in complex scenarios, pedestrian movement is not only influenced by their own historical trajectories but also has significant interactions with the behavior of surrounding pedestrians. For example, behaviors such as avoidance, following, or group movement can all alter the trajectory evolution trend. If predictions are made independently based solely on individual trajectories, the motion constraints brought about by group interactions are easily overlooked, leading to unreasonable fluctuations in the predicted trajectory. Therefore, introducing spatiotemporal interaction modeling can characterize the spatial relationships between pedestrians while modeling individual movements, and improve the rationality of the predicted trajectory by constraining the temporal evolution process of the trajectory through joint modeling. After obtaining the frequency domain representation of the trajectory, this invention introduces a graph-based spatiotemporal modeling module in the orthogonal coefficient space to jointly model the trajectories of multiple pedestrians.

[0189] Step 120: Obtain the target interaction graph by using the target as nodes and the interaction relationships between targets as edges.

[0190] In some embodiments, it is assumed that there are a total of Each pedestrian is considered a node in the graph, thus constructing a pedestrian interaction graph. , where the set of nodes Represents all pedestrians in the scene, and the edge set. This represents the potential interactions between pedestrians. The topological structure of this graph can be represented by the adjacency matrix. It indicates that the elements Used to describe pedestrians with pedestrians The strength of the connection between them.

[0191] Step 130: Using the frequency domain coefficients of the observed trajectory as the features of the nodes, and combining them with the target interaction graph, perform spatiotemporal interaction processing between targets to obtain preliminary prediction coefficients.

[0192] In some embodiments, please refer to Figure 4 , Figure 4 This is a flowchart illustrating the steps for obtaining preliminary prediction coefficients in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. Specifically, the method for obtaining preliminary prediction coefficients in this embodiment is implemented through steps 131 to 132:

[0193] Step 131: Using the frequency domain coefficients of the observed trajectory as node features, perform spatial convolution on the target interaction graph through a spatiotemporal convolutional network, and combine it with convolution operations in the time dimension to model spatial interaction and obtain spatial interaction output.

[0194] In some embodiments, this application employs a Spatio-Temporal Graph Convolutional Neural Network (ST-GCNN) (from A. Mohamed, K. Qian, M. Elhoseiny, et al., “Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in Proc. IEEE / CVF Conf. Compute.Vis. Pattern Recognition., 2020, pp. 14424–14432.) to model trajectory features. ST-GCNN achieves joint modeling of "spatial interaction + temporal evolution" by performing spatial convolution on the graph structure and combining it with convolution operations in the temporal dimension, obtaining spatial interaction output. Through multi-layer stacking, the model can effectively capture the interaction patterns between pedestrians and the trajectory change trends. The representation formula of the spatial interaction output includes:

[0195] ;

[0196] in, For spatial interactive output, the first Layer feature representation; For spatial interactive output, the first Layer feature representation; Adjacency matrices under different spatial relationships are used to characterize the topological structure of the target interaction graph; These are learnable parameters; It is a non-linear activation function.

[0197] Understandably, the frequency domain coefficients of each pedestrian's observed trajectory in the DCT space are used as node features of the target interaction graph. Let the frequency domain coefficients of the observed trajectory be... , of which List pedestrian The trajectory coefficient vector. Therefore, the graph structure... With node features Together they constitute the input for spatiotemporal interaction modeling: where, Depicting the spatial relationships between pedestrians, This represents the motion state and historical trajectory characteristics of each pedestrian. Based on this representation, the model can jointly model the trajectories of multiple pedestrians under graph structure constraints, thereby simultaneously capturing individual motion information and the influence of group interactions.

[0198] Step 132: Perform multi-layer one-dimensional convolution operation on the spatial interaction output through a temporal prediction network to obtain preliminary prediction coefficients.

[0199] In some embodiments, after completing spatial interaction modeling, to further enhance the modeling capability in the temporal dimension, this application introduces a convolution-based temporal prediction network (Time-Extrapolator CNN, TXP-CNN) (also from A. Mohamed, K. Qian, M. Elhoseiny, et al., “Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in Proc. IEEE / CVF Conf. Compute. Vis. Pattern Recognition., 2020, pp. 14424–14432.) to model and map the features output by ST-GCNN in the temporal dimension. TXP-CNN transforms the time series layer by layer through multiple one-dimensional convolutional operations, gradually mapping the features from the observation stage to the prediction stage, thereby generating preliminary prediction coefficients of the future trajectory in the frequency domain. .

[0200] Step 140: Obtain the trajectory pattern representation based on the preset future trajectory pattern prototype and the frequency domain coefficients of the observed trajectory.

[0201] Understandably, in multimodal trajectory prediction tasks, observed trajectories from the same historical period often correspond to multiple possible future evolution paths. However, although the aforementioned methods based on low-dimensional frequency domain representations have reduced modeling complexity and enhanced trajectory continuity and smoothness to some extent, the uncertainty of multimodal behavior still objectively exists. Specifically, in the frequency domain coefficient space, the representations corresponding to different future trajectories still exhibit obvious multi-distribution characteristics. Therefore, it is still necessary to further introduce a structured multimodal modeling mechanism in the low-dimensional representation space to explicitly characterize the distribution characteristics and differences between different potential motion modes. Intuitively, although future trajectories are diverse, a large number of real future trajectories can often be summarized into several representative trajectory pattern prototypes (such as straight-line, turning, or stationary). If these typical patterns can be extracted from the data and used as the basic structure for prediction, the complex multimodal generation problem can be transformed into a problem of selecting and combining finite patterns, thereby reducing the learning difficulty while ensuring expressive power. Based on the above considerations, this application constructs a trajectory pattern library of future trajectory pattern prototypes in the DCT coefficient space, and its construction diagram is shown in the figure. Figure 12 As shown, by clustering future trajectories in the training data, a set of representative trajectory pattern prototypes is extracted, providing a structured reference for subsequent multimodal combination and trajectory refinement.

[0202] In some embodiments, please refer to Figure 5 and Figure 12 , Figure 5 This is a flowchart illustrating the steps of obtaining a trajectory pattern prototype in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. Figure 12 This diagram illustrates the acquisition of trajectory pattern prototypes and the construction of a trajectory pattern library in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. The method for acquiring trajectory pattern prototypes in this embodiment is specifically implemented through steps a to b:

[0203] Step a: During the model training phase of the target trajectory model, the future actual trajectory of the target, which serves as the model input sample, is mapped to the frequency domain space based on the DCT orthogonal basis to obtain the frequency domain coefficients of the future actual trajectory.

[0204] In some embodiments, a set of future actual trajectories in the given training data ,in , The number of targets in the sample can be expanded into a one-dimensional vector in chronological order. First, all future actual trajectories undergo preprocessing consistent with the observation phase, including dynamic / static classification and trajectory normalization, resulting in subsets of moving and stationary trajectories. Then, using a DCT orthogonal basis, the future trajectories are mapped to the frequency domain, yielding the corresponding targets as samples. Future actual trajectory frequency domain coefficient :

[0205] ;

[0206] in, Let be the orthogonal basis matrix of the future actual trajectory of the target as a sample. The transpose of .

[0207] Therefore, the trajectory samples of each subset can form a set of frequency domain coefficients for future trajectories. :

[0208] ;

[0209] in, Indicates the number of samples in the current subset.

[0210] Step b: Perform K-meas clustering on the frequency domain coefficients of the future actual trajectory and obtain the trajectory pattern prototype with the cluster center as the future trajectory.

[0211] In some embodiments, K-means clustering is performed on the frequency domain coefficient sets of the moving and stationary classes respectively (from JB McQueen, “Some methods of classification and analysis of multivariateobservations,” in Proc. 5th Berkeley Symp. Math. Stat. Probab., 1967, pp.281–297.), dividing the targets as samples into... Each cluster, i.e. There are 3 clusters, and each cluster center (cluster center) is used as the corresponding trajectory pattern prototype. Let the 1st cluster be 1. The sample set corresponding to each cluster is: The formulas representing the trajectory pattern prototype of the future trajectory include:

[0212] ;

[0213] in, For the first The cluster centers of the i-th cluster are used to characterize the i-th cluster. A trajectory pattern prototype; The number of clusters is used to characterize the number of trajectory pattern prototypes; for A set of samples for each cluster; For the target of the sample The frequency domain coefficients of the future actual trajectory.

[0214] Furthermore, a future trajectory pattern library can be obtained. :

[0215] ;

[0216] Understandably, the future trajectory pattern library is built offline in one go before model training begins and remains unchanged during subsequent training and inference. This design avoids introducing additional unstable factors during training and allows the model to learn in a stable pattern space, thereby improving training efficiency and prediction stability.

[0217] Understandably, after completing the construction of the future trajectory pattern library, although the model has obtained a set of structured representations that can cover multiple trajectory pattern prototypes, how to effectively utilize these prototypes in the specific prediction process remains an important issue for achieving high-quality multimodal prediction. Specifically, the future trajectory pattern library essentially provides a set of discrete pattern primitives, with each prototype corresponding to only one typical motion trend. However, in real-world scenarios, pedestrians' future behavior often exhibits greater continuity and complexity, and their motion patterns may not strictly fall on a single fixed prototype, but are more likely to manifest as transitions or combinations between multiple typical patterns. If only a "single prototype selection" approach is used for prediction, it can easily lead to limited model expressive power and difficulty in characterizing fine-grained behavioral changes in complex scenarios. Therefore, it is necessary to introduce a learnable weight allocation mechanism based on existing future trajectory patterns, enabling the model to adaptively weight and combine multiple prototypes according to the observed trajectory, thereby achieving flexible expression of complex motion patterns while maintaining the advantages of structured representation. Based on the above considerations, this application further designs a weight allocation mechanism for the future trajectory pattern library, the principle of which is shown in the figure below. Figure 13 As shown, this provides a more refined and efficient modeling approach for multimodal prediction.

[0218] In some embodiments, please refer to Figure 6 and Figure 13 , Figure 6 This is a flowchart illustrating the steps of obtaining a trajectory pattern representation in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. Figure 13This is a schematic diagram illustrating the principle of the future trajectory pattern library weight allocation mechanism in the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in this application embodiment. The method for obtaining the step flow representation in this application embodiment is specifically implemented through steps 141 to 143:

[0219] Step 141: The pattern weight generator based on the multilayer sensing mechanism processes the frequency domain coefficients of the observed trajectory to obtain the combined weight of the trajectory patterns.

[0220] In some embodiments, let the first The coefficients of the observed trajectory of a pedestrian in the DCT space are represented as follows: Using this as input, a lightweight nonlinear mapping function is used to generate the corresponding trajectory pattern combination weights. The formulas for representing the weights of trajectory pattern combinations include:

[0221] ;

[0222] in, It is a pattern weight generator composed of a multilayer perceptron (MLP); Weights are combined for trajectory patterns. The Line number Column elements are denoted as , for the first The first goal in For the candidate predicted trajectory, the first The response strength of a trajectory pattern prototype; For the goal The frequency domain coefficients of the observed trajectory.

[0223] Step 142: Based on the trajectory pattern prototype, perform dimensional normalization processing on the trajectory pattern combination to obtain the normalized weights.

[0224] In some embodiments, the representation formula for the normalized weights includes:

[0225] ;

[0226] in, For the goal In the For the candidate predicted trajectory, the first Normalized weights for each trajectory pattern prototype; The summation index is used to iterate through all trajectory pattern prototypes; This represents the number of trajectory pattern prototypes.

[0227] Step 143: Perform a weighted combination of the normalized weights and the trajectory pattern prototype to obtain the trajectory pattern representation.

[0228] In some embodiments, the representation formula for trajectory pattern representation includes:

[0229] ;

[0230]

[0231] in, Represented as a trajectory pattern; This is the sequence number of the trajectory pattern prototype; The number of trajectory pattern prototypes; It is a collection that includes trajectory pattern prototypes, i.e., a future trajectory pattern library; For the first A trajectory pattern prototype. .

[0232] Step 150: Obtain the corrected prediction coefficients based on the preliminary prediction coefficients and trajectory pattern representation.

[0233] In some embodiments, the prediction coefficient is corrected. The characterization formulas include:

[0234] .

[0235] Step 160: After DCT decoding of the corrected prediction coefficients, perform inverse normalization to obtain the predicted trajectory.

[0236] Understandably, after completing multimodal prediction in the frequency domain, the model output is still represented in the form of low-dimensional frequency domain coefficients, which cannot be directly used for trajectory evaluation and downstream decision-making tasks in real-world scenarios. Therefore, it is necessary to map the prediction results back from the frequency domain to Euclidean coordinate space and to unify and integrate the prediction results of different branches to obtain a trajectory representation with practical physical meaning. Therefore, this application designs a trajectory reconstruction and result fusion module, which recovers the trajectory coordinates through inverse DCT transformation and performs consistent fusion with the prediction results of the dynamic and static branches to obtain the final multimodal trajectory output.

[0237] In some embodiments, please refer to Figure 7 , Figure 7 This is a flowchart illustrating the steps of obtaining a predicted trajectory in an assisted driving method for predicting object trajectories on a motorcycle smart AR helmet, as provided in an embodiment of this application. The method for obtaining the predicted trajectory in this embodiment is specifically implemented through steps 161 to 163:

[0238] Step 161: Perform DCT decoding on the corrected prediction coefficients to obtain the second orthogonal basis matrix.

[0239] Step 162: Map the second orthogonal basis matrix back to Euclidean space to obtain the Euclidean space representation.

[0240] In some embodiments, the representation formula for Euclidean space includes:

[0241]

[0242] in, The Euclidean space representation of a one-dimensional vector; It is the second orthogonal basis matrix; To correct the prediction coefficients.

[0243] Step 163: Reconstruct the Euclidean space representation into a two-dimensional time series and perform normalization operations to obtain the predicted trajectory.

[0244] In some embodiments, since the trajectory is normalized during the preprocessing stage (including translation, rotation, and scaling), it is necessary to perform a corresponding inverse transformation on the prediction result to restore it to the true coordinate space. Specifically, let the normalization parameters be the translation vectors. Rotation matrix and scaling factor The formulas for representing the predicted trajectories obtained from the moving branch and the stationary branch include:

[0245] ;

[0246] in, For the goal The predicted trajectory As a branch of motion, It is a stationary branch; Scale factor; For the goal A two-dimensional time series is used to characterize the predicted trajectory of the operation to be normalized; It is a rotation matrix; It is a translation vector.

[0247] In some embodiments of this application, the predicted trajectory of a stationary target is obtained. Predicted trajectory of moving targets Then, based on the fusion of dynamic and static masking, the predicted trajectories of moving targets and stationary targets are used to obtain a comprehensive predicted trajectory.

[0248] Specifically, set Indicates the first The movement and stillness markers of each pedestrian will ultimately output a comprehensive predicted trajectory. for:

[0249] ;

[0250] in .

[0251] Step 200: Driving assistance based on predicted trajectory.

[0252] Understandably, once the predicted pedestrian trajectory is obtained, this application can observe the possible interactions of that trajectory at the current speed of the motorcycle and provide this information to the driver in an AR manner, issuing a hazard warning. It also sends information about particularly dangerous situations, such as roads blocked by static objects, to other motorcyclists via a remote communication module. Thus, by implementing this invention, users can obtain a system suitable for low-speed conditions such as rural roads and residential areas, where there is a high risk of traffic accidents due to mixed pedestrian traffic. This system, based on a smart AR helmet, analyzes the trajectory of external pedestrians, provides risk warnings, and alerts drivers using the smart AR helmet.

[0253] Understandably, the loss function is used during the model training phase to measure the difference between the predicted result and the true trajectory, and to optimize the model parameters through backpropagation. During training, the model generates multiple candidate future trajectories based on the input observed trajectory, compares them with the true future trajectory, calculates the error using the loss function, and updates the model parameters. In the inference phase, the model directly outputs the predicted result, without further loss calculation. Structurally, the loss function corresponds to... Figure 10 The supervisory signal between the final predicted trajectory and the actual future trajectory plays a crucial role throughout the model training process, influencing the parameter learning of each functional module in the diagram, but it is not explicitly represented as an independent module in the diagram. To achieve accurate prediction of future trajectories, this application supervises the model output in both the low-dimensional frequency domain coefficient space and the Euclidean coordinate space. Considering that the model of this invention will output for each pedestrian... There are 3 candidate future trajectories, therefore the best-of-K strategy is used in both the training and evaluation phases:

[0254] In other words, among all candidate trajectories, only the one with the smallest error to the true trajectory is selected for loss calculation or performance evaluation. This strategy is consistent with existing mainstream methods (from A. Mohamed, K. Qian, M. Elhoseiny, et al., “Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction,” in Proc. IEEE / CVF Conf. Comput.Vis. Pattern Recognit., 2020, pp. 14424–14432.), which can effectively reflect the model's ability to cover multiple potential future trajectories and encourage the model to learn multimodal motion patterns.

[0255] In some embodiments, the formula for representing the total loss function includes:

[0256] ;

[0257] in, This is the total loss function; The loss of frequency domain coefficients directly constrains the model's representation accuracy in the low-dimensional frequency domain space, which helps to enhance the model's ability to model the overall motion trend of the trajectory. The Euclidean space average displacement loss is used to constrain the overall fitting accuracy of the entire predicted trajectory in spatial position. The Euclidean space endpoint displacement loss is used to constrain the accuracy of predicting the future endpoint position, which helps improve the model's ability to characterize long-term motion trends.

[0258] ;

[0259] in, The number of targets; The sort number for the target; Candidate predicted trajectories; The total number of candidate predicted trajectories; The model predicts the first The first goal in The coefficient representation of each candidate predicted trajectory in the frequency domain; For the first The coefficients representing the true future trajectory of an object in the frequency domain.

[0260] ;

[0261] in, The sort number for the target; The maximum length of the future forecast; For a specific moment; For the goal The Candidate predicted trajectories at future moments The predicted location; For the goal The future moment The actual location;

[0262] ;

[0263] in, For the goal The Candidate predicted trajectories at future moments The predicted location, where ; For the goal In the future The true location, among which .

[0264] Understandably, the assisted driving method for predicting object trajectories on a motorcycle smart AR helmet provided in the embodiments of this application includes inputting the observed trajectory of the target into a preset trajectory prediction model, and processing the observed trajectory through the trajectory prediction model, including: normalizing the observed trajectory and mapping it to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observed trajectory; obtaining a target interaction graph with the target as nodes and the interaction relationship between targets as edges; using the frequency domain coefficients of the observed trajectory as features of the nodes and combining them with the target interaction graph to perform spatiotemporal interaction processing between targets to obtain preliminary prediction coefficients; obtaining a trajectory pattern representation based on a preset trajectory pattern prototype of the future trajectory and the frequency domain coefficients of the observed trajectory; obtaining corrected prediction coefficients based on the preliminary prediction coefficients and the trajectory pattern representation; performing DCT decoding on the corrected prediction coefficients and then inverse normalizing them to obtain the predicted trajectory; and providing driving assistance based on the predicted trajectory.

[0265] This application has provided a detailed description of an assisted driving method and system for predicting object trajectories on a motorcycle smart AR helmet, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for assisting driving by predicting object trajectories on a motorcycle smart AR helmet, characterized in that, include: The observed trajectory of the target is input into a preset trajectory prediction model, and the observed trajectory is processed by the trajectory prediction model to obtain a predicted trajectory, including: After normalizing the observation trajectory, it is mapped to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observation trajectory; Using the target as nodes and the interaction relationships between the targets as edges, obtain the target interaction graph; Using the frequency domain coefficients of the observed trajectory as the features of the node, and combining them with the target interaction graph, spatiotemporal interaction processing is performed between the targets to obtain preliminary prediction coefficients; Based on the preset future trajectory pattern prototype and the frequency domain coefficients of the observed trajectory, a trajectory pattern representation is obtained; Based on the preliminary prediction coefficients and the trajectory pattern representation, corrected prediction coefficients are obtained; After DCT decoding of the modified prediction coefficients, inverse normalization is performed to obtain the predicted trajectory; Driving assistance is provided based on the predicted trajectory.

2. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, The steps for obtaining the frequency domain coefficients of the observation trajectory include: Based on the position of the target at each time step in the observed trajectory, DCT encoding is performed to obtain orthogonal DCT basis functions. The characterization formulas of the orthogonal DCT basis functions include: ； in, The orthogonal DCT basis functions are... , This represents the maximum length of the observation time. For frequency index, used to characterize the first One DCT basis function; The number of DCT basis functions; The orthogonal DCT basis functions are extended to the trajectory space based on the Kronecker product to obtain the first orthogonal basis matrix. The representation formula of the first orthogonal basis matrix includes: ； in, This is the first orthogonal basis matrix; It is a two-dimensional identity matrix; The frequency domain coefficients of the observation trajectory are obtained based on the first orthogonal basis matrix and the observation trajectory. The characterization formula of the frequency domain coefficients of the observation trajectory includes: ； in, The frequency domain coefficients of the observed trajectory; The first orthogonal basis matrix transpose; For the target The observation trajectory.

3. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, The step of using the frequency domain coefficients of the observed trajectory as features of the nodes, and combining them with the target interaction graph to perform spatiotemporal interaction processing between the targets to obtain preliminary prediction coefficients includes: Using the frequency domain coefficients of the observed trajectory as node features, a spatiotemporal convolutional network is used to perform spatial convolution on the target interaction graph, and spatial interaction modeling is performed by combining convolution operations in the time dimension to obtain the spatial interaction output. The representation formula of the spatial interaction output includes: ； in, For the spatial interaction output of the first Layer feature representation; For the spatial interaction output of the first Layer feature representation; The adjacency matrix under different spatial relationships is used to characterize the topological structure of the target interaction graph; These are learnable parameters; It is a non-linear activation function; The spatial interaction output is subjected to multi-layer one-dimensional convolution operations by a temporal prediction network to obtain the preliminary prediction coefficients.

4. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, The steps of presetting the trajectory pattern prototype of the future trajectory include: During the model training phase of the target trajectory model, the future actual trajectory of the target, which serves as the model input sample, is mapped to the frequency domain space based on the DCT orthogonal basis to obtain the frequency domain coefficients of the future actual trajectory. K-meas clustering is performed on the frequency domain coefficients of the future actual trajectory, and trajectory pattern prototypes with cluster centers of the future trajectory are obtained. The characterization formula of the trajectory pattern prototype includes: ； in, For the first The cluster centers of the i-th cluster are used to characterize the i-th cluster. The aforementioned trajectory pattern prototype; The number of clusters is used to characterize the number of trajectory pattern prototypes; for A set of samples for each cluster; For the target of the sample The frequency domain coefficients of the future actual trajectory.

5. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, The step of obtaining the trajectory pattern representation based on the preset future trajectory pattern prototype and the frequency domain coefficients of the observed trajectory includes: A pattern weight generator based on a multilayer sensing mechanism processes the frequency domain coefficients of the observed trajectory to obtain the trajectory pattern combination weight. The representation formula of the trajectory pattern combination weight includes: ； in, For the pattern weight generator; The trajectory pattern is combined with weights. The Line number Column elements are denoted as , for the first The aforementioned objective is in the first For the candidate predicted trajectory, the first The response strength of a trajectory pattern prototype; For the target The frequency domain coefficients of the observed trajectory; Based on the dimensional normalization of the trajectory pattern combination according to the trajectory pattern prototype, normalized weights are obtained. The representation formula of the normalized weights includes: ； in, For the target In the For the candidate predicted trajectory, the first Normalized weights for each trajectory pattern prototype; The summation index is used to iterate through all trajectory pattern prototypes; The number of the trajectory pattern prototypes; The normalized weights and the trajectory pattern prototypes are weighted and combined to obtain a trajectory pattern representation, the representation formula of which includes: ； ； in, This is a representation of the trajectory pattern; The sequence number of the trajectory pattern prototype; The number of the trajectory pattern prototypes; A set including the trajectory pattern prototypes; For the first The aforementioned trajectory pattern prototype. .

6. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, The step of performing DCT decoding on the corrected prediction coefficients and then inverse normalizing them to obtain the predicted trajectory includes: DCT decoding is performed on the corrected prediction coefficients to obtain the second orthogonal basis matrix; Mapping the second orthogonal basis matrix back to Euclidean space yields the Euclidean space representation, the representation formula of which includes: ； in, The Euclidean space representation of a one-dimensional vector; This is the second orthogonal basis matrix; The corrected prediction coefficients; The Euclidean space representation is reconstructed into a two-dimensional time series and normalized to obtain the predicted trajectory. The formula representing the predicted trajectory includes: ； in, For the target The predicted trajectory; Scale factor; For the target A two-dimensional time series is used to characterize the predicted trajectory of the operation to be normalized; It is a rotation matrix; It is a translation vector.

7. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, During the model training phase of the pre-set trajectory prediction model, the model generates multiple candidate predicted trajectories of the target as samples, compares them with the future actual trajectories of the target as samples, calculates the error using a total loss function, and updates the model parameters. The formula for the total loss function includes: ； in, The total loss function is... For frequency domain coefficient loss; This represents the average displacement loss in Euclidean space. This represents the displacement loss at the endpoint of Euclidean space. ； in, The number of targets; The sort number for the target; Candidate predicted trajectories; The total number of candidate predicted trajectories; For the first The first goal in The coefficient representation of each candidate predicted trajectory in the frequency domain; For the first The coefficient representation of the true future trajectory of an objective in the frequency domain; ； in, The sort number for the target; The maximum length of the future prediction time; For a specific moment; For the goal The Candidate predicted trajectories at future moments The predicted location; For the goal The future moment The actual location; ； in, For the goal The Candidate predicted trajectories at future moments The predicted location, where ; For the goal In the future The true location, among which .

8. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 1, characterized in that, The method further includes: Based on the target's speed of motion, the target is classified into a moving target or a stationary target; The observed trajectory information of the moving target is input into a preset trajectory prediction model of the moving target to obtain the predicted trajectory of the moving target; The observed trajectory information of the stationary target is input into a preset trajectory prediction model for the stationary target to obtain the predicted trajectory of the stationary target; The predicted trajectories of the moving target and the stationary target are fused to obtain a comprehensive predicted trajectory.

9. The assisted driving method for predicting object trajectories on a motorcycle smart AR helmet according to claim 8, characterized in that, The step of classifying the target into moving targets and stationary targets based on the target's speed includes: The average velocity of the target at the end of the observation period is obtained, and the formula for representing the average velocity includes: ； in, For the target The average speed; For the target At any moment The location, among which , This represents the maximum length of the observation time. For the target At any moment The location, among which ; The modulus of the average velocity is obtained, and a motion / static mask is obtained by combining it with a preset modulus threshold. The representation formula of the motion / static mask includes: ； in, For the target The motion mask; The average speed The modulus length; The modulus threshold; Based on the motion mask, the target is classified into the moving target or the stationary target; The step of fusing the predicted trajectories of the moving target and the stationary target to obtain a comprehensive predicted trajectory includes: Based on the dynamic and static mask, the predicted trajectory of the moving target and the predicted trajectory of the stationary target are fused to obtain a comprehensive predicted trajectory.

10. A driver assistance system for predicting object trajectories on a motorcycle smart AR helmet, characterized in that, include: A trajectory prediction module (10) is used to input the observed trajectory of the target into a preset trajectory prediction model, process the observed trajectory through the trajectory prediction model, and obtain a predicted trajectory. The trajectory prediction module (10) includes: Frequency domain coefficient unit (11) is used to normalize the observation trajectory and then map it to the frequency domain space through DCT encoding to obtain the frequency domain coefficients of the observation trajectory; Graph construction unit (12), the graph construction unit (12) is used to obtain a target interaction graph with the target as the node and the interaction relationship between the targets as the edge; Spatiotemporal interaction unit (13), the spatiotemporal interaction unit (13) is used to use the frequency domain coefficients of the observation trajectory as the features of the node, and combine the target interaction graph to perform spatiotemporal interaction processing between the targets to obtain preliminary prediction coefficients; The trajectory pattern unit (14) is used to obtain a trajectory pattern representation based on a preset trajectory pattern prototype of the future trajectory and the frequency domain coefficients of the observed trajectory; A prediction correction unit (15) is used to obtain corrected prediction coefficients based on the preliminary prediction coefficients and the trajectory pattern representation; The prediction output unit (16) is used to perform DCT decoding on the modified prediction coefficients and then perform inverse normalization to obtain the prediction trajectory. The driver assistance module (20) is used to provide driving assistance based on the predicted trajectory.

Images