Integrated motorcycle auxiliary driving system and method

Abstract

The invention relates to the technical field of vehicle safety, and discloses an integrated motorcycle auxiliary driving system and method. The method comprises the following steps: synchronously acquiring vehicle kinematics parameters and environment three-dimensional point cloud data through a multi-source sensing system of a motorcycle; executing a dynamic risk mapping operation by using the collected data to generate a risk probability distribution diagram; acquiring a color image and a depth image of a road scene through a stereoscopic vision camera, inputting the color image and the depth image into the multi-scale feature extraction network for analysis in combination with the risk probability distribution map, and outputting a comprehensive risk score; performing obstacle trajectory prediction according to the comprehensive risk score to obtain an obstacle prediction trajectory, scanning a target area by using an infrared sensor, processing point cloud data by using a point cloud segmentation algorithm based on the prediction trajectory, and extracting actual obstacle attributes; and inputting the comprehensive risk score and the actual obstacle attribute into a fuzzy logic controller for data fusion, and finally generating an integrated motorcycle aided driving control instruction.

Description

Technical Field

[0001] This invention relates to the field of vehicle safety technology, specifically to an integrated motorcycle driver assistance system and method. Background Technology

[0002] Current motorcycle driver assistance systems primarily employ a single sensor and independent decision-making module. Existing technologies rely heavily on single-type sensors for environmental perception, failing to effectively integrate visual, radar, and positioning data. Risk assessment methods are simplistic, typically based on fixed thresholds or simple rules, unable to dynamically evaluate potential hazards in complex road environments. Data processing is isolated, failing to coordinate the analysis of vehicle motion and environmental obstacle information. Visual recognition algorithms are static and fixed, unable to adjust analysis focus based on real-time risk distribution. Trajectory prediction models are linear, struggling to accurately predict the movement intentions of pedestrians, vehicles, and other dynamic obstacles. Control decision-making mechanisms are rigid, lacking the ability to handle uncertainty and fuzzy information. Existing methods need to address key technical challenges such as multi-source data fusion, dynamic risk assessment, adaptive visual analysis, and intelligent decision control.

[0003] Traditional motorcycle driver assistance systems (ADAS) suffer from significant shortcomings in environmental perception and decision-making intelligence. Sensor data acquisition is asynchronous, with timestamp discrepancies and coordinate system inconsistencies among sensor data. Risk mapping methods are simplistic and fail to consider the dynamic coupling between the vehicle's motion state and environmental characteristics. Probability distribution models are idealized, making it difficult to accurately quantify the spatiotemporal distribution characteristics of different risk sources. Feature extraction network structures are fixed, unable to adaptively adjust the granularity and scope of feature extraction based on risk distribution. Trajectory prediction accuracy is low, failing to adequately consider historical obstacle movement patterns and road topology constraints. Attribute extraction methods are coarse, resulting in inaccurate identification of obstacle physical characteristics and behavioral intentions. Control strategy generation is linear, struggling to handle multi-objective optimization and constraint satisfaction problems. System response latency is large, with an excessively long closed-loop cycle from perception to control. Existing technologies necessitate the development of a fully intelligent end-to-end solution encompassing perception fusion and decision-making control. Summary of the Invention

[0004] The purpose of this invention is to provide an integrated motorcycle driver assistance system and method to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides an integrated motorcycle driver assistance method, the method comprising: The vehicle's kinematic parameters and environmental 3D point cloud data are collected synchronously through a multi-source sensing system, which includes an inertial measurement unit, a lidar, and a global positioning system receiver. A dynamic risk mapping operation is performed using the vehicle kinematic parameters and the three-dimensional point cloud data of the environment to generate a risk probability distribution map. Color and depth images of a road scene are captured by a stereo vision camera. Based on the risk probability distribution map, the color and depth images are input into a multi-scale feature extraction network for analysis, and a comprehensive risk score is output. Based on the comprehensive risk score, obstacle trajectory prediction is performed to obtain the predicted obstacle trajectory. The target area is then scanned using an infrared sensor. Based on the predicted obstacle trajectory, a point cloud segmentation algorithm is used to process the point cloud data of the target area to extract the actual obstacle attributes. The comprehensive risk score and actual obstacle attributes are input into a fuzzy logic controller for data fusion to generate integrated motorcycle assisted driving control commands.

[0006] Preferably, the vehicle's kinematic parameters and the environment's three-dimensional point cloud data are simultaneously acquired through the motorcycle's multi-source sensing system, including: The angular velocity and linear acceleration of the motorcycle are monitored using an inertial measurement unit to obtain the vehicle's kinematic parameters; The environment around the motorcycle is scanned by LiDAR to generate 3D point cloud data of the environment. The real-time location coordinates and timestamp of the motorcycle are obtained using a GPS receiver; The vehicle kinematic parameters, environmental 3D point cloud data, real-time position coordinates, and timestamps are time-aligned and coordinate-transformed to form a multi-source data stream in a unified format.

[0007] Preferably, a dynamic risk mapping operation is performed using the vehicle kinematic parameters and the environmental 3D point cloud data to generate a risk probability distribution map, including: Collect historical driving data of the same model of motorcycles under various road conditions, extract the set of kinematic parameters of the sample vehicles, the set of three-dimensional point cloud data of the sample environment and the corresponding risk event labels to form a training dataset; Construct a dynamic risk mapping model based on graph convolutional networks; The dynamic risk mapping model is trained end-to-end using the training dataset, and the model parameters are optimized using the backpropagation algorithm. The real-time collected vehicle kinematic parameters and environmental 3D point cloud data are input into the trained dynamic risk mapping model to calculate the risk probability value of each spatial grid. A two-dimensional grid-like risk probability distribution map is generated based on the risk probability values.

[0008] Preferably, color and depth images of the road scene are captured using a stereo vision camera. Based on the risk probability distribution map, the color and depth images are input into a multi-scale feature extraction network for analysis, outputting a comprehensive risk score, including: Use a stereo vision camera to continuously capture road scenes, simultaneously acquiring color and depth images; Based on the high-risk areas in the risk probability distribution map, determine the region of interest; A multi-scale feature extraction network is constructed, comprising convolutional layers, pooling layers, and fully connected layers, for processing multimodal features of color images and depth images; The color and depth images are cropped to the region of interest and fed into a multi-scale feature extraction network to extract multi-level feature vectors. After fusing multi-level feature vectors, a comprehensive risk score is calculated through a regression layer.

[0009] Preferably, a multi-scale feature extraction network is constructed, which includes convolutional layers, pooling layers, and fully connected layers, for processing multimodal features of color and depth images, including: Design a dual-branch network structure, with one branch processing color images and the other processing depth images. Each branch contains multiple convolutional layers and pooling layers. Use convolutional kernels of different sizes in convolutional layers to capture multi-scale features; The feature maps of the two branches are concatenated at a specific layer, and then dimensionality reduction and fusion are performed through a fully connected layer; Enhance the nonlinearity of the network by using batch normalization layers and activation functions; During the training phase, the mean squared error loss function and the adaptive moment estimation algorithm are used for optimization.

[0010] Preferably, based on the comprehensive risk score, obstacle trajectory prediction is performed to obtain the predicted obstacle trajectory, including: Collect obstacle movement sequences and corresponding comprehensive risk score sequences from historical motorcycle driving data to form a sample sequence dataset; Construct a trajectory prediction model based on a long short-term memory network; The trajectory prediction model is trained using a sample sequence dataset, and sequence dependencies are learned through backpropagation over time steps. The real-time comprehensive risk score sequence is input into the trained trajectory prediction model, which outputs the obstacle prediction trajectory at the future time step.

[0011] Preferably, the target area is scanned by an infrared sensor, and based on the predicted trajectory of the obstacle, a point cloud segmentation algorithm is used to process the point cloud data of the target area to extract the actual obstacle attributes, including: The target area is actively illuminated using an infrared sensor, and the reflected signal is received to generate point cloud data of the target area. Based on the predicted obstacle trajectory, determine the search area for potential obstacles; A point cloud segmentation algorithm based on Euclidean clustering is applied to process the point cloud data of the target region, dividing the point cloud into multiple candidate clusters; Calculate the geometric features and reflection intensity features of each candidate cluster, and use a support vector machine classifier to determine whether the candidate cluster is an obstacle; Extract the size, shape, and location attributes of the confirmed obstacle clusters as actual obstacle attributes.

[0012] Preferably, a point cloud segmentation algorithm based on Euclidean clustering is applied to process the point cloud data of the target region, dividing the point cloud into multiple candidate clusters, including: Set the distance threshold and minimum number of points threshold for point cloud segmentation; Accelerate point cloud neighbor search using kd-tree structure; Traverse each point in the point cloud, calculate its Euclidean distance to its neighboring points, and group points whose distance is less than a threshold into the same cluster. Clusters with fewer points than the minimum threshold are filtered out, and valid candidate clusters are retained. For each candidate cluster, compute the centroid, bounding box, and point density descriptor.

[0013] Preferably, the comprehensive risk score and actual obstacle attributes are input into a fuzzy logic controller for data fusion to generate integrated motorcycle assisted driving control commands, including: Define fuzzy sets for the input variables, where the comprehensive risk score is divided into three fuzzy sets: low, medium, and high, and the actual obstacle attributes are divided into multiple fuzzy sets based on size and distance; Design a fuzzy rule base containing multiple "if-then" rules to describe the relationship between input variables and output control commands; Use the triangular membership function to fuzzify the input variables; The activation strength of each rule is calculated using a fuzzy inference mechanism. The centroid method is used to defuzzify the output fuzzy set, generating precise control command parameters; The design fuzzy rule base contains multiple "if-then" rules, including: Example rule: If the overall risk score is high, the obstacle size is large, and the obstacle distance is short, then an emergency braking command is output; Example rule: If the overall risk score is medium, the obstacle size is small, and the obstacle distance is far, then output a slight steering instruction; Fuzzy rules are generated by learning from expert experience or historical data. The importance of the rules is adjusted using rule weights. The defuzzified control command parameters include steering angle, throttle opening and braking force.

[0014] Preferably, the present invention also includes an integrated motorcycle driver assistance system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the steps of the aforementioned integrated motorcycle driver assistance method.

[0015] Compared with the prior art, the beneficial effects of the present invention are: The motorcycle's multi-source sensing system synchronously acquires vehicle kinematic parameters and environmental 3D point cloud data. Dynamic risk mapping is then performed using this data to generate a risk probability distribution map. The multi-source sensing system includes an inertial measurement unit (IMU), a lidar sensor, and a GPS receiver, providing vehicle attitude, environmental 3D structure, and absolute position information. Data synchronization employs a hardware triggering and software timestamp alignment mechanism to ensure spatiotemporal consistency. Vehicle kinematic parameters include dynamic information such as acceleration, angular velocity, and heading angle. Environmental 3D point cloud data is acquired through lidar scanning and includes obstacle distance, size, and shape features. The dynamic risk mapping operation uses a probabilistic grid method, discretizing the environment into grid cells and calculating the risk probability of each cell. Risk probability calculation considers factors such as the vehicle's motion state, obstacle distance, and relative speed. The risk probability distribution map is represented in a two-dimensional grid format, with each grid value representing the risk level of that area. The map update frequency is consistent with the sensor data acquisition frequency to ensure real-time performance.

[0016] Color and depth images of a road scene are captured by a stereo vision camera. Based on a risk probability distribution map, these images are input into a multi-scale feature extraction network for analysis, outputting a comprehensive risk score. The stereo vision camera provides texture and 3D geometric information of the scene. The color image contains color, texture, and semantic features, while the depth image provides precise distance information. The risk probability distribution map guides the feature extraction network to focus on high-risk areas, improving analysis efficiency. The multi-scale feature extraction network employs a convolutional neural network structure containing convolutional kernels of different scales. The lower layers of the network extract local detail features, while the higher layers extract global semantic features. Multi-scale features are fused through skip connections or feature pyramid structures, preserving information at different granularities. The comprehensive risk score is calculated using fully connected layers and activation functions, quantifying the overall hazard level of the current scene. The score values ​​are normalized for easy comparison between different scenes. The analysis process is performed in real time, meeting the timeliness requirements of driving decision-making.

[0017] Integrated motorcycle driver assistance control commands are generated by fusing comprehensive risk scores and actual obstacle attributes into a fuzzy logic controller. Actual obstacle attributes include obstacle type, size, speed, and motion trends. The fuzzy logic controller employs an IF-THEN rule base to handle uncertainty and fuzzy information. Input variables include parameters such as risk scores, obstacle distance, and relative speed. The fuzzification process converts precise input values ​​into membership degrees of fuzzy sets. Rule inference uses Mamdani or Sugeno inference mechanisms to generate fuzzy outputs. Defuzzification converts the fuzzy outputs into precise control commands. Data fusion considers the credibility and timeliness of different information sources, performing weighted integration. Control commands include driver assistance actions such as braking, steering, and warnings. The command generation process is smooth and continuous, avoiding abrupt changes that could affect driving comfort. Intelligent motorcycle driver assistance is achieved through the synergistic effect of risk mapping, visual analysis, and fuzzy control. Risk distribution maps provide the foundation for environmental perception, multi-scale analysis improves recognition accuracy, and fuzzy control enables smooth decision-making. This integrated approach significantly improves the safety and comfort of motorcycle driving. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the working principle of the integrated motorcycle driver assistance method described in this invention. Figure 2 A flowchart for data acquisition in a multi-source sensing system; Figure 3 A flowchart generated for the comprehensive risk score; Figure 4 This is a dynamic risk probability distribution diagram; Figure 5 A standardized comparison chart of control outputs under different risk levels. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1This invention provides an integrated motorcycle driver assistance system and method. The method includes: a multi-source sensing system for the motorcycle, comprising an inertial measurement unit, a lidar, and a global positioning system receiver, used to simultaneously acquire vehicle kinematic parameters and environmental three-dimensional point cloud data. A dynamic risk mapping operation generates a risk probability distribution map based on these data, which represents spatial risk probability values ​​in a two-dimensional grid. A stereo vision camera captures color and depth images of the road scene, determines the region of interest based on the risk probability distribution map, and inputs the images into a multi-scale feature extraction network for analysis, outputting a comprehensive risk score. Based on the comprehensive risk score, obstacle trajectory prediction is performed to obtain the predicted obstacle trajectory. Simultaneously, an infrared sensor scans the target area, and a point cloud segmentation algorithm is used to process the point cloud data based on the predicted obstacle trajectory to extract actual obstacle attributes. The comprehensive risk score and actual obstacle attributes are input into a fuzzy logic controller for data fusion to generate integrated motorcycle driver assistance control commands, such as steering angle, throttle opening, and braking force parameters.

[0021] Example 1: See Figure 2 An inertial measurement unit (IMU) monitors the motorcycle's angular velocity and linear acceleration to obtain vehicle kinematic parameters, including angular velocities around the X, Y, and Z axes, and linear accelerations along these axes. The IMU's sampling frequency is set to 100 Hz to ensure real-time data transmission. The acquired raw data is processed by a low-pass digital filter to reduce the impact of high-frequency noise. A lidar scanner scans the motorcycle's surroundings, generating 3D point cloud data. The lidar uses 905 nm wavelength laser pulses, with a horizontal field of view of 360 degrees, a vertical field of view of 30 degrees, an angular resolution of 0.1 degrees, and a maximum detection distance of 150 meters. The generated point cloud data includes the 3D coordinates and reflection intensity information of each point. A GPS receiver acquires the motorcycle's real-time location coordinates and timestamp. The GPS receiver supports GPS L1 and BeiDou B1I signals, employing real-time dynamic differential technology to improve positioning accuracy to the centimeter level, and timestamp accuracy to the microsecond level.

[0022] In practice, vehicle kinematic parameters, environmental 3D point cloud data, real-time position coordinates, and timestamps are time-aligned and coordinate-transformed to form a unified format multi-source data stream. The time alignment process utilizes hardware synchronization signals to ensure that the inertial measurement unit (IMU), lidar, and GPS receivers acquire data based on the same clock reference. For asynchronously arriving data packets, linear interpolation is used for timestamp alignment. The coordinate transformation process converts the IMU data from the carrier coordinate system to the northeast-northeast coordinate system and the lidar point cloud data from the radar coordinate system to a unified global coordinate system. The global coordinate system adopts the universal transverse Mercator projection coordinate system, and the transformation process involves the calculation of rotation matrices and translation vectors. The unified format multi-source data stream is encapsulated in a data packet structure, with each data packet containing a timestamp, coordinate system identifier, and sensor data payload.

[0023] In its implementation, the dynamic risk mapping operation generates a risk probability distribution map using vehicle kinematic parameters and environmental 3D point cloud data. Historical driving data of the same model of motorcycle under various road conditions are collected. The kinematic parameter set of the sample vehicles, the environmental 3D point cloud data set, and the corresponding risk event labels are extracted to form a training dataset. Risk event labels are manually labeled, with four levels: no risk, low risk, medium risk, and high risk. A dynamic risk mapping model based on a graph convolutional network is constructed. The input graph structure of the graph convolutional network is built from the environmental 3D point cloud data. Nodes in the graph represent points in the point cloud, and edges represent the spatial proximity relationships between points. Chebyshev polynomials are used for graph convolution operations in the graph convolutional layers.

[0024] In some embodiments, the risk probability value calculation of the graph convolutional network dynamic risk mapping model can be expressed in the following mathematical form: in: This represents the risk probability value of a node. This represents the Sigmoid activation function. This represents the maximum order of a Chebyshev polynomial. It is a k-th order Chebyshev polynomial. It is a normalized graph Laplace matrix. It is the node feature matrix. This is the k-th order trainable weight matrix. Node feature matrix. It is composed of the three-dimensional coordinates of the point, the reflection intensity, and the corresponding vehicle kinematic parameters.

[0025] In practice, the dynamic risk mapping model is trained end-to-end using a training dataset. Model parameters are optimized via backpropagation, with the Adam algorithm used as the optimizer and an initial learning rate of 0.001. Early stopping is employed during training to prevent overfitting. Real-time vehicle kinematic parameters and 3D point cloud data of the environment are input into the trained dynamic risk mapping model. The model outputs a risk probability value for each spatial grid, dividing the motorcycle's surrounding environment into 0.5m × 0.5m grid cells. The risk probability value is a continuous value between 0 and 1. A two-dimensional grid-based risk probability distribution map is generated based on these values. Centered on the motorcycle's current location, the map covers an area 50 meters in front and behind, and 20 meters to the left and right. The risk probability distribution map is visualized using a heatmap format, with different colors representing different risk levels.

[0026] In some embodiments, the generation process of the risk probability distribution map includes gridding and probability interpolation. The environmental space is discretized into a regular two-dimensional grid, with each grid cell measuring 0.5 meters × 0.5 meters. For each grid cell, the risk probability values ​​corresponding to all point cloud points falling within that cell are statistically analyzed, and the average value is taken as the final risk probability value for that grid cell. For grid cells not covered by point cloud data, an inverse distance-weighted interpolation method is used to interpolate risk probability values ​​from neighboring grid cells, with the interpolation weight inversely proportional to the distance. The final generated risk probability distribution map is in the form of a two-dimensional matrix, where the rows and columns of the matrix correspond to the indices of the spatial grid, and the matrix element values ​​are the risk probability values ​​of the corresponding grids.

[0027] Optionally, data augmentation techniques can be introduced during the construction of the training dataset to improve the model's generalization ability. Data augmentation techniques include random rotation, translation, and noise addition to point cloud data, and random perturbation of vehicle kinematic parameters. The proportion of data augmentation should be controlled within 20% of the original data volume. It is understood that data augmentation helps the model learn more robust feature representations and improves the prediction accuracy of the risk probability distribution map under different road conditions. The graph structure construction of the dynamic risk mapping model can adopt dynamic graph learning methods. Dynamic graph learning methods adaptively adjust the edge connectivity in the graph according to the point cloud density, reducing the number of edges in dense point cloud regions and increasing edge connectivity in sparse point cloud regions. Dynamic graph learning methods use the K-nearest neighbor algorithm to dynamically construct the adjacency matrix of the graph. It is understood that the dynamic graph structure can better adapt to the uneven distribution of point clouds at different distances, improving the accuracy of risk probability calculation.

[0028] Example 2: See Figure 3The system captures color and depth images of a road scene using a stereo vision camera. Based on a risk probability distribution map, these images are input into a multi-scale feature extraction network for analysis and output of a comprehensive risk score. The stereo vision camera is a binocular camera system mounted at the front of the motorcycle, with a horizontal field of view of 90 degrees, a vertical field of view of 60 degrees, and a baseline distance of 12 cm. It can simultaneously acquire color and depth images. The color images are in RGB format, and the depth images are generated through parallax calculation. The image resolution is 1920 pixels × 1080 pixels, and the acquisition frame rate is set to 30 frames per second to ensure real-time performance. In specific implementation, the region of interest is determined based on the high-risk areas in the risk probability distribution map. The risk probability distribution map, derived from the output of Example 1, is represented in the form of a two-dimensional grid with a grid size of 0.5 meters × 0.5 meters. High-risk areas are defined as grid cells with a risk probability value greater than 0.7. The region of interest is selected within a 30-meter radius in front and behind and a 15-meter radius to the left and right of the motorcycle's current position. After mapping the region to image coordinates, the color and depth images are cropped to a uniform size, such as 640 pixels × 480 pixels, using bounding boxes. In the specific implementation, a multi-scale feature extraction network is constructed. The multi-scale feature extraction network adopts a dual-branch architecture, with one branch processing color images and the other branch processing depth images. Each branch contains multiple convolutional layers and pooling layers. The convolutional layers use convolutional kernels of different sizes to capture multi-scale features, including 3×3, 5×5 and 7×7 convolutional kernels. The pooling layers adopt max pooling operation with a pooling window of 2×2 and a stride of 2.

[0029] In some embodiments, the two-branch feature fusion process of a multi-scale feature extraction network can be expressed in the following mathematical form: in: This represents the fused feature vector. This represents the transformation function of the fully connected layer. This represents the feature map extracted from the branch of the color image. This represents the feature map extracted from the branch of the depth image. This represents the feature stitching operation. Feature map of a color image branch. Feature maps of deep image branches are extracted through multiple convolutional and pooling layers. Networks with the same structure but independent parameters are used for feature extraction. In some embodiments, the training process of the multi-scale feature extraction network is optimized using a mean squared error loss function and an adaptive moment estimation algorithm. The mean squared error loss function calculates the squared difference between the comprehensive risk score output by the network and the true risk label. The true risk label comes from manually labeled historical data. The learning rate of the adaptive moment estimation algorithm is set to 0.001, and the batch size is set to 32.

[0030] In the implementation, the cropped color and depth images are input into a multi-scale feature extraction network. Image preprocessing includes normalization, scaling pixel values ​​to the range of 0 to 1. The network first extracts features in two branches: the color image branch has 3 input channels, and the depth image branch has 1 input channel. Each branch's convolutional layer is followed by a batch normalization layer and a ReLU activation function to enhance non-linearity. Feature maps are concatenated at a specific layer, chosen after the second pooling layer. The concatenated feature maps are then subjected to dimensionality reduction and fusion through two fully connected layers with 1024 and 512 nodes, respectively. Finally, a regression layer outputs a comprehensive risk score, which is a continuous value between 0 and 1. The training dataset for the multi-scale feature extraction network consists of historical driving data, including color images, depth images, and corresponding risk score labels. The risk score labels are defined based on actual risk events. During training, five-fold cross-validation is used to evaluate model performance and prevent overfitting.

[0031] Optionally, the calibration process for the stereo vision camera may include the calibration of both intrinsic and extrinsic parameters. Intrinsic parameter calibration uses a checkerboard method to calculate the focal length and principal point coordinates, while extrinsic parameter calibration calculates the camera's rotation and translation matrices relative to the motorcycle coordinate system. Calibration data is stored in the system for image coordinate transformation. Optionally, a residual connection mechanism can be introduced into the feature extraction stage of the multi-scale feature extraction network. This mechanism adds shallow and deep features, mitigating the gradient vanishing problem and improving network training stability. It can be understood that the dual-branch design of the multi-scale feature extraction network effectively utilizes the complementary information of color and depth images. Color images provide texture and color features, while depth images provide spatial distance features, capturing details at different levels through multi-scale convolutional kernels. The regression output of the comprehensive risk score provides a continuous risk measure for subsequent obstacle trajectory prediction, supporting more refined decision control.

[0032] Example 3: Obstacle movement sequences and corresponding comprehensive risk score sequences are collected from historical motorcycle driving data to form a sample sequence dataset. The obstacle movement sequence includes the three-dimensional position coordinates, speed, and direction angle of the obstacle. The comprehensive risk score sequence comes from the output of Example 2. The sequence length is set to 10 seconds, and the sampling frequency is 10 Hz. Therefore, each sequence contains 100 time step data points. The sample sequence dataset covers a variety of typical road conditions, such as straight roads, curves, and intersections. Each sample includes an input sequence and a corresponding future trajectory label. The future trajectory label is the position sequence of the obstacle within the next 5 seconds. The sampling interval is 0.1 seconds. In the specific implementation, a trajectory prediction model based on a Long Short-Term Memory (LSTM) network is constructed. LSTM is a recurrent neural network structure specifically designed for processing time-series data. The input layer of the trajectory prediction model receives a comprehensive risk score sequence and an obstacle state sequence. The obstacle state sequence includes position coordinates and velocity information. The input vector dimension is 6 dimensions, the comprehensive risk score is 1 dimension, and the obstacle state is 5 dimensions. The LSTM network model contains two LSTM layers, each with 128 hidden units. The LSTM layer is followed by a fully connected output layer with a 3-dimensional dimension, corresponding to the obstacle position coordinates at future time steps.

[0033] In some embodiments, the hidden state update process of a Long Short-Term Memory (LSTM) network can be represented in the following mathematical form: in: This represents the hidden state vector at time step t. This represents the sigmoid activation function. This represents the input weight matrix. The input vector at time step t includes the overall risk score and obstacle status. This represents the hidden state weight matrix. This represents the hidden state vector at time step t-1. This represents the bias vector. The hidden state vector. The trajectory prediction model is transformed into output trajectory points through a fully connected layer. These output trajectory points represent the predicted positions of obstacles in future time steps. In some embodiments, the trajectory prediction model is trained using a time-step backpropagation algorithm. Backpropagation unfolds the network along the sequence using a time algorithm, calculating the gradient at each time step. Gradient clipping is applied to prevent gradient explosion, and the gradient clipping threshold is set to 1.0.

[0034] In the specific implementation, the trajectory prediction model is trained using a sample sequence dataset. Sequence dependencies are learned through backpropagation over time steps. The training dataset is divided into training, validation, and test sets, with proportions of 70%, 15%, and 15%, respectively. Mean squared error loss is used as the loss function, calculating the squared difference between predicted and actual trajectory points. The optimizer uses stochastic gradient descent with a learning rate of 0.01 and a momentum of 0.9. The batch size is set to 32, and the maximum training epoch is 500. Early stopping is implemented based on validation set loss monitoring; training terminates when the validation loss does not decrease for 10 consecutive epochs. In the specific implementation, the real-time comprehensive risk score sequence is input into the trained trajectory prediction model, which outputs the obstacle prediction trajectory for the next time step. The real-time sequence is acquired using a sliding window method with a window length of 10 seconds and a sliding step size of 0.1 seconds. The model outputs the trajectory sequence for the next 5 seconds, with trajectory points represented in three-dimensional coordinates. Post-processing of the predicted trajectory uses a Kalman filter for smoothing, and the process noise and observation noise covariance matrices of the Kalman filter are calibrated based on historical data.

[0035] Optionally, the input features of the trajectory prediction model can be expanded to include environmental context information, such as road type and weather conditions. This environmental context information is concatenated to the input vector as an embedding vector, improving the model's adaptability to complex scenarios. Optionally, the Long Short-Term Memory (LSTM) network model can be replaced with a Gated Recurrent Unit (GRU) network. GRU networks have a simplified structure, higher training efficiency, and are suitable for resource-constrained embedded systems. It is understood that LSM networks can effectively learn long-term dependencies in time series, which is crucial for obstacle trajectory prediction tasks because trajectory changes are often influenced by historical states. It is also understood that the comprehensive risk score, as an input feature, provides a risk-aware context, helping the trajectory prediction model output more conservative predictions in high-risk scenarios, thus enhancing safety.

[0036] Example 4: Based on the obstacle prediction trajectory, a point cloud segmentation algorithm is used to process the point cloud data of the target area to extract the actual obstacle attributes. The infrared sensor uses a far-infrared band with a wavelength of 10.6 micrometers and a scanning frequency of 20 Hz. The detection distance ranges from 5 meters to 200 meters. The generated point cloud data includes the three-dimensional coordinates and reflection intensity value of each point. The reflection intensity value represents the echo intensity of the infrared signal and is quantized between 0 and 255. The obstacle prediction trajectory comes from the output of Example 3 and is represented as a sequence of future location points. Each location point includes east coordinates, north coordinates, and elevation coordinates. Based on the obstacle prediction trajectory, the search range of potential obstacles is determined. The search range is a spherical area with a radius of 15 meters centered on the trajectory point to ensure coverage of possible obstacle position changes. The point cloud data within the search range is extracted through spatial filtering to filter out points outside the range.

[0037] In the specific implementation, a point cloud segmentation algorithm based on Euclidean clustering is applied to process the point cloud data of the target area, dividing the point cloud into multiple candidate clusters. The point cloud segmentation algorithm sets a distance threshold of 0.3 meters and a minimum number of points threshold of 5. A kd-tree structure is used to accelerate the point cloud neighbor search. The kd-tree constructs the spatial distribution based on the point cloud, and the search efficiency is optimized by recursively segmenting the space. Each point in the point cloud is traversed, and its Euclidean distance to its neighboring points is calculated. Points with a distance less than the threshold are grouped into the same cluster. The clustering process is iterated until all points are processed. Clusters with fewer than the minimum number of points are filtered out, and valid candidate clusters are retained. For each candidate cluster, the centroid coordinates, bounding box size, and point density descriptor are calculated. The centroid coordinates are obtained by averaging the coordinates of points within the cluster. The bounding box size is calculated by calculating the extreme differences of the point cloud of the cluster on the three axes. The point density descriptor is defined as the ratio of the number of points within the cluster to the volume of the bounding box.

[0038] In some embodiments, the key parameter settings for the point cloud segmentation algorithm are shown in Table 1, and the parameter values ​​are calibrated and optimized based on historical data. Table 1: Key Parameter Settings for Point Cloud Segmentation Algorithm Parameter name numerical values unit describe Distance threshold 0.3 rice The maximum Euclidean distance between points is used to determine whether they belong to the same cluster. Minimum number of points 5 point Minimum number of points required for a cluster to be preserved kd leaf size 10 point The maximum number of leaf nodes when constructing a kd-tree In some embodiments, Euclidean distance calculation is used for neighbor determination in point cloud clustering, and the formula is expressed as follows: in: It is the Euclidean distance between points i and j. These are the three-dimensional coordinates of point i. These are the three-dimensional coordinates of point j. Distance calculations are based on the Cartesian coordinate system of the point cloud data, with coordinate values ​​in meters.

[0039] In practice, the geometric features and reflection intensity features of each candidate cluster are calculated. Geometric features include size, shape, and location attributes. Size is represented by the length, width, and height of the bounding box. Shape is calculated using the ratio of eigenvalues ​​from principal component analysis, such as the ratio of the first to the third eigenvalue as a flatness index. Location is represented by the three-dimensional coordinates of the cluster centroid. Reflection intensity features calculate the average reflection intensity value of points within the cluster, normalized to the range of 0 to 1. A support vector machine (SVM) classifier is used to determine whether a candidate cluster is an obstacle. The SVM classifier uses a radial basis function kernel with the kernel parameter gamma set to 0.1 and the penalty parameter C set to 1.0. The training data comes from a historical point cloud annotation dataset, with annotation categories including obstacles and non-obstacles. The feature vector integrates geometric and reflection intensity features, resulting in a 10-dimensional vector, including 3D size, 2D shape, 3D location, and 2D reflection intensity.

[0040] In practice, actual obstacle attributes are extracted from the confirmed obstacle clusters. These attributes include size, shape, and location. Size attributes record the length, width, and height of the bounding box in meters. Shape attributes store the feature vectors and eigenvalues ​​from principal component analysis, used to describe orientation and shape characteristics. Location attributes represent the obstacle core in three-dimensional coordinates in the global coordinate system. The attribute data is stored in a structured format, containing timestamps and confidence scores. The confidence scores are derived from the output probability values ​​of the support vector machine classifier.

[0041] Optionally, the infrared sensor's scanning mode can be adjusted to fan-shaped scanning to increase the point cloud density in specific areas. Fan-shaped scanning achieves 60-degree horizontal coverage and 30-degree vertical coverage by controlling lens deflection. Optionally, the point cloud segmentation algorithm can integrate density clustering methods such as DBSCAN as a supplement to handle non-spherical cluster distributions. The neighborhood radius of the density clustering method is set to 0.5 meters, and the minimum number of neighbors is set to 3 points. It can be understood that the Euclidean clustering-based point cloud segmentation algorithm is suitable for obstacle detection in structured environments, enabling rapid grouping of point clouds into candidate objects. Reflectance intensity features provide material information to assist classification, distinguishing obstacles with different reflective properties, such as metal and plastic.

[0042] See Figure 4 The risk probability of a spatial region is presented in a two-dimensional grid format, with X and Y coordinates in meters, covering a spatial range from -100m to 100m. The color gradient bar on the right represents the risk probability, ranging from dark green (0.0) to dark red (≥0.8), with higher values ​​indicating higher risk. The blue area in the figure represents the road area, and the pixel colors in other areas correspond to different risk probabilities. It can be seen that multiple high-risk pixel clusters (such as red and orange areas) exist around the road and in certain areas. These areas are potential risk areas that the motorcycle assisted driving system needs to focus on, providing an intuitive basis for spatial risk distribution for subsequent comprehensive risk scoring, obstacle trajectory prediction, and assisted driving control command generation. This demonstrates the application effect of the dynamic risk mapping model based on multi-source sensor data and graph convolutional networks in motorcycle assisted driving environmental perception.

[0043] Example 5: The comprehensive risk score and actual obstacle attributes are input into the fuzzy logic controller for data fusion to generate integrated motorcycle assisted driving control commands. The comprehensive risk score comes from the output of Example 2 and is a continuous value between 0 and 1. The actual obstacle attributes come from the output of Example 4 and include the size, shape, and position attributes of the obstacle. The size attribute is represented by length, width, and height in meters, and the position attribute is represented by three-dimensional coordinates. The input variables of the fuzzy logic controller are defined as the comprehensive risk score and the actual obstacle attributes. The actual obstacle attributes are divided into multiple fuzzy sets based on size and distance. The size fuzzy set includes three levels: small, medium, and large. The distance fuzzy set includes three levels: far, medium, and near. The distance value is calculated as the Euclidean distance between the motorcycle and the obstacle. In practical implementation, fuzzy sets of input variables are defined. The comprehensive risk score is divided into three fuzzy sets: low, medium, and high. The low fuzzy set corresponds to a score of 0 to 0.3, the medium fuzzy set corresponds to a score of 0.3 to 0.7, and the high fuzzy set corresponds to a score of 0.7 to 1.0. The size fuzzy set of the actual obstacle attributes is based on the obstacle volume. Small size corresponds to a volume of less than 0.1 cubic meters, medium size corresponds to a volume of 0.1 to 1.0 cubic meters, and large size corresponds to a volume of more than 1.0 cubic meters. The distance fuzzy set is based on relative distance. Long distance corresponds to greater than 20 meters, medium distance corresponds to 10 to 20 meters, and short distance corresponds to less than 10 meters.

[0044] In practical implementation, a fuzzy rule base is designed, containing multiple "if-then" rules to describe the relationship between input variables and output control commands. There are 27 rules, covering all combinations of input fuzzy sets. Example rules include: if the overall risk score is high, the obstacle size is large, and the obstacle distance is short, then an emergency braking command is output; if the overall risk score is medium, the obstacle size is small, and the obstacle distance is far, then a slight steering command is output. The rules are generated through expert experience and based on historical driving scenario analysis. Each rule is assigned a rule weight, with the weight value between 0 and 1, adjusting the importance of the rule. High-weight rules correspond to common dangerous scenarios. In practice, triangular membership functions are used to fuzzify the input variables. The parameters of the triangular membership functions are set according to the input domain. For fuzzy sets with low comprehensive risk scores, the vertices of the triangular functions are 0, 0.15, and 0.3. For fuzzy sets with small obstacle sizes, the vertices are 0, 0.05, and 0.1 cubic meters. For fuzzy sets with large obstacle distances, the vertices are 0, 10, and 20 meters. The fuzzification process converts the precise input values ​​into membership values ​​for each fuzzy set, with membership values ​​ranging from 0 to 1.

[0045] In some embodiments, the defuzzification process of the fuzzy logic controller uses the centroid method to generate precise control command parameters, and the centroid method calculates the weighted average of the output fuzzy set, as expressed by the formula: in: This represents the output control parameters after defuzzification, such as steering angle, throttle opening, or braking force. This represents the activation strength of the k-th rule. This indicates that the k-th rule outputs the center value of the fuzzy set. Indicates the total number of rules. Activation strength. The center value of the fuzzy set is output through fuzzy inference calculation. Based on the domain settings of the control commands, the domain of the steering angle is -30 degrees to 30 degrees, the domain of the throttle opening is 0% to 100%, and the domain of the braking force is 0 to 1000 Newtons. In some embodiments, the fuzzy inference mechanism applies Mamdani-type fuzzy inference. Mamdani-type fuzzy inference uses the minimum operator to calculate the satisfaction degree of the rule antecedent, and the output fuzzy set of the rule consequent is generated through a truncation operation. The output fuzzy sets of all rules are combined into the overall output fuzzy set, which is then defuzzified using the centroid method.

[0046] In practical implementation, a fuzzy inference mechanism is applied to calculate the activation strength of each rule. Based on the membership values ​​of the rule's antecedents, the fuzzy inference mechanism uses a product inference method, where the activation strength equals the product of the membership values ​​of the input variables. For example, for the rule "if the overall risk score is high, the obstacle size is large, and the obstacle distance is short," the activation strength is the product of the membership degrees of the fuzzy set with the high overall risk score, the fuzzy set with the large obstacle size, and the fuzzy set with the short obstacle distance. In practical implementation, the centroid method is used to defuzzify the output fuzzy set, generating precise control command parameters. These parameters include steering angle, throttle opening, and braking force. The steering angle represents the steering wheel angle in degrees; a positive value indicates right turn, and a negative value indicates left turn. The throttle opening represents the throttle opening as a percentage, and the braking force represents the braking force in Newtons. The defuzzified parameters are sent to the motorcycle actuators via the controller area network bus.

[0047] Optionally, the generation of the fuzzy rule base can be optimized through learning from historical data. Historical data learning uses clustering algorithms to automatically extract rules. Clustering algorithms such as K-means clustering group the input and output data, with each cluster center corresponding to a rule. Optionally, the triangular membership function can be replaced with a Gaussian membership function. The Gaussian membership function provides a smooth transition, and its mean and variance are calibrated based on the data distribution. It can be understood that the fuzzy logic controller can handle the uncertainty and fuzziness of input variables, making it suitable for decision-making in complex driving environments. The centroid method for defuzzification provides smooth output changes, avoiding abrupt changes in control commands and improving driving comfort.

[0048] See Figure 5This paper demonstrates the standardized control output of steering angle, throttle opening, and braking force under different risk levels. Steering angle (blue bar), throttle opening (green bar, unit % / 10), and braking force (red bar, unit N / 100), as standardized control output indicators, show a clear changing pattern with risk levels (low, medium, and high): the standardized output of steering angle increases significantly with increasing risk level, rising from 12.2 in low-risk to 22.0 in high-risk, indicating that a larger steering input is needed to avoid risks in high-risk scenarios; the standardized output of throttle opening decreases with increasing risk level, falling from 4.9 in low-risk to 2.1 in high-risk, reflecting the need to reduce throttle input to control vehicle speed in high-risk situations; the standardized output of braking force increases sharply with increasing risk level, rising from 1.8 in low-risk to 6.0 in high-risk, indicating that greater braking force is needed to ensure safety in high-risk scenarios. This variation of multiple control output indicators with risk level demonstrates that the fuzzy logic controller in integrated motorcycle driver assistance systems can accurately generate control commands for steering angle, throttle opening, and braking force based on comprehensive risk scores and actual obstacle attributes, thereby achieving safe and stable driving assistance in different risk scenarios.

[0049] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An integrated motorcycle driver assistance method, characterized in that, The method includes: The vehicle's kinematic parameters and environmental 3D point cloud data are collected synchronously through a multi-source sensing system, which includes an inertial measurement unit, a lidar, and a global positioning system receiver. A dynamic risk mapping operation is performed using the vehicle kinematic parameters and the three-dimensional point cloud data of the environment to generate a risk probability distribution map. Color and depth images of a road scene are captured by a stereo vision camera. Based on the risk probability distribution map, the color and depth images are input into a multi-scale feature extraction network for analysis, and a comprehensive risk score is output. Based on the comprehensive risk score, obstacle trajectory prediction is performed to obtain the predicted obstacle trajectory. The target area is then scanned using an infrared sensor. Based on the predicted obstacle trajectory, a point cloud segmentation algorithm is used to process the point cloud data of the target area to extract the actual obstacle attributes. The comprehensive risk score and actual obstacle attributes are input into a fuzzy logic controller for data fusion to generate integrated motorcycle assisted driving control commands.

2. The integrated motorcycle assisted driving method according to claim 1, characterized in that, The motorcycle's multi-source sensing system synchronously collects vehicle kinematic parameters and environmental 3D point cloud data, including: The angular velocity and linear acceleration of the motorcycle are monitored using an inertial measurement unit to obtain the vehicle's kinematic parameters; The environment around the motorcycle is scanned by LiDAR to generate 3D point cloud data of the environment. The real-time location coordinates and timestamp of the motorcycle are obtained using a GPS receiver; The vehicle kinematic parameters, environmental 3D point cloud data, real-time position coordinates, and timestamps are time-aligned and coordinate-transformed to form a multi-source data stream in a unified format.

3. The integrated motorcycle assisted driving method according to claim 2, characterized in that, Using the vehicle kinematic parameters and environmental 3D point cloud data, a dynamic risk mapping operation is performed to generate a risk probability distribution map, including: Collect historical driving data of the same model of motorcycles under various road conditions, extract the set of kinematic parameters of the sample vehicles, the set of three-dimensional point cloud data of the sample environment and the corresponding risk event labels to form a training dataset; Construct a dynamic risk mapping model based on graph convolutional networks; The dynamic risk mapping model is trained end-to-end using the training dataset, and the model parameters are optimized using the backpropagation algorithm. The real-time collected vehicle kinematic parameters and environmental 3D point cloud data are input into the trained dynamic risk mapping model to calculate the risk probability value of each spatial grid. A two-dimensional grid-like risk probability distribution map is generated based on the risk probability values.

4. The integrated motorcycle assisted driving method according to claim 1, characterized in that, Color and depth images of a road scene are captured using a stereo vision camera. Based on the risk probability distribution map, the color and depth images are input into a multi-scale feature extraction network for analysis, outputting a comprehensive risk score, including: Use a stereo vision camera to continuously capture road scenes, simultaneously acquiring color and depth images; Based on the high-risk areas in the risk probability distribution map, determine the region of interest; A multi-scale feature extraction network is constructed, comprising convolutional layers, pooling layers, and fully connected layers, for processing multimodal features of color images and depth images; The color and depth images are cropped to the region of interest and fed into a multi-scale feature extraction network to extract multi-level feature vectors. After fusing multi-level feature vectors, a comprehensive risk score is calculated through a regression layer.

5. The integrated motorcycle assisted driving method according to claim 4, characterized in that, A multi-scale feature extraction network is constructed, comprising convolutional layers, pooling layers, and fully connected layers, to process multimodal features of color and depth images, including: Design a dual-branch network structure, with one branch processing color images and the other processing depth images. Each branch contains multiple convolutional layers and pooling layers. Use convolutional kernels of different sizes in convolutional layers to capture multi-scale features; The feature maps of the two branches are concatenated at a specific layer, and then dimensionality reduction and fusion are performed through a fully connected layer; Enhance the nonlinearity of the network by using batch normalization layers and activation functions; During the training phase, the mean squared error loss function and the adaptive moment estimation algorithm are used for optimization.

6. The integrated motorcycle assisted driving method according to claim 1, characterized in that, Based on the comprehensive risk score, obstacle trajectory prediction is performed to obtain the predicted obstacle trajectory, including: Collect obstacle movement sequences and corresponding comprehensive risk score sequences from historical motorcycle driving data to form a sample sequence dataset; Construct a trajectory prediction model based on a long short-term memory network; The trajectory prediction model is trained using a sample sequence dataset, and sequence dependencies are learned through backpropagation over time steps. The real-time comprehensive risk score sequence is input into the trained trajectory prediction model, which outputs the obstacle prediction trajectory at the future time step.

7. The integrated motorcycle assisted driving method according to claim 1, characterized in that, The target area is scanned by an infrared sensor. Based on the predicted trajectory of the obstacle, the point cloud data of the target area is processed using a point cloud segmentation algorithm to extract the actual obstacle attributes, including: The target area is actively illuminated using an infrared sensor, and the reflected signal is received to generate point cloud data of the target area. Based on the predicted obstacle trajectory, determine the search area for potential obstacles; A point cloud segmentation algorithm based on Euclidean clustering is applied to process the point cloud data of the target region, dividing the point cloud into multiple candidate clusters; Calculate the geometric features and reflection intensity features of each candidate cluster, and use a support vector machine classifier to determine whether the candidate cluster is an obstacle; Extract the size, shape, and location attributes of the confirmed obstacle clusters as actual obstacle attributes.

8. The integrated motorcycle assisted driving method according to claim 7, characterized in that, A point cloud segmentation algorithm based on Euclidean clustering is applied to process the point cloud data of the target region, dividing the point cloud into multiple candidate clusters, including: Set the distance threshold and minimum number of points threshold for point cloud segmentation; Accelerate point cloud neighbor search using kd-tree structure; Traverse each point in the point cloud, calculate its Euclidean distance to its neighboring points, and group points whose distance is less than a threshold into the same cluster. Clusters with fewer points than the minimum threshold are filtered out, and valid candidate clusters are retained. For each candidate cluster, compute the centroid, bounding box, and point density descriptor.

9. The integrated motorcycle assisted driving method according to claim 1, characterized in that, The comprehensive risk score and actual obstacle attributes are input into a fuzzy logic controller for data fusion to generate integrated motorcycle assisted driving control commands, including: Define fuzzy sets for the input variables, where the comprehensive risk score is divided into three fuzzy sets: low, medium, and high, and the actual obstacle attributes are divided into multiple fuzzy sets based on size and distance; Design a fuzzy rule base containing multiple "if-then" rules to describe the relationship between input variables and output control commands; Use the triangular membership function to fuzzify the input variables; The activation strength of each rule is calculated using a fuzzy inference mechanism. The centroid method is used to defuzzify the output fuzzy set, generating precise control command parameters; The design fuzzy rule base contains multiple "if-then" rules, including: Example rule: If the overall risk score is high, the obstacle size is large, and the obstacle distance is short, then an emergency braking command is output; Example rule: If the overall risk score is medium, the obstacle size is small, and the obstacle distance is far, then output a slight steering instruction; Fuzzy rules are generated by learning from expert experience or historical data. The importance of the rules is adjusted using rule weights. The defuzzified control command parameters include steering angle, throttle opening and braking force.

10. An integrated motorcycle driver assistance system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the integrated motorcycle assisted driving method according to any one of claims 1 to 9.

Images