Electric two-wheeled vehicle lane departure detection and early warning method and related equipment

Abstract

The invention provides an electric two-wheeled vehicle lane departure detection and early warning method and related equipment, and the method comprises the steps: obtaining the driving state information and road environment information of a vehicle, the driving state information comprises inertial motion parameters collected by an inertial sensor, the road environment information comprises road image data acquired by a camera and target point cloud data acquired by a millimeter wave radar; performing time synchronization and space coordinate calibration on the driving state information and the road environment information, and constructing a fusion data frame under a unified space-time coordinate reference system; state characteristic parameters of the vehicle are extracted based on the fused data frame, and a state vector is constructed; inputting the state vector into a deviation judgment model for analysis, and judging whether the vehicle has lane deviation or not; and if lane departure exists, early warning prompt is executed. The lane departure detection method and the lane departure detection device are suitable for sensing requirements of the two-wheeled vehicle in an unstructured road environment, and accuracy and robustness of lane departure detection on a two-wheeled vehicle platform with low computing power and low cost are improved.

Description

Technical Field

[0001] This application relates to the technical field of intelligent control of electric vehicles, and particularly to a method and related equipment for detecting and warning lane departure of electric two-wheel vehicles. Background Art

[0002] With the wide application of electric two-wheel vehicles in urban traffic, their traffic safety issues have attracted increasing attention. Due to reasons such as unstable rider control, complex road conditions, and the lack of assisted driving functions in the vehicle itself, electric two-wheel vehicles are prone to lane departure during driving, which may lead to traffic accidents such as collisions. Therefore, developing a lane departure warning system suitable for electric two-wheel vehicles has become an important direction to improve riding safety.

[0003] Currently, in the automotive field, although several lane departure warning systems based on image recognition (such as Mobileye) have been put into commercial use. Automobiles have a closed body structure and abundant sensor installation space, which can support high-power consumption and complexly installed vision systems and radar systems. However, electric two-wheel vehicles have a compact and exposed body, limited positions available for sensor installation, and require devices to have good dustproof, waterproof, and shockproof performance. This makes it difficult to "port" the highly integrated solutions of automobiles to the electric two-wheel vehicle platform with cost control, resource limitation, and compact size at the physical level.

[0004] In the existing lane departure detection technology for electric two-wheel vehicles, image processing algorithms based on a monocular camera are usually used to identify and locate lane lines, such as traditional image feature extraction methods like Hough transform. However, such technologies have problems in practical applications such as poor environmental adaptability, high requirements for computing resources, and lack of multi-source perception ability, resulting in weak perception ability and overall anti-interference ability of the vehicle, and being prone to misjudgment or missed judgment in complex or unstructured road environments, and insufficient system robustness. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, this application provides a method and related equipment for detecting and warning lane departure of electric two-wheel vehicles, so as to at least solve the problems of low recognition accuracy and insufficient system robustness existing in the existing lane departure detection technology for electric two-wheel vehicles.

[0006] To achieve the above purposes and other advantages, some embodiments of this application provide the following aspects:

[0007] In a first aspect, some embodiments of this application provide a method for detecting and warning lane departure of electric two-wheel vehicles, including:

[0008] Obtain the driving state information and road environment information of the vehicle. The driving state information includes inertial motion parameters collected by an inertial sensor, and the road environment information includes road image data collected by a camera and target point cloud data collected by a millimeter-wave radar;

[0009] Synchronize the driving state information and the road environment information in time and calibrate their spatial coordinates to construct a fused data frame in a unified spatio-temporal coordinate reference system;

[0010] Extract the state characteristic parameters of the vehicle based on the fused data frame and construct a state vector;

[0011] Input the state vector into a deviation judgment model for analysis to determine whether the vehicle has deviated from the lane;

[0012] If it is determined that there is a lane deviation, execute corresponding warning prompts according to the deviation degree and vehicle speed.

[0013] In a second aspect, some embodiments of the present application provide an electronic device, which includes:

[0014] One or more processors; and a memory storing computer program instructions, which when executed cause the processor to execute the electric two-wheeler lane departure detection and warning method as described in any one of the above.

[0015] In a third aspect, some embodiments of the present application provide a computer-readable storage medium, on which computer programs and / or instructions are stored, and when the computer programs and / or instructions are executed by a processor, the electric two-wheeler lane departure detection and warning method as described in any one of the above is implemented.

[0016] In a fourth aspect, the present application provides a computer program product, including a computer program and / or instructions, and when the computer program / instructions are executed by a processor, the electric two-wheeler lane departure detection and warning method as described in any one of the above is implemented.

[0017] Compared with the related technologies, in the solution provided by the embodiments of the present application, the technical solution of the present application is oriented to the operation scenario of electric two-wheel vehicles, integrates multi-source perception data collected by cameras, millimeter-wave radars and inertial sensors, constructs a fusion data frame in a unified spatio-temporal coordinate reference system, and can adapt to the perception requirements of two-wheel vehicles in unstructured road environments such as drastic changes in illumination, frequent occlusions or blurred lane lines, improving the accuracy and robustness of lane departure detection on low-computing-power and low-cost two-wheel vehicle platforms. This solution can adaptively process the deviation determination in complex traffic scenarios with and without lane lines, solving the problems of traditional two-wheel vehicles relying only on visual perception, having low detection accuracy and high false positive rate under dynamic working conditions. It significantly enhances the environmental adaptability of the system and the actual application effect. In view of the characteristics of two-wheel vehicles such as light weight and sensitive handling, the system combines the deviation degree and vehicle speed to set a multi-level early warning response mechanism, and timely prompts the driver to correct the direction through visual and auditory means, enhancing the riding safety and the user's operation perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these drawings.

[0019] Figure 1 It is a schematic flow chart of a method for lane departure detection and warning of an electric two-wheel vehicle provided by an embodiment of the present application;

[0020] Figure 2 It is a schematic logical diagram of a lane departure detection and warning system for an electric two-wheel vehicle provided by an embodiment of the present application;

[0021] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0023] Currently, in the automotive field, lane departure warning systems based on image recognition (such as Mobileye) have been commercially applied. However, such systems generally have problems such as high cost (unit price of equipment exceeding $500) and high power consumption (exceeding 5W). Moreover, automobiles have a closed body structure and abundant sensor layout space, and usually carry a large-capacity 12V battery, which, combined with a main control chip or embedded processor with relatively high performance, can fully support a visual perception system and GPU computing platform with a power consumption greater than 5W. In contrast, electric two-wheelers mainly rely on small lithium batteries for power supply, and the overall vehicle power budget is limited. The MCU control unit or low-computing-power chips are difficult to support complex image processing algorithms, resulting in the difficulty of transplanting or deploying mature solutions for the automotive side on the two-wheeler platform.

[0024] In existing lane departure detection technologies for electric two-wheelers, image processing algorithms based on monocular cameras are often used to identify and locate lane lines. For example, traditional image feature extraction methods such as Hough transform are used. However, such technologies have the following problems in practical applications:

[0025] Poor environmental adaptability: Existing image processing methods are highly sensitive to changes in illumination. Especially in low-illumination environments such as at night, in tunnels, or on rainy and cloudy days, the image contrast and clarity decrease, resulting in a significant reduction in the accuracy of lane line recognition. In some scenarios, the recognition rate is lower than 60%, which cannot meet the all-weather application requirements.

[0026] High computational resource requirements: Traditional image processing algorithms have a relatively high computational complexity and are difficult to achieve real-time processing on low-power embedded chips (such as ARM Cortex-M3) commonly used in electric two-wheeler platforms. The image processing frame rate is less than 10 frames per second, and it cannot respond in a timely manner to rapidly changing road conditions, posing a risk of processing delay.

[0027] Lack of multi-source perception ability: Existing systems do not integrate other sensors (such as inertial measurement units or other types of sensors such as millimeter-wave radars) for multi-source perception data fusion, lack the ability to perceive the dynamic attitude of the vehicle and the distance to the front target, and the overall anti-interference ability is weak. It is prone to misjudgment or missed judgment in complex or unstructured road environments, and the system robustness is insufficient.

[0028] Electric two-wheeler: A light vehicle that uses electric energy as the driving energy, has two wheels, and is steered and powered by a rider. It usually includes components such as an electric motor, a battery pack, a frame, front and rear wheels, and a controller. Compared with traditional four-wheel vehicles, electric two-wheelers have the characteristics of small size, simple structure, flexible handling, and small inertia, and are suitable for urban short-distance travel scenarios. In this application, the electric two-wheeler includes, but is not limited to, types such as electric bicycles and electric motorcycles.

[0029] The embodiments of this application are also applicable to electric three-wheeler platforms that have certain structural similarities.

[0030] The first embodiment

[0031] The first embodiment of the present application relates to a method for detecting and warning lane departure of an electric two-wheeler. Referring to Figure 1 as shown, the method may include the following steps:

[0032] Step S1: Obtain the driving state information and road environment information of the vehicle. The driving state information includes inertial motion parameters collected by an inertial sensor, and the road environment information includes road image data collected by a camera and target point cloud data collected by a millimeter-wave radar.

[0033] Regarding step S1, specifically, a data acquisition module for multi-sensor collaborative work is constructed to simultaneously obtain the driving state information of the vehicle and the surrounding road environment information. This data acquisition module integrates sensing units such as a camera, a millimeter-wave radar, and an inertial sensor. Through the fusion acquisition of multi-source information, the system can stably sense the vehicle operation state and environmental changes under different lighting, climate, and road structure conditions, ensuring comprehensive and accurate basic data support for subsequent state estimation and lane departure judgment.

[0034] The camera is set at a position slightly above the front of the vehicle head. It is usually installed near the headlight cover, above the instrument panel, or near the headlight assembly, facing directly in front of the vehicle, and maintaining a horizontal or slightly downward tilted installation angle to ensure that its shooting field of view covers the road images within a certain distance range in front, including lane lines, traffic signs, obstacles, etc. Its image frame rate is generally 20 - 30fps.

[0035] To improve the quality and stability of road image acquisition, the camera preferably uses an image sensor product with high resolution, low illuminance sensing ability, and wide dynamic range characteristics. Specifically, the camera can still clearly capture the road information in front under low light conditions, meeting the image sensing requirements in weak light scenarios such as at night or on cloudy days; at the same time, in environments with direct strong light, alternating shadows, or backlighting, relying on its wide dynamic range characteristics, it can effectively suppress overexposure and underexposure phenomena and maintain good imaging effects.

[0036] The millimeter-wave radar is used for target detection of the vehicle's surrounding environment. Specifically, it emits electromagnetic wave signals in the millimeter-wave band, receives the reflected echoes from the target objects, and combines information such as the time delay and Doppler frequency shift of the signals to calculate the distance, relative speed, and angle parameters of the target. The millimeter-wave radar has the characteristics of long detection range, high measurement accuracy, strong penetrability, etc., and is not affected by external environmental factors such as rain, fog, strong light, or low illuminance, and can maintain stable and reliable sensing performance in complex road scenarios. In the embodiments of this application, the millimeter-wave radar is preferably installed at the center of the front end of the vehicle or on the side of the vehicle body (such as the front fender or the front bumper) so that its detection field of view covers the front or lateral area, thereby real-time sensing the distance and relative speed between the vehicle and surrounding vehicles, boundary structures, or obstacles, and providing key sensing data support for lane departure risk identification and state vector construction.

[0037] The inertial sensor includes an accelerometer and a gyroscope, which are used to collect the inertial motion parameters of the vehicle during driving. The accelerometer reflects the linear motion states of the vehicle such as acceleration, deceleration, or up and down bumps by measuring the linear acceleration of the vehicle in different directions; the gyroscope is used to measure the angular velocity of the vehicle around each axis, and then calculate the steering, tilting, or attitude changes generated by the vehicle during driving. The inertial sensor is usually fixedly installed at a position close to the center of mass of the vehicle main frame (such as the middle axis position of the frame) to reduce the influence of environmental interference on the measurement accuracy. During the operation of the electric two-wheeler, the inertial sensor can real-time collect the dynamic motion changes caused by driving operations (such as acceleration, braking, steering), and transmit the collected acceleration and angular velocity data to the data processing module for constructing the state vector and attitude solution of the vehicle, providing high-timeliness and high-precision motion feature input for lane departure judgment, and improving the detection accuracy and response ability of the system under dynamic working conditions.

[0038] Step S2: Synchronize the driving state information and the road environment information in time and calibrate the spatial coordinates to construct a fusion data frame in a unified spatio-temporal coordinate reference system.

[0039] Regarding step S2, specifically, a data structure unit containing various sensing information is formed after the data collected by multiple sensors are processed by time alignment and coordinate transformation under the same time reference point and unified coordinate reference system. This data structure unit organizes and stores the timestamp, road image data, target point cloud data, inertial motion parameters, etc. in the same frame in a structured manner. The timestamp serves as a unified anchor point for various sensor data to ensure that the image data, target point cloud data, and inertial motion parameters all reflect the vehicle state and environmental perception results at the same moment, so as to form a snapshot of the vehicle's motion state and surrounding environment perception results at the current moment. This step ensures the data consistency and high complementarity between different sensors, and is an important prerequisite for realizing high-precision multi-modal perception.

[0040] Step S3: Extract the state characteristic parameters of the vehicle based on the fused data frame and construct a state vector.

[0041] Regarding step S3, specifically, the system extracts state characteristic parameters that can reflect the current position, attitude, and motion trend of the vehicle based on multi-source information such as image data, target point cloud data, and inertial motion parameters included in the fused data frame, and constructs a vehicle state vector on this basis. The state vector is used to describe the dynamic state of the vehicle at the current moment and usually includes elements such as the lateral position, longitudinal position, velocity components, heading angle, and yaw rate of the vehicle in the vehicle coordinate system. This step converts the perception data into a standardized input format that can be used for model calculation through multi-source feature fusion and structured parameter modeling, providing an accurate and stable vehicle state representation for subsequent deviation judgment.

[0042] Step S4: Input the state vector into the deviation judgment model for analysis to determine whether the vehicle has deviated from the lane.

[0043] Regarding step S4, specifically, the system takes the state vector constructed in step S3 as input and inputs it into the deviation judgment model for analysis and processing. The deviation judgment model is used to comprehensively analyze the state characteristics of the vehicle such as the current position, speed, and attitude, and combine reference information such as the lane center line, road boundary, or historical trajectory to determine whether the vehicle has a risk of deviating from the current lane. This model can be flexibly adapted to different scenarios with or without lane lines. By jointly modeling the elements in the state vector, it realizes the intelligent recognition of lane deviation behavior. The execution of this step helps to timely identify potential deviation trends during dynamic driving and provides a judgment basis for subsequent warning strategies.

[0044] Step S5: If it is determined that there is a lane deviation, execute corresponding warning prompts according to the deviation degree and vehicle speed.

[0045] Regarding step S5, specifically, when the analysis result output by the deviation judgment model indicates that the vehicle has a risk of lane deviation, the system will comprehensively consider the deviation degree and the current driving speed of the vehicle and execute the corresponding warning prompt strategy. The warning prompt can be jointly output through controlling visual prompt devices (such as vehicle lights, indicator lights) and auditory prompt devices (such as buzzers, voice modules) in the front end or instrument area of the vehicle, and remind the driver of the risk with different prompt intensities or modes. So that the system can issue prompts in a timely manner when detecting a lane deviation trend, prompting the driver to quickly perceive and make corrective operations, thereby effectively reducing the safety risks caused by deviation and enhancing the active prevention and control ability of the whole vehicle during dynamic driving.

[0046] It is not difficult to find that, compared with the related technologies, the solution provided in the embodiment of the present application proposes a concept of lightweight multi-source perception fusion and lane departure intelligent recognition for the operation scenario of electric two-wheelers. By fusing the multi-source perception data collected by cameras, millimeter-wave radars, and inertial sensors, a fusion data frame in a unified spatio-temporal coordinate reference system is constructed, which can meet the perception requirements of two-wheelers in unstructured road environments with drastic changes in illumination, frequent occlusions, or blurred lane lines, and improve the accuracy and robustness of lane departure detection on low-computing-power and low-cost two-wheeler platforms. This solution can adaptively process the deviation determination in complex traffic scenarios with and without lane lines, and solves the problems of traditional two-wheelers relying only on visual perception, having low detection accuracy and a large misjudgment rate under dynamic working conditions. It significantly enhances the environmental adaptability of the system and the actual application effect. In view of the characteristics of two-wheelers such as light weight and sensitive control, the system combines the deviation degree and vehicle speed to set a multi-level early warning response mechanism, and timely prompts the driver to correct the direction through visual and auditory means, enhancing the riding safety and the user's operation perception.

[0047] Second Embodiment

[0048] The second embodiment of the present application relates to a method for detecting and warning lane departure of an electric two-wheeler. The second embodiment is an improvement based on the first embodiment. The specific improvement lies in: in the second embodiment of the present application, a specific implementation method for time synchronization and spatial coordinate calibration of multi-source perception data is provided. That is, step S2 can further include the following steps:

[0049] Step S201: Send a unified GPIO trigger signal to the camera, millimeter-wave radar, and inertial sensor through the controller to control each sensor to collect data at the same moment;

[0050] Step S202: Take the data output frequency of the millimeter-wave radar as the time synchronization reference, and align the data frames collected by the inertial sensor and the camera respectively to generate a joint data frame under the synchronous timestamp;

[0051] Step S203: Set multiple two-dimensional image feature points on the calibration board, and obtain the corresponding coordinate points of the two-dimensional image feature points in the camera image coordinate system and the millimeter-wave radar point cloud coordinate system;

[0052] Step S204: Construct an error function based on the corresponding coordinate points, use the Euclidean distance between the image feature points and the point cloud reflection points as the metric, use the Levenberg-Marquardt optimization algorithm to minimize the error function, and jointly solve the external parameter matrix between the sensors;

[0053] Step S205: Based on the extrinsic parameter matrix, map the inertial motion parameters, road image data, and target point cloud data included in the combined data frame to the vehicle coordinate system to complete the spatial coordinate calibration, thereby forming a fused data frame with time synchronization and spatial registration.

[0054] Specifically, the system controller (such as the ECU control module), as the control center of various sensors, can simultaneously send trigger signals to the camera, millimeter-wave radar, and inertial sensor through the GPIO interface to achieve hardware-level linkage control, enabling each sensor to start the sampling process synchronously at the same moment, ensuring the time consistency of various perception data from the source and reducing the registration error caused by acquisition delay. The GPIO trigger signal is a digital level signal (high level or low level) sent by the system controller or the main control chip, used to simultaneously control multiple sensors to start collecting data to achieve a hardware-level synchronous trigger mechanism. This synchronous trigger mechanism has high precision and small delay and is usually used in conjunction with an algorithm for alignment compensation (such as an interpolation algorithm) to achieve the time synchronization conditions required for precise fusion.

[0055] Exemplarily, in the time synchronization alignment stage, the sampling frequency of the camera is 20 frames per second (20fps), the output frequency of the millimeter-wave radar is 50Hz, and the data update frequency of the inertial sensor is 100Hz. For the above data sources with different frequencies, the system uses linear interpolation to downsample the high-frequency data of the inertial sensor to align it with the radar data to a 50Hz time reference; at the same time, through bilinear interpolation, the camera image frames are aligned in time to the corresponding radar data frames, enabling different types of perception data to be processed collaboratively under the same time reference point. The above synchronization alignment process ensures that the time difference between the data of each sensor in the finally constructed fused data frame is less than 10 milliseconds, effectively improving the timeliness and synchronization accuracy of multi-source data fusion.

[0056] In the spatial coordinate calibration stage, to achieve the spatial registration of the perception data between the camera and the millimeter-wave radar, a spatial coordinate calibration method based on the hand-eye calibration algorithm is used. Specifically, the camera internal parameters are calibrated by the Zhang Zhengyou calibration method, and the extrinsic parameter matrix between the camera and the millimeter-wave radar, including the three-dimensional rotation matrix R and the translation vector T, is solved by combining the Levenberg-Marquardt optimization algorithm. In the calibration stage, no less than 10 two-dimensional image feature points are set on the calibration board, and the corresponding coordinate points of the two-dimensional image feature points in the camera image coordinate system and the radar point cloud coordinate system are obtained. Subsequently, an error function is constructed to measure the registration error between the corresponding coordinate points, and the error function is defined as follows:

[0057]

[0058] where p cam,idenotes the image feature point corresponding to the i-th coordinate point in the camera coordinate system, p radar,i denotes the point cloud feature point corresponding to the i-th coordinate point in the radar coordinate system, where R and T are the three-dimensional rotation matrix and translation vector respectively.

[0059] The Levenberg-Marquardt optimization algorithm combines the advantages of the gradient descent method and the Gauss-Newton method, and can achieve fast convergence in non-linear least squares problems, and is used to accurately fit the spatial correspondence relationship between the image feature points and the point cloud feature points. Jointly solve the optimal extrinsic parameter matrix R and T to ensure that the spatial registration error between the camera and the radar is less than 5 cm.

[0060] Based on the solved extrinsic parameter matrix, the road image data, the target point cloud data, and the inertial motion parameters are uniformly mapped into the vehicle coordinate system. Thus, the system can complete the time synchronization and spatial registration of multi-source perception data in terms of time and space, and finally generate a fused data frame as the input basis for subsequent state estimation.

[0061] It is not difficult to find that in the embodiments of the present application, the GPIO trigger signal is used to realize the synchronous sampling of multiple sensors at the same moment, combined with the time interpolation alignment method based on the millimeter-wave radar, effectively solving the time drift problem caused by different sampling frequencies; at the same time, by setting the feature corresponding points of the image and the point cloud, the Levenberg-Marquardt optimization algorithm is used to iteratively solve the error function, accurately obtaining the extrinsic parameter matrix between the camera and the millimeter-wave radar, ensuring the spatial geometric consistency of the data in the same coordinate system. Further enhancing the spatio-temporal consistency of multi-sensor data fusion, significantly improving the time synchronization accuracy and spatial registration accuracy of the fused data frame.

[0062] In this embodiment, after completing the time synchronization and spatial coordinate calibration, it further includes data preprocessing of the data collected by each sensor, specifically including:

[0063] Performing image enhancement and semantic segmentation on the road image data, extracting the lane line area and determining the lane center line coordinates;

[0064] Performing a constant false alarm rate detection algorithm on the target point cloud data for filtering, removing clutter points, and retaining effective boundary target points;

[0065] Processing the acceleration and angular velocity data in the inertial motion parameters using a complementary filtering algorithm to calculate the vehicle's real-time attitude angle and acceleration.

[0066] Exemplarily, for the road image data collected by the camera, first, a bilateral filtering method is used to denoise the image to retain the image edge information and suppress the high-frequency noise interference; subsequently, the contrast of the image in the low-illumination area is enhanced by the adaptive histogram equalization (CLAHE) method, thereby improving the overall image clarity. On the basis of image enhancement, the system further uses a semantic segmentation algorithm to identify the lane lines in the road image, and extracts the sequence of center points of the lane line area L = {(x i , y i )} according to the segmentation result, forming the coordinate set of the lane center line.

[0067] For the target point cloud data collected by the millimeter-wave radar, a constant false alarm rate detection (CFAR) algorithm is used to screen the original radar reflection points to filter out the invalid clutter interference in the environmental background, and only retain the effective point cloud data with boundary structure characteristics (R = {(d j , θ j , v j )}). The output point cloud data includes the distance d j , angle θ j and relative velocity v j and other attribute information of each target point, which can be used to construct the road boundary or obstacle contour.

[0068] Meanwhile, the inertial motion parameters output by the inertial sensor are fused. Specifically, the data of the three-axis accelerometer and gyroscope are used as inputs, and a complementary filtering algorithm is used for data fusion to real-time calculate the attitude information of the vehicle, and output the roll angle φ, pitch angle θ, yaw angle ψ of the vehicle and the three-axis acceleration (A = (a x , a y , a z )) of the vehicle.

[0069] The above preprocessing operations can significantly improve the structural degree and physical meaning of various types of original sensor data, providing high-quality inputs for subsequent construction of the state vector and deviation judgment.

[0070] Third Embodiment

[0071] The third embodiment of the present application relates to a lane departure detection and warning method for an electric two-wheeler. The third embodiment is an improvement based on the first embodiment. The specific improvement lies in: in the third embodiment of the present application, a specific implementation manner based on the fusion estimation of feature-level correlation matching and extended Kalman filter algorithm is provided. That is, step S3 can further include the following steps:

[0072] Step S301: Under a unified spatio-temporal coordinate reference system, establish a matching correlation matrix between sensor observation data. Use the Hungarian algorithm to associate and match the center points of lane lines extracted from road image data with static target points reflecting boundary features in the target point cloud data, and eliminate mismatched point pairs with a distance change rate exceeding a set threshold to construct a fused feature set as the observation input for state estimation.

[0073] Step S302: Construct a state vector with a multi-dimensional structure. Based on the fused feature set and the historical state vector, use the extended Kalman filter algorithm to estimate and update the vehicle state to output the current state vector.

[0074] Exemplarily, under a unified spatio-temporal coordinate reference system, the system establishes an observation feature correlation matrix between the camera and the millimeter-wave radar, and uses the Hungarian algorithm to perform data matching on the observation results from the two sensors. Specifically, the center point coordinates of the lane lines extracted from the camera image are corresponded one by one with the static target points with boundary features in the millimeter-wave radar point cloud data, and a cost matrix is constructed by calculating indicators such as the distance change rate and spatial offset to screen out the feature point pairs that meet the matching rules.

[0075] To enhance the credibility of feature matching, the system analyzes the relative distance change rate of the matched point pairs. If the distance change rate exceeds a preset threshold (such as 0.5 m / s), it is determined as a dynamic target and eliminated to ensure that the matching result only retains static boundary features. Finally, a fused feature set is obtained as the observation input for the subsequent state estimation model, including observable quantities such as the lateral offset of the lane and the radar target distance.

[0076] In the state estimation stage, the system constructs a state vector X = [x, y, v x , v y , ψ, ω] including dimensions such as the lateral position x, longitudinal position y, lateral velocity v x , v y , heading angle ψ, and yaw angular velocity ω of the vehicle, and combines the historical state and the fused observation features for dynamic estimation through the extended Kalman filter (EKF) method. The state transition model adopted is as follows:

[0077]

[0078] where x k , y k are the lateral position and longitudinal position of the vehicle at the current time k, x k+1 , y k+1 are the predicted positions of the vehicle at the next time k + 1, v x , v y are the lateral velocity and longitudinal velocity of the vehicle in the vehicle coordinate system, ax 、a y are the lateral and longitudinal accelerations of the vehicle, ψ k is the heading angle of the vehicle at the current moment, ψ k+1 is the predicted value of the heading angle at the next moment k+1, ω is the yaw rate of the vehicle, and Δt is the time step.

[0079] During the prediction process, multi-source observables such as the lateral offset from the image, the target distance deviation from the radar, and the attitude angle information solved by the IMU are fused to correct the estimation error and update the state covariance matrix, thereby achieving stable tracking of the current vehicle state. The system can ensure that the state estimation error is less than 10 cm, meeting the lane departure determination accuracy requirements.

[0080] In scenarios with severe noise and complex environments (such as rainy days, radar interference, etc.), the system can also introduce the particle filter algorithm as a supplementary model. The particle filter algorithm refers to a Bayesian filtering method based on Monte Carlo sampling, which is suitable for state estimation of non-linear and non-Gaussian systems.

[0081] In the embodiment of the present invention, the particle filter is used to handle the problem of abnormal observation data in high-noise or complex scenarios such as rainy days and radar clutter. This method randomly generates multiple particles (for example, 1000) in the state space, constructs a likelihood function based on multi-sensor observation data, and calculates the weight distribution of each particle, thereby achieving a probability-weighted estimation of the true state. Compared with the extended Kalman filter under the Gaussian assumption, the particle filter has stronger robustness and adaptability under conditions such as multi-modal distribution and enhanced uncertainty.

[0082] It should be noted that the third embodiment of this application can also be an improvement based on any one or more of the first embodiment to the second embodiment.

[0083] It is not difficult to find that in the embodiment of this application, by introducing the correlation matching mechanism of the image and radar features and combining the joint state estimation method of the extended Kalman filter, the accuracy and robustness of vehicle state perception are effectively improved. This solution is especially suitable for working conditions where there is perception uncertainty or road structure ambiguity during the driving of electric two-wheelers, and significantly enhances the reliability and accuracy of the departure detection system under practical deployment conditions.

[0084] Fourth Embodiment

[0085] The fourth embodiment of this application relates to a method for detecting and warning lane departure of an electric two-wheeler. The fourth embodiment is an improvement based on the first embodiment. The specific improvement lies in: in the fourth embodiment of this application, a specific implementation method suitable for lane departure detection in a lane-less scenario is provided.

[0086] That is, step S4 may further include the following steps:

[0087] When lane line features meeting a preset confidence threshold are not recognized from the road image data, an automatic trigger for the no-lane-line scenario detection mode is activated, and the state vector is input into the deviation judgment model for analysis. The steps for judging whether the vehicle has deviated from the lane include:

[0088] Step SA401: Based on the target point cloud data, use the density clustering algorithm to identify the cluster of boundary feature points on both sides of the road, and fit them to form a virtual road boundary line;

[0089] Step SA402: Input the current speed and heading angle of the vehicle into the pre-constructed kinematic trajectory prediction model to predict the driving path of the vehicle within a certain period of time in the future;

[0090] Step SA403: Determine whether the minimum lateral offset distance between the driving path and the virtual road boundary line is less than the preset safety distance threshold. If it is less than the safety distance threshold, it is determined that the vehicle has a deviation risk and a deviation warning is triggered.

[0091] For electric two-wheelers, the lane lines mainly refer to the ground marking lines indicating the boundaries of non-motor vehicle lanes or areas where electric vehicles can pass. It can include: the edge lines of non-motor vehicle lanes, the dividing lines between the sidewalk and the motor vehicle lane, non-motor vehicle icon lines, and auxiliary marking lines with actual navigation meanings, etc. One side boundary of the road boundary line can be the dividing line between the non-motor vehicle lane and the motor vehicle lane, guardrail, green belt, isolation pier, solid line, etc., and the other side boundary can be the boundary between the non-motor vehicle lane and the sidewalk, green belt, road edge, wall, steps, etc. For example, on an urban road, one side of the road boundary may be a metal isolation guardrail separating the non-motor vehicle lane from the motor vehicle lane, and the other side may be a stone road edge connecting to the sidewalk. The driving area of the electric two-wheeler is the non-motor vehicle lane between the two sides.

[0092] Since the millimeter-wave radar directly recognizes the "lane lines in the legal sense", but it can be recognized from point cloud features, such as surface material mutation, height change (such as road edge), vertical obstacles (guardrail, trees), and point cloud density mutation bands in geometric space. These are subjected to boundary extraction and fitting by the density clustering algorithm, and the "point clusters" aggregated represent the interface between the driving space and the non-driving space of the electric two-wheeler, constituting the virtual road boundary line.

[0093] To improve the adaptability of the lane departure detection system in complex road environments, this embodiment provides a multi-modal fusion decision-making mechanism, which is used to dynamically select appropriate departure detection strategies when there are fluctuations in the sensing capabilities of different sensors or the environment changes. Specifically, the system establishes a set of fuzzy logic controllers. Taking the sensing results of cameras, millimeter-wave radars, and inertial sensors as inputs, it outputs a fusion strategy switching signal through a pre-set logical mapping relationship and weight adjustment mechanism between multi-input conditions and output results. The logical mapping relationship means that if the lane line confidence is low and the radar boundary density is high, switch to radar fusion; if the image clarity is high and the stability is good, maintain image-dominated detection. The weight adjustment mechanism means that the system will assign different weight coefficients according to the credibility and importance of different inputs. For example, in low-light scenarios, it reduces the image weight and increases the inertial weight. The input parameters of this fuzzy logic controller include: the lane line confidence of the camera, with a value range of 0 to 1, which is used to evaluate the stability and credibility of lane line feature extraction in the current image; the clustering density of the radar road boundary, expressed as the number of target points per unit distance (unit: points / m), which is used to characterize the effectiveness of the millimeter-wave radar in road boundary detection; the variance of the acceleration signal, which is used to determine whether the current vehicle is in a stable running state. Based on the above inputs, the controller comprehensively evaluates the effectiveness and stability of each modal sensor in the current scene and outputs a fusion strategy switching signal. Exemplarily, when the lane line confidence in the camera image is lower than 0.7 and the radar clustering density reaches the set threshold, the system automatically switches to a fusion detection strategy dominated by millimeter-wave radars and inertial sensors, and can still maintain stable and reliable departure detection performance in scenarios without obvious lane line features.

[0094] Through this multi-modal fusion decision-making mechanism, the system can achieve smooth switching between lane structured and unstructured scenarios, avoid misjudgment or missed detection caused by the failure of a single sensor, and significantly improve the environmental adaptability and decision-making robustness of the lane departure detection system during actual operation.

[0095] Exemplarily, when no effective lane line features meeting the confidence requirements (such as the confidence being lower than the set threshold of 0.7) can be identified from the camera image data, the system automatically switches to the lane-less scene detection mode. At this time, based on the target point cloud data collected by the millimeter-wave radar, the system uses a density clustering algorithm (such as DBSCAN) to cluster the ground echo points and identify the boundary feature point clusters on both sides of the road. The system performs curve fitting on the extracted boundary points to form virtual road boundary lines representing the left and right boundaries of the lane.

[0096] After completing the boundary modeling, the system calls the established single-track kinematic model of the vehicle to perform trajectory prediction based on dynamic information such as speed and heading angle in the current state vector of the vehicle. This model assumes that the vehicle moves in a uniform straight line in the current heading angle direction to calculate the expected driving path in the future (e.g., within 1 second). The trajectory calculation can be formalized as:

[0097] X(t + 1) = X(t) + [v x , v y , ω]·Δt

[0098] where X(t + 1) is the predicted position at the next moment, X(t) is the current position, [v x , v y , ω] are the lateral speed, longitudinal speed, and heading angular velocity of the vehicle respectively, and Δt is the time interval.

[0099] Based on the geometric relationship between the predicted trajectory and the boundary line, the system further calculates the minimum lateral offset distance between the predicted path and the virtual boundary line. If this distance is less than the preset safety distance threshold (e.g., 0.5 m), it is determined that the vehicle has a deviation risk, and the system will trigger a deviation warning mechanism to prompt the driver to correct the driving direction in a timely manner.

[0100] Furthermore, to improve the accuracy of deviation judgment in the scenario without lane lines, the system uses the lateral variance σ of the historical trajectory error as a dynamic threshold. This variance threshold is an empirical value obtained based on statistical experience or test verification and is usually set to be less than 0.3 m. When making a deviation judgment, if the distance between the predicted path of the vehicle and the boundary line is less than σ, it is considered that no deviation has occurred; only when this threshold is exceeded will the deviation warning be triggered, thus effectively avoiding false alarms caused by minor fluctuations and ensuring that the overall false alarm rate of the system is controlled below 3%.

[0101] It is not difficult to find that in the embodiment of the present application, compared with the traditional method relying on lane line detection, this solution can effectively adapt to unstructured environments with blurred road markings, complex lighting, or unclear structures, significantly improving the environmental adaptability and continuity of the lane departure detection system, and ensuring that electric two-wheel vehicles still have a stable and reliable warning ability under diverse road conditions.

[0102] This embodiment also provides a specific implementation method for lane departure detection applicable to the scenario with lane lines. That is, step S4 can further include the following steps:

[0103] Step SB401: When lane line features that meet the preset confidence threshold and structural continuity conditions are recognized from the road image data, automatically trigger the lane line scenario detection mode, input the state vector into the deviation judgment model for analysis, and the steps of judging whether the vehicle has a lane departure include:

[0104] Step SB402: Extract the lane line area from the road image data based on the semantic segmentation model, and extract the sequence of lane line center points to form the lane center line;

[0105] Step SB403: Combine the current vehicle position with the lane center line to construct the lateral offset distance as an input feature, and fuse it with the state vector;

[0106] Step SB404: Input the fused state vector into the pre-trained lane departure judgment model, and output the lateral offset distance between the current vehicle center point and the lane center line. If the lateral offset distance exceeds the set threshold, it is determined that the vehicle has deviated, and a deviation warning is triggered.

[0107] Exemplarily, when the semantic segmentation confidence of the lane line feature area recognized in the image is higher than the set threshold (such as a confidence exceeding 0.7) and has a continuous structure, the system determines that the current is a structured road environment and starts the image-dominated deviation detection mode. To improve the image recognizability under complex lighting conditions, a low-light enhancement module is introduced as an image input preprocessing branch in this step.

[0108] The low-light enhancement module is constructed based on the Retinex-Net architecture, which can separate the reflection component and the illumination component of the image through the illumination estimation network, and perform histogram equalization processing on the illumination component, thereby enhancing the overall brightness uniformity and edge sharpness of the image. Under night, tunnel or backlight conditions, this module can improve the lane line recognition accuracy from the original about 60% to over 95%, significantly enhancing the robustness of the lane line semantic segmentation model.

[0109] The image enhanced by low light will be input into the lightweight DeepLabv3+ semantic segmentation network to extract the lane line area, and further use the pixel center fitting algorithm to extract the sequence of lane line center points to construct the lane center line representing the road geometric structure. At the same time, a region of interest (ROI) extraction strategy is introduced in the image post-processing stage, and the range of the region of interest in the image is dynamically adjusted according to the heading angle information provided by the inertial sensor in real time, and only the lane area 5 - 15 meters in front of the vehicle is retained for processing, so as to reduce the participation of redundant background information in the calculation and improve the processing efficiency.

[0110] The system combines the current vehicle position coordinates calculated by the inertial navigation unit with the lane centerline extracted from the image to calculate the lateral offset distance between the vehicle center point and the lane centerline. To improve the continuity and anti-jitter ability of detection, the system further introduces the Kalman filter tracking algorithm to track and dynamically update the lane center points in three consecutive frames of images to filter out occasional misdetection points. The trajectory association success rate of this filtering strategy exceeds 98%, which can effectively improve the temporal stability of the offset feature. Finally, the system combines the current vehicle position, lateral offset distance, and vehicle state vectors (speed, heading angle, yaw rate, etc.) for fusion to construct a unified fusion feature vector. This fusion feature vector is input into a pre-trained deviation judgment model, which can be a lightweight neural network structure or a logistic regression structure, and is used to output whether the current offset value exceeds the set threshold. If the threshold is exceeded, the system immediately triggers a warning prompt.

[0111] In this embodiment, a lightweight optimization is performed on the lane line semantic segmentation model. Specifically, the lane line semantic segmentation model is improved based on the DeepLabv3+ framework, and the backbone network is replaced from ResNet to MobileNetV3, so that the overall model parameter quantity is compressed to about 3.2MB, meeting the performance requirements for real-time inference at a speed of about 25 frames per second (fps) on the ARM Cortex-A7 chip.

[0112] To enhance the model's detection ability for curved lane lines in complex road conditions, a multi-scale atrous convolution structure is introduced into the original ASPP (Atrous Spatial Pyramid Pooling) module, specifically including using atrous convolution kernels with sizes of 3×3, 5×5, and 7×7 to achieve the fusion extraction of multi-scale receptive fields. Through the above improvements, while maintaining the high-resolution feature expression ability, the model further improves the detection robustness for lane line features of different scales and enhances the recognition ability for complex scenarios such as curved lane lines and merging sections in urban roads.

[0113] A lightweight DeepLabv3+ model is adopted to achieve semantic segmentation processing of road images on a low-power platform. Specifically, first, the knowledge distillation method is used to perform transfer training on the student model with the MobileNetV3 architecture by taking DeepLabv3+ as the teacher model. This process minimizes the prediction differences between the student model and the teacher model at the intermediate feature layer and the output layer, while significantly compressing the model structure and maintaining the accuracy without decline. After distillation optimization, the single-frame inference time of the model is reduced from the original about 120ms to about 50ms, achieving a more than 2-fold improvement in the inference speed.

[0114] On this basis, the model is further quantized to 8-bit fixed-point, that is, the floating-point operations in the model are replaced with integer operation forms, and stored and calculated in a fixed-point parameter format. Through this quantization strategy, the model's memory occupancy is reduced by about 75%, and at the same time, the computing power consumption is significantly reduced, making it well-suited for embedded deployment. After actual measurement, this model can be adapted to typical low-power MCU platforms such as STM32H7 and meet the real-time and accuracy requirements of lane departure detection tasks.

[0115] During the implementation of the embodiments of the present invention, to meet the real-time computing requirements of a low-power hardware platform, the system selects an embedded ECU integrated with an NPU (neural network processor) as the computing core and adopts a multi-task scheduling strategy to achieve parallel execution of camera image semantic segmentation and millimeter-wave radar point cloud processing. After actual measurement, the total processing delay of the image and radar tasks is controlled within 80 ms, and the system response time ≤ 200 ms, meeting the real-time requirements of the lane departure warning function for the response speed.

[0116] Through the above system-level optimization, the present invention realizes a high-precision alignment and deep fusion strategy for multi-sensor data at the algorithm logic level. At the same time, at the hardware deployment level, through the parallel computing architecture integrating a neural network processor, it realizes high robustness and low-latency operation under complex working conditions, ensuring the realization of high-precision and low-false-alarm lane departure detection and warning functions on a low-cost and low-power platform.

[0117] It should be noted that the fourth embodiment of this application can also be an improvement based on any one or more of the first to third embodiments.

[0118] The Fifth Embodiment

[0119] The fifth embodiment of this application relates to a lane departure detection and warning method for electric two-wheelers. The fifth embodiment is an improvement based on the first embodiment. The specific improvement lies in that in the fifth embodiment of this application, a specific implementation method of a lane departure warning strategy applicable to electric two-wheelers is provided. After the vehicle deviates, it automatically matches and executes a hierarchical and multi-channel warning response strategy according to the deviation degree and the current vehicle speed, so as to improve the timeliness of the deviation warning, the perception intensity and the operation safety of the driver. Specifically, step S5 further includes deviation degree determination and execution of a hierarchical warning strategy.

[0120] Step S501: Hierarchical determination of the lane departure degree, specifically including:

[0121] When the lateral offset distance is greater than or equal to the first offset threshold and less than the second offset threshold, and the vehicle speed is lower than the first speed threshold, it is determined as a slight deviation;

[0122] When the lateral offset distance is greater than or equal to the second offset threshold and less than the third offset threshold, and the vehicle speed is between the first speed threshold and the second speed threshold, it is determined as a moderate deviation;

[0123] When the lateral offset distance is greater than or equal to the third offset threshold, and the vehicle speed is greater than or equal to the second speed threshold, it is determined as a severe deviation;

[0124] Among them, the first offset threshold is less than the second offset threshold, the second offset threshold is less than the third offset threshold, and the first speed threshold is less than the second speed threshold.

[0125] Step S502: Execute corresponding warning prompts according to the deviation degree and vehicle speed, including:

[0126] When a minor deviation occurs, trigger the flashing of the blue vehicle light;

[0127] When a moderate deviation occurs, trigger the flashing of the yellow vehicle light and / or the buzzer prompt;

[0128] When a severe deviation occurs, trigger the flashing of the red vehicle light and / or the continuous alarm of the buzzer and / or the vibration of the seat;

[0129] And dynamically adjust the warning intensity and duration according to the vehicle deviation degree and driving speed.

[0130] In this embodiment, when it is determined that the vehicle has a lane deviation, in order to effectively remind the rider to correct the deviation in time and avoid the expansion of danger, the system adopts a multi-modal collaborative warning method based on the deviation degree and vehicle speed state, including three channels: sound warning, vibration warning and light warning.

[0131] The sound warning module uses a specially designed sound alarm, which can emit a clear, loud and highly recognizable alarm sound. The frequency of the alarm signal is set to about 2000 Hz, which is in the most sensitive auditory area of the human ear, so as to be accurately perceived by the rider in a noisy road environment. In order to adapt to different user preferences and noise environments, the system further provides a sound adjustment function, allowing the rider to independently adjust the volume of the warning sound within a set range according to their own needs, so as to reduce false alarm interference.

[0132] The system can also perform tactile warning through a vibration device integrated at the handlebar or seat. When the system detects the occurrence of a lane deviation event, the controller drives the vibration module to generate a vibration signal with a certain intensity and frequency to stimulate the rider's attention response.

[0133] Exemplarily, the vibration intensity and frequency can be dynamically adjusted according to parameters such as the current speed of the vehicle and the deviation level. For example, in the state of high-speed driving of the vehicle, the system enhances the vibration feedback to improve alertness; in the low-speed state, the vibration amplitude and duration are appropriately reduced to avoid unnecessary interference to the rider.

[0134] In addition, the system can also link with the vehicle body lighting module to execute visual warning prompts. When it is determined that there is a deviation risk, the turn signal or a dedicated warning light is automatically controlled to flash, transmitting the abnormal state of the vehicle to the rider and surrounding traffic participants. The lighting prompt not only has an intuitive and eye-catching visual effect but also has good visibility at a long distance, especially suitable for night or low-visibility scenarios. To improve the environmental adaptability of the lighting warning, the system optimizes the parameters of the lighting color, flashing frequency, and intensity, enabling it to effectively play the warning effect under various meteorological and road conditions.

[0135] Exemplarily, if it is a mild deviation, the control system triggers the blue warning light to flash; if it is a moderate deviation, the yellow warning light is triggered to flash, accompanied by the intermittent sounding of the buzzer; if it is a severe deviation, the red warning light is triggered to flash continuously, and the buzzer can be synchronously started to sound continuously and the seat vibration prompt device is activated, forming a multi-channel and enhanced alarm feedback.

[0136] Furthermore, to avoid misjudgment or over-prompting due to environmental changes or differences in driving habits, the system can also dynamically adjust the intensity (such as lighting brightness, buzzer volume, or vibration amplitude) and duration length of the warning signal based on the change trend of the deviation degree and vehicle speed, so as to improve the prompting effect and take into account user comfort.

[0137] It should be noted that the fifth embodiment of this application can also be an improvement based on any one or more of the first to fourth embodiments.

[0138] It is not difficult to find that in the embodiments of this application, the warning module combines multiple warning methods such as sound, vibration, and lighting, and dynamically adjusts the warning mode and intensity according to the driving speed and deviation degree of the vehicle, which can effectively improve the rider's perception ability of deviation risks and ensure that the rider can receive warning signals in a timely and accurate manner. It significantly enhances the driving safety and user control perception.

[0139] Refer to Figure 2 As shown, the embodiment of the present invention provides a lane departure detection and warning system for an electric two-wheeler. The system mainly includes: a high-definition camera, a millimeter-wave radar, an inertial sensor, a central control unit (ECU), an instrument panel, an acoustic-optic system, a vibration device, and a driver interaction interface.

[0140] Among them, the high-definition camera is used to collect image information of the road ahead, identify lane lines, lane boundaries and other visual elements; the millimeter-wave radar is used to detect target point cloud data in the front and on both sides to assist in identifying static environmental features such as road boundaries and obstacles; the inertial sensor is responsible for obtaining motion state parameters of the vehicle such as acceleration, angular velocity and attitude angle. The above three perception devices transmit the collected data to the central control unit ECU. As the core processing module of the system, ECU is responsible for time synchronization, spatial calibration, fusion processing of multi-source perception data, and judging whether there is a lane departure risk for the vehicle based on the fusion data; if a departure is detected, it further generates multi-level warning instructions according to the vehicle speed and the degree of departure.

[0141] The warning control signal output by the ECU can be transmitted to three output modules respectively: one is the instrument panel, which is used to display the vehicle operation state and departure information in real time; the second is the sound and light system, which is used to control the flashing of vehicle lights and the buzzer alarm to achieve double reminders of vision and hearing; the third is the vibration device, which can be set on the seat cushion or the handlebar, and outputs a vibration signal when a departure occurs to provide tactile feedback. As the ultimate response entity of the system, the driver can observe the status display through the instrument panel interface, and can also detect the departure risk in time through the sound and light and vibration signals, and accordingly adjust the riding posture or steering operation, so as to realize the stable control of the electric two-wheeler and avoid potential safety accidents caused by deviating from the road.

[0142] Through the multi-source information perception and fusion of the camera, radar and inertial sensor, this system realizes the accurate understanding of the driving state of the electric two-wheeler and the road environment, and enhances the response ability of the system and the perception ability of the driver through the multi-modal warning mechanism. It has high real-time performance, high robustness and wide adaptability, and is suitable for ensuring riding safety in various urban and unstructured road environments.

[0143] In order to verify the feasibility and effectiveness of the electric two-wheeler lane departure detection and warning system proposed by the present invention, a series of systematic laboratory simulation tests and comprehensive verifications in actual road scenarios were carried out.

[0144] In the laboratory, a simulation test platform was built to simulate the state of electric two-wheelers under various driving conditions. Through high-precision simulation equipment, parameters such as the driving speed, steering angle, and acceleration of the vehicle were precisely controlled. At the same time, different road scenarios were set, including straight roads, curves, different types of lane lines, and various weather and lighting conditions. When simulating a rainy environment, a spraying device was used to spray water on the surface of the simulated road to simulate the impact of real rainy road conditions on camera vision and millimeter-wave radar detection. When simulating night lighting conditions, different night lighting scenarios were created by adjusting the brightness and angle of the lights. In multiple simulation experiments, the system successfully detected the lane departure of the vehicle and sent out warning signals in a timely manner, with a warning accuracy rate of over 98%.

[0145] In the actual road test phase, various types of roads were selected, including urban arterial roads, suburban roads, and rural paths, etc., to comprehensively test the performance of the system under various actual road conditions. The test vehicle was installed with the lane departure function system of the present invention and equipped with professional test equipment for recording the driving data of the vehicle and the operating state of the system. During the test, cyclists with different levels of experience were invited to participate to simulate real riding scenarios. In the test on the urban arterial road, in the face of heavy traffic flow and complex road conditions, the system could accurately identify the lane lines and timely warn of the lane departure behavior of the vehicle.

[0146] In the tests on suburban roads and rural paths, the system also showed good adaptability. On suburban roads, in the face of relatively few road signs and many curves, the system accurately judged the driving trajectory of the vehicle by fusing the data of the camera and the millimeter-wave radar, and timely detected and handled the lane departure problem. On rural paths, despite the relatively complex road conditions, such as potholes and narrowness, the system could still operate stably, effectively ensuring riding safety.

[0147] Through a large number of experiments, simulations, and actual road tests, it was fully proven that the lane departure function system of the electric two-wheeler of the present invention is technically feasible and can significantly improve the driving safety of electric two-wheelers in practical applications. The test results show that the system can accurately detect the lane departure of the vehicle, send out warning signals in a timely manner, and remind the cyclist to correct the driving direction of the vehicle, with high practical value and promotion prospects.

[0148] In addition to being applied to ordinary electric two-wheelers, the lane departure function system of the electric two-wheeler of the present invention has a wide range of other uses. In the field of shared electric bicycles, this system can effectively improve the use safety of vehicles and reduce the accident risk caused by irregular riding of users. By connecting to the shared platform, the system can also upload the driving status and warning information of the vehicle in real time, facilitating the platform to manage and maintain the vehicle. In the field of logistics distribution, as a common distribution tool, the electric two-wheeler installed with the lane departure function system of the present invention can ensure the safety of the distribution process, improve the distribution efficiency, and reduce the distribution delay and cargo loss caused by traffic accidents. This system can also be applied to electric motorcycle races, providing real-time lane departure warnings and auxiliary controls for riders, helping riders better maintain the driving trajectory of the racing motorcycles on the track and improving the race results.

[0149] The step division of the above various methods is only for clear description. When implemented, they can be combined into one step or some steps can be split into multiple steps. As long as the same logical relationship is included, they are all within the protection scope of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of its algorithm and process are all within the protection scope of this application.

[0150] In addition, some embodiments of the present application also provide an electronic device. The electronic device can be various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and so on. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0151] The electronic device includes: one or more processors; and a memory storing computer program instructions, which when executed cause the processors to execute a method for detecting and warning lane departure of an electric two-wheeler provided by any one or more of the above embodiments. Figure 3An exemplary structural diagram of the electronic device is disclosed. The electronic device includes: one or more processors 1101, a memory 1102, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component is interconnected using different buses and can be mounted on a common motherboard or otherwise installed as required. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple electronic devices can be connected, and each device provides part of the necessary operations. Among them, the components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0152] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103, and the output device 1104 can be connected by a bus or other means. Figure 3 Taking connection by bus as an example.

[0153] The input device 1103 can receive input digital or character information and generate key signal inputs related to the user settings and function controls of the electronic device, such as input devices like a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 1104 can include a display device, an auxiliary lighting device (such as an LED), a haptic feedback device (such as a vibration motor), etc. The display device can include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0154] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device for displaying information to the user (such as a cathode ray tube or an LCD monitor); and a keyboard and a pointing device (such as a mouse), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (such as visual feedback, auditory feedback); and the input from the user can be received in any form (such as voice input or tactile input).

[0155] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium. When the computer program / instruction is executed by a processor, it implements a method for detecting and warning lane departure of an electric two-wheeler provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist alone without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0156] The memory 1102 can be used as a non-transitory computer-readable storage medium for storing non-transitory software programs, non-transitory computer-executable programs, and modules. By running the non-transitory software programs, instructions, and modules stored in the memory 1102, the processor 1101 executes various functional applications and data processing of the server to implement the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0157] The memory 1102 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 1102 may optionally include a memory remotely provided relative to the processor 1101, and these remote memories can be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0159] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, compact disc read-only memory, digital versatile disc or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0160] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network or a wide area network, or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0161] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit, a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and similar devices. Additionally, some steps or functions of this application can be implemented using hardware, for example, as a circuit that cooperates with a processor to execute each step or function.

[0162] The computer program product provided by the embodiments of the present application includes one or more computer programs / instructions. When the computer programs / instructions are executed by a processor, they wholly or partly generate the processes or functions described in the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0163] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0164] The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes that fall within the meaning and scope of the equivalent elements of the claims are intended to be encompassed within the present application. Any reference signs in the claims should not be construed as limiting the claimed elements. In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. The multiple elements or devices recited in the apparatus claims may also be implemented by one element or device through software or hardware. The words "first", "second", etc. are only used for distinguishing descriptions and do not represent any specific order, nor can they be understood as indicating or implying relative importance.

[0165] As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily make changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. An electric two-wheeler lane departure detection and warning method, characterized in that, Including: Obtain the driving state information and road environment information of the vehicle. The driving state information includes inertial motion parameters collected by an inertial sensor, and the road environment information includes road image data collected by a camera and target point cloud data collected by a millimeter-wave radar; Synchronize the driving state information and the road environment information in terms of time and calibrate their spatial coordinates to construct a fused data frame in a unified spatio-temporal coordinate reference system; Extract the state characteristic parameters of the vehicle based on the fused data frame and construct a state vector; Input the state vector into a lane departure judgment model for analysis to determine whether the vehicle has deviated from its lane; If it is determined that there is a lane departure, execute corresponding warning prompts according to the degree of deviation and the vehicle speed.

2. The lane departure detection and warning method for an electric two-wheeler according to claim 1, wherein, The step of synchronizing the driving state information and the road environment information in terms of time and calibrating their spatial coordinates to construct a fused data frame in a unified spatio-temporal coordinate reference system includes: Send a unified GPIO trigger signal to the camera, millimeter-wave radar, and inertial sensor through a controller to control each sensor to collect data at the same moment; Use the data output frequency of the millimeter-wave radar as the time synchronization reference to align the data frames collected by the inertial sensor and the camera in terms of time respectively to generate a joint data frame under the synchronized time stamps; Set a plurality of two-dimensional image feature points on a calibration board and obtain the corresponding coordinate points of the two-dimensional image feature points in the camera image coordinate system and the millimeter-wave radar point cloud coordinate system; Construct an error function based on the corresponding coordinate points, use the Euclidean distance between the image feature points and the point cloud reflection points as a metric, and use the Levenberg-Marquardt optimization algorithm to minimize the error function and jointly solve the external parameter matrix between the sensors; Based on the external parameter matrix, map the inertial motion parameters, road image data, and target point cloud data included in the joint data frame to the vehicle coordinate system uniformly to complete the spatial coordinate calibration, so as to form a fused data frame with time synchronization and spatial registration.

3. The electric two-wheeler lane departure detection and warning method according to claim 1 or 2, characterized in that After the step of synchronizing the driving state information and the road environment information in terms of time and calibrating their spatial coordinates, it further includes data preprocessing of the data collected by each sensor, specifically including: Perform image enhancement and semantic segmentation on the road image data, extract the lane line area and determine the lane center line coordinates; Execute a constant false alarm rate detection algorithm on the target point cloud data for filtering, eliminate clutter points, and retain effective boundary target points; Process the acceleration and angular velocity data in the inertial motion parameters by using a complementary filtering algorithm to solve the real-time attitude angle and acceleration of the vehicle.

4. The lane departure detection and warning method for an electric two-wheeler according to claim 1, wherein The step of extracting the state characteristic parameters of the vehicle based on the fused data frame and constructing a state vector includes: In a unified spatio-temporal coordinate reference system, establish a matching association matrix between the sensor observation data, and use the Hungarian algorithm to associate and match the lane center points extracted from the road image data with the static target points reflecting the boundary features in the target point cloud data, and eliminate the mismatched point pairs whose distance change rate exceeds a set threshold to construct a fused feature set as the observation input for state estimation; Construct a state vector of a multi-dimensional structure, and based on the fusion feature set and the historical state vector, use the extended Kalman filter algorithm to estimate and update the vehicle state to output the current state vector.

5. The lane departure detection and warning method for an electric two-wheeler according to claim 1, wherein, When lane line features satisfying a preset confidence threshold are not recognized from the road image data, an un-lane-line scenario detection mode is automatically triggered. The step of inputting the state vector into a deviation judgment model for analysis to determine whether the vehicle has deviated from the lane includes: Based on the target point cloud data, use a density clustering algorithm to identify clusters of boundary feature points on both sides of the road and fit them to form a virtual road boundary line. Input the current speed and heading angle of the vehicle into the constructed kinematic trajectory prediction model to predict the driving path of the vehicle in the next period of time. Judge whether the minimum lateral offset distance between the driving path and the virtual road boundary line is less than a preset safety distance threshold. If it is less than the safety distance threshold, it is determined that the vehicle has a deviation risk and a deviation warning is triggered.

6. The lane departure detection and warning method for an electric two-wheeler according to claim 1, wherein, When lane line features satisfying the preset confidence threshold and structural continuity conditions are recognized from the road image data, a lane-line scenario detection mode is automatically triggered. The step of inputting the state vector into a deviation judgment model for analysis to determine whether the vehicle has deviated from the lane includes: Extract the lane line region from the road image data based on a semantic segmentation model and extract a sequence of lane line center points to form a lane center line. Combine the current position of the vehicle and the lane center line to construct a lateral offset distance as an input feature and fuse it with the state vector. Input the fused state vector into a pre-trained lane deviation judgment model to output the lateral offset distance between the current vehicle center point and the lane center line. If the lateral offset distance exceeds the set threshold, it is determined that the vehicle has deviated and a deviation warning is triggered.

7. The electric two-wheeler lane departure detection and warning method according to claim 5 or 6, characterized in that If it is determined that there is a lane deviation, determine the degree of lane deviation, including: When the lateral offset distance is greater than or equal to the first offset threshold and less than the second offset threshold, and the vehicle speed is lower than the first speed threshold, it is determined to be a minor deviation. When the lateral offset distance is greater than or equal to the second offset threshold and less than the third offset threshold, and the vehicle speed is between the first speed threshold and the second speed threshold, it is determined to be a moderate deviation. When the lateral offset distance is greater than or equal to the third offset threshold, and the vehicle speed is greater than or equal to the second speed threshold, it is determined to be a severe deviation. Wherein, the first offset threshold is less than the second offset threshold, the second offset threshold is less than the third offset threshold, and the first speed threshold is less than the second speed threshold. The step of performing corresponding warning prompts according to the degree of deviation and the vehicle speed includes: When a minor deviation occurs, trigger the blue vehicle light to flash. When a moderate deviation occurs, trigger the yellow vehicle light to flash and / or a buzzer prompt. When a severe deviation occurs, trigger the red vehicle light to flash and / or the buzzer to continuously alarm and / or the seat to vibrate. And dynamically adjust the warning intensity and duration according to the degree of vehicle deviation and the driving speed.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and a memory storing computer program instructions that, when executed, cause the processors to execute the electric two-wheeler lane departure detection and warning method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that, When the computer program and / or instructions are executed by the processor, the electric two-wheeler lane departure detection and warning method according to any one of claims 1-7 is implemented.

10. A computer program product, comprising a computer program and / or instructions, characterized in that, When the computer and / or is executed by the processor, the electric two-wheeler lane departure detection and warning method according to any one of claims 1-7 is implemented.

Images