Method, device, program product for prompting driving risk

Abstract

This application provides a method, apparatus, and program product for alerting driving risks. The method includes: acquiring first image data of the two-wheeled vehicle's driving direction and first state data of the two-wheeled vehicle while the vehicle is in motion; performing a region compensation operation on the first image data to obtain second image data, wherein the region compensation operation is used to perform non-uniform compensation on the road surface mapping area within one image data, and the road surface mapping area is: the mapping area of ​​the road surface area in the driving direction within one image data; identifying road surface anomalies and collision targets of the two-wheeled vehicle in the driving direction based on the second image data, and determining the bump risk of the road surface anomalies and the collision risk of the collision targets of the two-wheeled vehicle based on the first state data; determining the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk, and providing a risk alert based on the driving risk.

Description

Technical Field

[0001] This application relates to the field of electric vehicles, and more specifically, to a method, device, and program product for alerting drivers of driving risks. Background Technology

[0002] Two-wheeled vehicles, such as electric scooters, are lightweight and flexible short-distance transportation tools that can achieve intelligent obstacle avoidance using local path planning algorithms based on the Dynamic Window (DWA) method. However, existing DWA methods are insufficient in predicting the risks of collisions and bumps in the direction of travel for two-wheeled vehicles (such as electric scooters), which can easily lead to inaccurate warnings about the risks during the travel of two-wheeled vehicles.

[0003] This shows that there is a technical problem with the relevant technology: the warnings about driving risks during the operation of two-wheeled vehicles are inaccurate. Summary of the Invention

[0004] This application provides a method, device, and program product for alerting driving risks, in order to at least solve the technical problem of inaccurate alerts for driving risks during the driving of two-wheeled vehicles in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for alerting driving risks is provided, comprising: acquiring first image data of the driving direction of the two-wheeled vehicle and first state data of the two-wheeled vehicle while the two-wheeled vehicle is in motion, wherein the first image data and the first state data are time-aligned; performing a region compensation operation on the first image data to obtain second image data, wherein the region compensation operation is used to perform non-uniform compensation on a road surface mapping region within one image data, the road surface mapping region being: the mapping region of the road surface area in the driving direction within the one image data; identifying road surface anomalies in the driving direction based on the second image data, identifying collision targets of the two-wheeled vehicle in the driving direction based on the second image data, and determining the bump risk of the road surface anomaly to the two-wheeled vehicle and the collision risk of the collision target to the two-wheeled vehicle based on the first state data; determining the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk, and providing a risk alert based on the driving risk.

[0006] In some exemplary embodiments, performing a region compensation operation on the first image data to obtain second image data includes: cutting out a first image region from the first image data according to a preset shape, wherein the first image region is the road surface mapping region within the first image data; dividing the first image region into vertical partitions, and performing a compensation operation on each vertical partition according to the distance of each obtained vertical partition relative to the target edge of the first image data, to obtain a compensated first image region, wherein the compensation operation includes at least one of the following: scale magnification, contrast magnification; filling the compensated first image region with an outer rectangle to obtain the second image data.

[0007] In some exemplary embodiments, identifying road surface anomalies in the driving direction based on the second image data includes: inputting the second image data into a first target detection model and obtaining at least one candidate road surface anomaly output by the first target detection model, and the longitudinal position corresponding to each candidate road surface anomaly; determining a first confidence level for each candidate road surface anomaly according to the longitudinal region where each candidate road surface anomaly is located, wherein the road surface mapping region in the first image data is divided into multiple longitudinal regions, and the longitudinal region where each candidate road surface anomaly is located is the longitudinal region to which the longitudinal position corresponding to each candidate road surface anomaly belongs among the multiple longitudinal regions; and identifying candidate road surface anomalies whose first confidence level is higher than the confidence threshold corresponding to the longitudinal region as the road surface anomaly, wherein one of the multiple longitudinal regions corresponds to a confidence threshold.

[0008] In some exemplary embodiments, the plurality of longitudinal regions include a first longitudinal region and a second longitudinal region, wherein the distance of the first longitudinal region relative to the target edge of the first image data is less than the distance of the second longitudinal region relative to the target edge; determining the first confidence level of each candidate road surface anomaly according to the longitudinal region where each candidate road surface anomaly is located includes: performing the following determination operation on each candidate road surface anomaly as a current candidate road surface anomaly to obtain the first confidence level of each candidate road surface anomaly: when the longitudinal region where the current candidate road surface anomaly is located is the first longitudinal region, determining the second confidence level output by the first target detection model for the current candidate road surface anomaly as the first confidence level of the current candidate road surface anomaly; when the longitudinal region where the current candidate road surface anomaly is located is the second longitudinal region, determining the cumulative confidence level of the current candidate road surface anomaly in the image data sequence as the first confidence level of the current candidate road surface anomaly; wherein, the image data sequence is obtained by sequentially performing the region compensation operation on each image data in the two-wheeled vehicle's driving direction, and the image data sequence includes the second image data.

[0009] In some exemplary embodiments, identifying the collision target of the two-wheeled vehicle in the driving direction based on the second image data includes: inputting the second image data into a second target detection model and obtaining at least one candidate collision target output by the second target detection model; determining the candidate collision target whose detection box center point is located within the second image area as the collision target among the at least one candidate collision target, wherein the second image area is: the mapping area of ​​the driving channel area within the road surface area in the driving direction in the second image data.

[0010] In some exemplary embodiments, determining the bump risk of the road surface anomaly to the two-wheeled vehicle based on the first state data includes: determining the initial bump risk of the road surface anomaly to the two-wheeled vehicle by using the confidence level of the identified road surface anomaly, the area coefficient corresponding to the road surface anomaly, and the type coefficient corresponding to the road surface anomaly, wherein the area coefficient is used to indicate the proportional relationship between the detection frame parameter of the road surface anomaly and a first distance, and the type coefficient is determined by the type of the road surface anomaly; the first distance is the distance of the road surface anomaly relative to the two-wheeled vehicle determined by the detection frame parameter of the road surface anomaly; the confidence level of the road surface anomaly, the detection frame parameter of the road surface anomaly, and the type coefficient are all considered. The bounding box parameters and the type of the road surface anomaly are determined by the first target detection model and the second image data. The initial bump risk is corrected by a correction coefficient to obtain the bump risk of the road surface anomaly to the two-wheeled vehicle. The correction coefficient includes at least one of the following: a first coefficient corresponding to the first state data; a second coefficient corresponding to the offset, wherein the offset is the offset of the position of the road surface anomaly mapped in the second image data relative to the second image region, and the second image region is: the mapping region of the driving channel region in the road surface region in the driving direction in the second image data; and a third coefficient corresponding to the bump state of the vehicle traveling in front of the two-wheeled vehicle.

[0011] In some exemplary embodiments, determining the collision risk of the collision target to the two-wheeled vehicle based on the first state data includes: determining the estimated arrival time of the two-wheeled vehicle to the collision target by using a second distance of the collision target relative to the two-wheeled vehicle and the first state data, wherein the second distance is determined by the detection bounding box parameters of the collision target; the detection bounding box parameters of the collision target are determined by a second target detection model and the second image data; and determining the collision risk by the estimated arrival time, wherein the collision risk is negatively correlated with the estimated arrival time.

[0012] In some exemplary embodiments, determining the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk includes: determining a target collision risk threshold that matches the collision risk from a preset set of collision risk thresholds, and determining a first risk level corresponding to the target collision risk threshold, wherein one collision risk threshold in the set of collision risk thresholds corresponds to one collision risk level; determining a target bump risk threshold that matches the bump risk from a preset set of bump risk thresholds, and determining a second risk level corresponding to the target bump risk threshold, wherein one bump risk threshold in the set of bump risk thresholds corresponds to one bump risk level; and determining the driving risk of the two-wheeled vehicle through the first risk level and the second risk level.

[0013] In some exemplary embodiments, acquiring first image data in the direction of travel of the two-wheeled vehicle and first state data of the two-wheeled vehicle includes: acquiring third image data through an image acquisition component, wherein the image acquisition component is deployed on the two-wheeled vehicle; acquiring second state data through a display component or control component of the two-wheeled vehicle; and aligning the third image data and the second state data with timestamps to obtain the first image data and the first state data.

[0014] According to another aspect of the embodiments of this application, a driving risk warning device is also provided, comprising: an acquisition module, configured to acquire first image data in the driving direction of the two-wheeled vehicle and first state data of the two-wheeled vehicle when the two-wheeled vehicle is in a driving state, wherein the first image data and the first state data are time-aligned; a compensation module, configured to perform a region compensation operation on the first image data to obtain second image data, wherein the region compensation operation is used to perform non-uniform compensation on a road surface mapping region within an image data, the road surface mapping region being: the mapping region of the road surface area in the driving direction within the image data; a risk determination module, configured to identify road surface anomalies in the driving direction based on the second image data, identify collision targets of the two-wheeled vehicle in the driving direction based on the second image data, and determine the bump risk of the road surface anomalies to the two-wheeled vehicle and the collision risk of the collision targets to the two-wheeled vehicle based on the first state data; and a risk warning module, configured to determine the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk, and provide a risk warning based on the driving risk.

[0015] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in any of the method embodiments described above.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0017] According to the embodiments provided in this application, when a two-wheeled vehicle is in motion, first image data in the direction of travel of the two-wheeled vehicle and first state data of the two-wheeled vehicle that are time-aligned with the first image data are acquired. Then, a region compensation operation is performed on the first image data to obtain second image data. The region compensation operation is used to perform non-uniform compensation on the road surface mapping area within one image data. The road surface mapping area is the mapping area of ​​the road surface area in the direction of travel within the one image data. Furthermore, based on the second image data, road surface anomalies in the direction of travel are identified. Based on the second image data, collision targets of the two-wheeled vehicle in the direction of travel are identified. Based on the first state data, the bump risk of the road surface anomaly to the two-wheeled vehicle and the collision risk of the collision target to the two-wheeled vehicle are determined respectively. Based on the bump risk and the collision risk, the driving risk of the two-wheeled vehicle is determined, and a risk warning is given based on the driving risk. By adopting the above scheme, road surface anomalies and collision targets can be identified from the non-uniformly compensated image data. Combined with the driving status data of the two-wheeled vehicle that is time-aligned with the image data, the bump risk and collision risk in the driving direction of the two-wheeled vehicle can be accurately determined and a risk warning can be issued. This solves the problem of inaccurate warnings of driving risks during the driving process of two-wheeled vehicles in related technologies. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a schematic diagram illustrating an application scenario of a driving risk warning method according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of an optional method for alerting driving risks according to an embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating an optional method for alerting driving risks according to an embodiment of this application;

[0022] Figure 4 This is a structural block diagram of an optional driving risk warning device according to an embodiment of this application;

[0023] Figure 5 This is another structural block diagram of an optional driving risk warning device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] According to one aspect of the embodiments of this application, a method for alerting driving risks is provided. Optionally, in this embodiment, the above-described method for alerting driving risks may be applied, but is not limited to, to applications such as... Figure 1 The hardware environment shown includes a two-wheeled vehicle 102, a control terminal 104, and a server 106. Optionally, the two-wheeled vehicle can be an electric scooter. The two-wheeled vehicle 102 may have network connectivity, and the server 106 can connect to the two-wheeled vehicle 102 via a network. It can be used to provide services (e.g., application services, location services, etc.) to the two-wheeled vehicle 102 or clients installed on it. A database can be set up on or independently of the server 106 to provide data storage services to the server 106.

[0027] The control terminal 104 can be a mobile terminal or controller bound to the two-wheeled vehicle 102. The two-wheeled vehicle 102 can communicate with the control terminal 104 via a wireless network, which can include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI) and Bluetooth. Optionally, the control terminal 104 can also be directly connected to the two-wheeled vehicle 102 via a data cable or other connecting cable, and transmit signals based on the connecting cable to realize the interaction between the control terminal 104 and the two-wheeled vehicle 102.

[0028] The two-wheeled vehicle 102 may include components such as the body, front wheels, rear wheels, and seat, and may also include a Vehicle Control Unit (VCU), a complex system integrating hardware and software. For the hardware portion, the VCU may contain one or more microprocessors, microcontroller units (MCUs), and necessary input / output interfaces, memory, power modules, communication modules, etc. These hardware components constitute the physical foundation of the VCU, enabling it to receive signals, process data, send control commands, and communicate with other vehicle subsystems or external devices. For the software portion, the VCU's software may include an embedded operating system, application programs, control algorithms, and diagnostic programs. The software is responsible for parsing data from sensors and subsystems, performing complex calculations and logical judgments, and generating control signals for actuators. The software is typically written to handle functions such as vehicle powertrain, energy management, and safety control.

[0029] When two-wheeled vehicles are in motion, the risks associated with the direction of travel, such as the risk of bumps and collisions, are often not accurately predicted. Inaccurate predictions will lead to inaccurate warnings about the risks.

[0030] To at least partially address the aforementioned issues, this embodiment proposes a multimodal fusion-based driving risk warning mechanism for two-wheeled vehicles. This mechanism collaboratively processes time-aligned image and state data, identifies road anomalies and collision targets in parallel from a road surface mapping region based on longitudinal non-uniform compensation, quantifies bump risk and collision risk separately by combining state data, and finally fuses them to generate a comprehensive driving risk and triggers a warning. This significantly improves the collaborative perception and safety response capabilities of scooters to hidden bumps and sudden obstacles in complex urban roads.

[0031] The method for alerting driving risks in this application embodiment can be executed by the two-wheeled vehicle 102, or it can be executed jointly by the two-wheeled vehicle 102 and at least one of the control terminal 104 and the server 106.

[0032] Taking the driving risk warning method in this embodiment as an example, which is implemented by a two-wheeled vehicle 102, Figure 2 This is a flowchart of an optional driving risk warning method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include the following steps:

[0033] S202, when the two-wheeled vehicle is in motion, acquire first image data in the direction of travel of the two-wheeled vehicle and first state data of the two-wheeled vehicle, wherein the first image data and the first state data are time-aligned.

[0034] In one optional embodiment, the two-wheeled vehicle is an electric two-wheeled vehicle that operates on electricity and possesses intelligent control capabilities and an electronic sensing system. Specifically, the two-wheeled vehicle can be an electric scooter, an electric two-wheeled pedal vehicle, etc. Therefore, the embodiments of this application can also solve the problem of inaccurate warnings regarding the driving risks of electric scooters.

[0035] In an optional embodiment, the two-wheeled vehicle further includes an image acquisition component, a display component, and a control component. The image acquisition component is an external sensor connected to the vehicle control unit (VCU) via an interface (such as MIPI or USB). The display component is the output display terminal of the VCU, and the control component is the execution unit of the VCU.

[0036] The two-wheeled vehicle can acquire first image data and first state data of the two-wheeled vehicle in the direction of travel based on the aforementioned image acquisition component, display component, and control component, including: acquiring third image data through the image acquisition component, wherein the image acquisition component is deployed on the two-wheeled vehicle; acquiring second state data through the display component or control component of the two-wheeled vehicle; and aligning the third image data and the second state data with timestamps to obtain the first image data and the first state data.

[0037] Optionally, the image acquisition component is a camera, the display component is an instrument panel, and the control component is a motor controller, brake controller, etc. The image acquisition component is installed at the front of the two-wheeled vehicle, at a height of approximately 1.5 meters. Taking an electric scooter as an example, the image acquisition component can be installed at the handlebars or at the connection between the handlebars and the front stem of the electric scooter. In this embodiment, the image acquisition component acquires first image data in the direction of travel of the two-wheeled vehicle, and the display component or control component determines first state data, which includes the vehicle's speed and its bump status. For example, real-time state data (such as current speed, bump status, etc.) during the electric vehicle's travel can be obtained from the electric scooter's instrument panel or controller. The camera frames are timestamped with the scooter's state data to ensure that distance, speed, and bump judgments are based on data from the same moment.

[0038] Optionally, the bumpy state can be determined by the vertical acceleration data of the two-wheeled vehicle. The bumpy state is characterized by short-duration, high-frequency, high-amplitude vertical acceleration pulses. For example, if the vertical acceleration shows an instantaneous peak value >1.5g when passing over speed bumps or manhole covers, then the two-wheeled vehicle is confirmed to be in a bumpy state. Optionally, the bumpy state can also be determined by the pitch angle of the two-wheeled vehicle. For example, when the two-wheeled vehicle passes over road irregularities, it exhibits rapid, small-amplitude oscillations (amplitude range: ±2°~5°), which are positively correlated with the road surface unevenness.

[0039] S204, Perform a region compensation operation on the first image data to obtain second image data, wherein the region compensation operation is used to perform non-uniform compensation on the road surface mapping region within one image data, and the road surface mapping region is: the mapping region of the road surface region in the driving direction within the one image data.

[0040] S206, based on the second image data, identify road surface anomalies in the driving direction; based on the second image data, identify collision targets of the two-wheeled vehicle in the driving direction; and based on the first state data, determine the bump risk of the road surface anomalies to the two-wheeled vehicle and the collision risk of the collision targets to the two-wheeled vehicle, respectively.

[0041] S208, based on the bump risk and the collision risk, determine the driving risk of the two-wheeled vehicle, and provide a risk warning based on the driving risk.

[0042] Optionally, the display components of the two-wheeled vehicle are also used to provide visual warnings of driving risks; the control components can also receive control commands (such as deceleration) and adjust the vehicle speed after the two-wheeled vehicle provides risk warnings based on the driving risks.

[0043] According to the embodiments provided in this application, when a two-wheeled vehicle is in motion, first image data in the direction of travel of the two-wheeled vehicle and first state data of the two-wheeled vehicle that are time-aligned with the first image data are acquired. Then, a region compensation operation is performed on the first image data to obtain second image data. The region compensation operation is used to perform non-uniform compensation on the road surface mapping area within one image data. The road surface mapping area is the mapping area of ​​the road surface area in the direction of travel within the one image data. Furthermore, based on the second image data, road surface anomalies in the direction of travel are identified. Based on the second image data, collision targets of the two-wheeled vehicle in the direction of travel are identified. Based on the first state data, the bump risk of the road surface anomaly to the two-wheeled vehicle and the collision risk of the collision target to the two-wheeled vehicle are determined respectively. Based on the bump risk and the collision risk, the driving risk of the two-wheeled vehicle is determined, and a risk warning is given based on the driving risk. By adopting the above scheme, road surface anomalies and collision targets can be identified from the non-uniformly compensated image data. Combined with the driving status data of the two-wheeled vehicle that is time-aligned with the image data, the bump risk and collision risk in the driving direction of the two-wheeled vehicle can be accurately determined and a risk warning can be issued. This solves the problem of inaccurate warnings of driving risks during the driving process of two-wheeled vehicles in related technologies.

[0044] In some exemplary embodiments, performing a region compensation operation on the first image data to obtain second image data includes: cutting out a first image region from the first image data according to a preset shape, wherein the first image region is the road surface mapping region within the first image data; dividing the first image region into vertical partitions, and performing a compensation operation on each vertical partition according to the distance of each obtained vertical partition relative to the target edge of the first image data, to obtain a compensated first image region, wherein the compensation operation includes at least one of the following: scale magnification, contrast magnification; filling the compensated first image region with an outer rectangle to obtain the second image data.

[0045] In an optional embodiment, the first image region is a region of interest (ROI) on the road surface, with a preset shape of rectangle or trapezoid. This embodiment is performed in the early stage of detection, first cutting out the ROI of interest on the road surface ahead to reduce false detections. The specific cutting method includes: since electric scooters mostly travel straight and have a large turning radius, it is not easy for them to enter the riding area from the side road surface nearby, so the ROI is not strictly cut along the lane line, but rather narrowed at the bottom edge of the lane, cutting out a trapezoid that is relatively closer to a rectangle.

[0046] After segmenting the first image region, to address the severe perspective compression caused by the low installation height of the electric scooter's image acquisition component, which affects distant road surfaces (such as flattening, low pixel count, and weak contrast), non-uniform compensation based on the target edge is required for the first image region. The target edge is a reference benchmark used to define the vertical partitions; it refers to the bottom edge of the first image data. The boundary of the region corresponding to this bottom edge is perpendicular to the two-wheeled vehicle's travel direction and close to the vehicle. The bottom edge is the mapped boundary of this region in the first image data. Non-uniform compensation differentially performs scale magnification and contrast enhancement based on the distance (i.e., nearness) of each vertical region obtained from the vertical partitions from the target edge. This achieves the effect of preserving original details in near objects and enhancing the visibility of weak targets in distant objects, thereby compensating for the perspective flattening and brightness attenuation caused by the low camera installation height. This makes subsequent road anomaly and obstacle recognition based on the compensated first image region more accurate.

[0047] Non-uniform compensation operations include scale magnification and contrast magnification. Scale magnification refers to interpolating and enlarging the image region (such as bilinear interpolation) to increase the pixel density of small targets at a distance. Contrast magnification refers to enhancing local grayscale differences (such as CLAHE) to improve the texture discernibility of low-contrast, flattened areas. Compensation operations based on scale magnification and / or contrast magnification can be performed by dividing the first image region (also known as the road surface candidate region) vertically into 2-3 regions. For regions at greater distances, scale magnification and contrast magnification are applied. For example, if the first image region is vertically partitioned to obtain vertical region 1 (distance A from the target edge), vertical region 2 (distance B from the target edge), and vertical region 3 (distance C from the target edge), where A>B>C, then vertical region 1 can be simultaneously scaled and contrast-enhanced, vertical region 2 can be contrast-enhanced, and vertical region 3 can be left unprocessed or only slightly contrast-enhanced.

[0048] Finally, the circumscribed rectangle of the compensated first image region is stitched together and adjusted to the size required for the target detection model to perform detection of road anomalies and collision targets, resulting in the second image data. This embodiment significantly improves the sensitivity of recognizing distant road anomalies and obstacles, effectively mitigating the perception attenuation caused by low-angle perspective distortion.

[0049] Regarding step S206 above, in an optional embodiment, identifying road surface anomalies in the driving direction based on the second image data includes: inputting the second image data into a first target detection model, and obtaining at least one candidate road surface anomaly output by the first target detection model, and the longitudinal position corresponding to each candidate road surface anomaly; determining a first confidence level for each candidate road surface anomaly according to the longitudinal region where each candidate road surface anomaly is located, wherein the road surface mapping region in the first image data is divided into multiple longitudinal regions, and the longitudinal region where each candidate road surface anomaly is located is the longitudinal region to which the longitudinal position corresponding to each candidate road surface anomaly belongs among the multiple longitudinal regions; identifying candidate road surface anomalies whose first confidence level is higher than the confidence threshold corresponding to the longitudinal region as the road surface anomaly, wherein one of the multiple longitudinal regions corresponds to a confidence threshold.

[0050] In this embodiment, second image data is input into a first target detection model. The first target detection model can output road anomaly information for at least one candidate road surface anomaly detected from the second image data. This road surface anomaly information includes at least: a detection box, a first confidence level, and a type for each candidate road surface anomaly. The second image data includes a compensated first image region. The compensation operation for the first image region is a non-uniform compensation performed after vertically partitioning the first image region. In this embodiment, the first confidence level of each candidate road surface anomaly is determined based on the multiple vertical regions divided in the compensation operation. Specifically, the vertical region to which the vertical position corresponding to each candidate road surface anomaly belongs in the multiple vertical regions is determined, and candidate road surface anomalies with a first confidence level higher than the confidence threshold corresponding to their respective vertical region are identified as road surface anomalies.

[0051] This embodiment can significantly improve the detection rate of weak targets at long distances and reduce false alarms in the near area by identifying road surface anomalies from the second image data obtained based on non-uniform compensation and by determining the confidence threshold based on longitudinal partitioning.

[0052] Optionally, the plurality of longitudinal regions include a first longitudinal region and a second longitudinal region, wherein the distance of the first longitudinal region relative to the target edge of the first image data is less than the distance of the second longitudinal region relative to the target edge; determining the first confidence level of each candidate road surface anomaly according to the longitudinal region where each candidate road surface anomaly is located includes: performing the following determination operation on each candidate road surface anomaly as a current candidate road surface anomaly to obtain the first confidence level of each candidate road surface anomaly: when the longitudinal region where the current candidate road surface anomaly is located is the first longitudinal region, determining the second confidence level output by the first target detection model for the current candidate road surface anomaly as the first confidence level of the current candidate road surface anomaly; when the longitudinal region where the current candidate road surface anomaly is located is the second longitudinal region, determining the cumulative confidence level of the current candidate road surface anomaly in the image data sequence as the first confidence level of the current candidate road surface anomaly; wherein, the image data sequence is obtained by sequentially performing the region compensation operation on each image data in the two-wheeled vehicle's driving direction, and the image data sequence includes the second image data.

[0053] In other words, the target edge is the bottom edge of both the first and second image data. The distance from each of the multiple vertical regions to the target edge in the first image data varies. For vertical regions closer to the target edge, such as the first vertical region, the second confidence score output by the first target detection model for the current candidate road surface anomaly located within that region can be used as the first confidence score. For vertical regions farther from the target edge, such as the second vertical region, the first confidence score for the current candidate road surface anomaly located within that region needs to be determined based on the cumulative confidence score of the current candidate road surface anomaly in the image data sequence. Specifically, a time weight is assigned to the confidence score of the current candidate road surface anomaly in each image data in the image data sequence. The cumulative confidence score is obtained by weighted summation of the multiple confidence scores of the current candidate road surface anomaly in the image data sequence based on the time weight.

[0054] This embodiment ensures the real-time and accuracy of road surface anomalies detected in the near longitudinal region. Meanwhile, for the far longitudinal region, the effects of perspective compression and low signal-to-noise ratio are overcome by using time-series accumulation, which improves the robustness of detecting road surface anomalies at a distance.

[0055] In an optional embodiment, the road surface anomaly detection process is as follows: Figure 3 As shown, it includes:

[0056] Step S31: Use the trained first object detection model to detect candidate road surface anomalies in the second image data. Road surface anomalies include: bumps, potholes, manhole covers, cracks, etc.

[0057] Step S32: Dynamically correct the detection confidence based on the longitudinal location of candidate road surface anomalies, and determine the road surface anomalies among the candidate anomalies based on the corrected confidence:

[0058] Based on the longitudinal position of each candidate road surface anomaly in the second image data, its corresponding imaging region is determined, and different confidence correction strategies are set for different longitudinal regions. Specifically, this includes:

[0059] Based on the longitudinal position of each candidate road surface anomaly in the second image data, the road surface is divided into three regions: far-distance, medium-distance, and near-distance (the far-distance region is equivalent to the second longitudinal region in the above embodiment, and the medium-distance and near-distance regions are equivalent to the first longitudinal region in the above embodiment). The near-distance region is directly determined based on the high confidence of a single frame, the confidence threshold of the medium-distance region is appropriately reduced, such as by 10%, and the far-distance region, based on the reduced threshold, introduces a multi-frame temporal consistency mechanism. By performing time-weighted accumulation of the detection confidence of multiple consecutive frames (the weighted accumulation result is equivalent to the cumulative confidence in the above embodiment), stable identification of road surface anomalies with weak features at a distance is achieved.

[0060] The specific method for time-weighted cumulative calculation is as follows:

[0061] Let the height of the second image data be H, the longitudinal position of each candidate road surface anomaly be y, and the normalized position be defined as . The confidence threshold corresponding to the near-field region is defined as follows: The confidence threshold corresponding to the mid-range region is The confidence threshold for distant regions is .in .

[0062] For the near-field region, the confidence level is greater than The candidate road surface anomalies are road surface anomalies; for the mid-range region, the confidence level is greater than [value missing]. The candidate road surface anomalies are road surface anomalies;

[0063] For distant regions, calculate the cumulative confidence score across multiple frames within a time window of N frames (equivalent to the image data sequence in the above embodiment): .in Let be the detection confidence score for the i-th frame. As time weight, ,and The confidence level increases over time (as the candidate road surface anomaly approaches). If some frames within the time window N do not detect a candidate road surface anomaly, their corresponding... =0, but since it's a summation of confidence scores for N frames, it allows for a single frame to go undetected. When S≥ At that time, the road surface was determined to be abnormal.

[0064] In this optional embodiment, since the electric scooter's speed is lower than that of a car, it has more reaction time to road anomalies. Therefore, it is permissible to acquire several consecutive frames of road images for forward vision and then make a comprehensive judgment. The corrected anomaly detection results in consecutive frames are correlated to determine the temporal consistency or evolution trend of candidate road anomalies. In the implementation process, for targets that are not obvious or difficult to detect in distant areas, a low threshold and multi-frame detection method is used. Temporal consistency is used to accumulate confidence, so that when the cumulative confidence of a distant candidate road anomaly in N consecutive frames exceeds a certain low threshold, it is judged that there is a distant road anomaly.

[0065] While identifying road surface anomalies, the step of identifying the collision target of the two-wheeled vehicle in the driving direction based on the second image data includes: inputting the second image data into a second target detection model and obtaining at least one candidate collision target output by the second target detection model; determining the candidate collision target whose detection box center point is located within the second image area as the collision target among the at least one candidate collision target, wherein the second image area is: the mapping area of ​​the driving channel area within the road surface area in the driving direction within the second image data.

[0066] The second image data is input into the second target detection model, which outputs obstacle information identifying at least one candidate collision target from the second image data. This obstacle information includes at least one detection box for each candidate collision target. A travel lane area exists within the road surface region along the two-wheeled vehicle's direction of travel; this travel lane area is mapped to the second image data as the second image region. Optionally, the first image region includes the second image region; that is, the travel lane area of ​​the two-wheeled vehicle is often within the two-wheeled vehicle's region of interest on the road surface. In this embodiment, candidate collision targets whose detection box centers are within the second image region are considered collision targets, posing a potential collision risk to the two-wheeled vehicle.

[0067] In this embodiment, only candidate collision targets within the driving lane area are considered collision targets, thereby filtering out irrelevant objects (such as shoulders, guardrails, and pedestrians) on both sides of the two-wheeled vehicle's driving lane in the road. This allows for the identification of collision targets that may intersect with the two-wheeled vehicle's path. The above embodiment can effectively reduce false detections of obstacles and save operational resources.

[0068] In an optional embodiment, the second object detection model employs a lightweight deep learning object detection model (YOLOv5). Collision targets are also referred to as obstacles. Obstacle detection and initial screening include: using YOLOv5 to detect obstacles within 50m ahead, such as pedestrians, two-wheeled vehicles, and cars. The detection results are then filtered for confidence, and combined with the relative position of the target in the image, potential collision targets are identified based on whether the center point of the detection box is located within a predefined driving lane area. Optionally, the first object detection model can also employ a lightweight deep learning object detection model (YOLOv5).

[0069] In the above embodiments, after identifying road surface anomalies, a first distance relative to the two-wheeled vehicle can be detected using a preset ranging method. Similarly, after identifying a collision, a second distance relative to the two-wheeled vehicle can also be detected using a preset ranging method. Specifically, the preset ranging method is a monocular vision ranging method, which uses a monocular vision ranging method based on the principle of similar triangles to estimate the actual height or width of the detected target. The distance is then estimated using the height or width of the detection frame and the calibrated camera intrinsics (focal length).

[0070] Specifically, determining the first distance and the second distance includes:

[0071] First, obtain the detection boxes of road surface anomalies detected in the second image data by the YOLOv5 model, extract the width or height of the detection boxes, and combine them with the physical dimensions of known targets (road surface anomalies or obstacles) (such as the average height of pedestrians, the width of vehicles, the width of manhole covers, etc.) as a benchmark.

[0072] Secondly, monocular ranging calculation is performed. Based on the principle of similar triangles, the initial distance of the target is calculated using the width and height of the detection frame, the physical size, and the calibrated camera focal length. For targets at a distance or whose detection frame is truncated, the ranging dimension is dynamically switched, such as from height to width, as the ranging basis to improve stability.

[0073] Finally, multi-frame temporal filtering is performed, and the ranging results from multiple consecutive frames are input into a Kalman filter to fuse the motion model and observation noise, suppressing instantaneous jumps caused by scooter vibration or detection jitter, and outputting either the first or second distance. This ranging method can eliminate abrupt changes and output stable target distance estimates, thus solving the detection box jitter problem caused by scooter vibration.

[0074] Optionally, the first distance and the road surface anomaly information detected by the first target detection model can be used to calculate the bump risk caused by the road surface anomaly. Determining the bump risk of the two-wheeled vehicle based on the first state data includes: determining the initial bump risk of the two-wheeled vehicle by the confidence level of the identified road surface anomaly, the area coefficient corresponding to the road surface anomaly, and the type coefficient corresponding to the road surface anomaly. The area coefficient indicates the proportional relationship between the detection frame parameters of the road surface anomaly and the first distance; the type coefficient is determined by the type of the road surface anomaly; the first distance is the distance of the road surface anomaly relative to the two-wheeled vehicle, determined by the detection frame parameters of the road surface anomaly; the confidence level of the road surface anomaly, the area coefficient corresponding to the road surface anomaly, and the type coefficient corresponding to the road surface anomaly are used to calculate the bump risk. The types of road surface anomalies are determined using a first target detection model and the second image data. The initial bump risk is corrected using correction coefficients to obtain the bump risk of the road surface anomaly to the two-wheeled vehicle. The correction coefficients include at least one of the following: a first coefficient corresponding to the first state data; a second coefficient corresponding to the offset, where the offset is the offset of the position of the road surface anomaly mapped within the second image data relative to a second image region, where the second image region is the mapped region of the driving channel area within the road surface area in the driving direction within the second image data; and a third coefficient corresponding to the bump state of the vehicle traveling ahead of the two-wheeled vehicle.

[0075] The road anomaly information includes: the detection frame and parameters of the road anomaly, the type of road anomaly, and the confidence level of the road anomaly. The detection frame parameters include: the area of ​​the detection frame. In this embodiment, the ratio of the detection frame area to a first distance is used as the area coefficient. A type coefficient is assigned to the road anomaly according to its type; for example, a larger type coefficient is assigned to potholes and missing manhole covers, while a smaller type coefficient is assigned to speed bumps.

[0076] The initial bump risk is calculated by multiplying the confidence level of the road surface anomaly, the area coefficient corresponding to the road surface anomaly, and the type coefficient. The initial bump risk is then multiplied by the correction coefficient, and the resulting product is the bump risk of the road surface anomaly to the two-wheeled vehicle.

[0077] Optionally, road surface anomalies such as potholes and missing manhole covers can be classified with a type coefficient of [1.5, 3.0] (high risk); road surface anomalies such as cracks and small bumps can be classified with a type coefficient of [0.5, 1.2] (medium to low risk); and road surface anomalies such as speed bumps can be classified with a type coefficient of [0.3, 0.8] (low risk, as they are predictable and vehicles can absorb them). The type coefficient is determined based on the severity of bumps statistically analyzed from offline labeled samples, assigning higher weights to high-risk road surface anomalies.

[0078] Optionally, the first coefficient can be determined from the travel speed in the first state data. For low travel speeds (v ≤ 8 km / h), the first coefficient can be 1.0; for medium travel speeds (8 < v ≤ 15 km / h), the first coefficient can be 1.2 ~ 1.6; and for high travel speeds (v > 15 km / h), the first coefficient can be 1.8 ~ 2.5. The higher the speed, the more dangerous the bumps and impacts, so the correction coefficient increases incrementally, with the upper limit constrained by the structural safety threshold of the scooter.

[0079] Optionally, for the second coefficient, if the offset is ≤0.2 × the width of the second image region, the second coefficient is 1.0 (directly facing the channel); if -0.2 < offset ≤0.5 × the width of the second image region, the second coefficient is 1.1 ~ 1.4 (edge ​​risk); if the offset > 0.5 × the width of the second image region, the second coefficient is (considered non-threat and not corrected). The smaller the offset, the closer the road anomaly is to the center of travel, the higher the risk, and the coefficient increases slightly; if it exceeds 50% of the channel, it is automatically considered an indirect impact, and the coefficient is reduced to 1.0 to avoid false alarms.

[0080] Optionally, for the third coefficient, if there is no preceding vehicle or no bump signal, the third coefficient is 1.0; if the preceding vehicle has slight vertical shaking (detection frame height fluctuation <5%), the third coefficient is 1.1 ~ 1.3; if the preceding vehicle has significant bumps (detection frame height fluctuation ≥5% and lasts ≥2 frames), the third coefficient is 1.5 ~ 2.0. Road conditions are indirectly inferred based on the preceding vehicle's motion state. The coefficient increment is set according to the intensity and duration of the shaking, serving as supplementary evidence and not used for independent decision-making, but only for cumulative correction.

[0081] This embodiment integrates confidence level, area ratio, type coefficient, and multiple maintenance positive coefficients to achieve dynamic and refined assessment of the risk of abnormal road surface bumps, thereby improving the accuracy and safety of early warning in complex scenarios.

[0082] In some optional embodiments, determining the collision risk of the collision target to the two-wheeled vehicle based on the first state data and the second distance includes: determining the estimated arrival time of the two-wheeled vehicle to the collision target by using the identified second distance of the collision target relative to the two-wheeled vehicle and the first state data, wherein the second distance is determined by the detection bounding box parameters of the collision target; the detection bounding box parameters of the collision target are determined by the second target detection model and the second image data; and determining the collision risk by the estimated arrival time, wherein the collision risk is negatively correlated with the estimated arrival time.

[0083] The first state data includes the speed of the two-wheeled vehicle, and the ratio of the second distance to the speed is used as the estimated arrival time. Since the collision risk is negatively correlated with the estimated arrival time, determining the collision risk using the estimated arrival time includes: using the estimated arrival time as the collision risk, and subsequently classifying the collision risk using multi-level time thresholds based on the principle that the collision risk is negatively correlated with the estimated arrival time.

[0084] This embodiment transforms spatial distance into a time safety window, which better aligns with the reaction logic of electric scooter users or the animated version of the system, significantly improving the real-time performance and proactivity of obstacle avoidance decisions.

[0085] After determining the bump risk and collision risk, the step of determining the driving risk of the two-wheeled vehicle based on the bump risk and collision risk includes: determining a target collision risk threshold that matches the collision risk from a preset set of collision risk thresholds, and determining a first risk level corresponding to the target collision risk threshold, wherein one collision risk threshold in the set of collision risk thresholds corresponds to one collision risk level; determining a target bump risk threshold that matches the bump risk from a preset set of bump risk thresholds, and determining a second risk level corresponding to the target bump risk threshold, wherein one bump risk threshold in the set of bump risk thresholds corresponds to one bump risk level; and determining the driving risk of the two-wheeled vehicle through the first risk level and the second risk level.

[0086] In an optional embodiment, a preset set of collision risk thresholds includes a first collision risk threshold and a second collision risk threshold, wherein the first risk threshold is greater than the second risk threshold. When the collision risk is greater than the first collision risk threshold, the first risk level is high collision risk, requiring the electric scooter to slow down and a high-risk warning to be issued. When the collision risk is between the first and second collision risk thresholds, the first risk level is medium collision risk, requiring a collision deceleration warning, and different levels of bump warnings (visual signals) to be issued based on the level of bump risk. When the collision risk is less than the second collision risk threshold, the first risk level is low collision risk, and different degrees of visual bump warnings can be issued based solely on the second risk level of bump risk.

[0087] For the risk classification of turbulence risk, a preset set of turbulence risk thresholds includes a first turbulence risk threshold and a second turbulence risk threshold, where the first turbulence risk threshold is greater than the second turbulence risk threshold. When the turbulence risk is greater than the first turbulence risk threshold, the second risk level is high turbulence risk; when the turbulence risk is between the first and second turbulence risk thresholds, the second risk level is medium turbulence risk; and when the turbulence risk is less than the second turbulence risk threshold, the second risk level is low turbulence risk.

[0088] Based on the above, the driving risk of the two-wheeled vehicle is determined by the first risk level and the second risk level. The driving risk specifically includes one of the following: high collision risk; a second risk level of medium collision risk and bump risk; and a second risk level of bump risk.

[0089] This embodiment achieves layered and adaptive driving risk response through a decision-making mechanism that combines collision and bump risk levels: high collision risk directly triggers forced deceleration, medium collision risk is linked to bump level for graded visual prompts, and low collision risk only provides a light warning based on the bump condition, significantly improving the accuracy of the system's driving risk warning in complex road conditions.

[0090] In an optional embodiment, the above-described process for determining driving risks further includes:

[0091] For road surface abnormalities: the risk of bumps and jolts associated with road surface abnormalities. =Confidence level S Area index a Type coefficient w; where a represents the detection box area for road surface anomalies at a fixed distance. The value 'w' corresponds to different types of road surface anomalies. For potholes and missing manhole covers, 'w' is assigned a larger value; for speed bumps, 'w' is assigned a smaller value.

[0092] Optional, corresponding to the risk of bumps. Further corrections can be made using: spatial correlation (the offset between road surface anomalies and the expected travel area / path), the electric scooter's speed (low / medium / high speed), the current vibration level of the scooter (i.e., the scooter's bumpy state), and indirect road condition judgments based on the bumpy state of the vehicle ahead. The specific correction method involves... Multiply by the corresponding correction factor. For example, when the speed exceeds a certain threshold, the correction factor is greater than 1.

[0093] Indirect road condition assessment based on the preceding vehicle's bumpy condition includes: continuously tracking moving targets (mainly electric two-wheelers / scooters) in the same lane ahead while the electric scooter is in motion. Based on its motion state prediction model, the deviation characteristics between the actual observation results and the predicted results are analyzed. If the height of the preceding vehicle's detection frame remains relatively constant, but its position exhibits sudden up-and-down jitters relative to the tracker's predicted position in certain frames, then it is determined that there may be bumps ahead, and the indirect road condition assessment result for the preceding vehicle's bumpy condition is that bumps exist. It should be noted that indirect road condition assessment based on the preceding vehicle's bumpy condition is susceptible to noise interference and should only be used as an auxiliary assessment criterion.

[0094] For obstacles: Based on the scooter's travel data (such as the speed of an electric scooter) and the distance to the detected collision target, calculate the estimated time of arrival (TTC) and collision risk for each collision target. Represented by ttc.

[0095] Risk of bumps and collision risk The risk level is divided into three categories based on the value (using a threshold). , , , (Distinguish), when When this happens, force the scooter to slow down and give a high-risk warning; when At the same time, issue a collision deceleration warning and adjust the warning based on the risk of bumps. The levels of turbulence are used to issue different levels of turbulence warnings (visual signals); when At that time, based on the risk of bumps The system will issue visual turbulence warnings at different levels.

[0096] Optionally, after obtaining the bump risk level and collision risk level, the assessment results of the driving risk (collision hazard level, road surface anomaly risk level) are output to the instrument panel or the front display screen. The scooter's front display screen will illuminate icons corresponding to bumps and obstacles such as vehicles, pedestrians, and speed bumps to alert the rider of potential collision or bump risks ahead. This allows for timely slowing down and, based on the scooter's own controller and shock absorption capabilities, prepares the user for road bumps from their perspective.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0098] According to another aspect of the embodiments of this application, an electric vehicle is also provided, which can be used to implement the driving risk warning method provided in the above embodiments, and will not be repeated hereafter. As used below, the terms "unit" and "module" are equivalent to a combination of software and / or hardware that can perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0099] Figure 4 This is a structural block diagram of an optional driving risk warning device according to an embodiment of this application, such as... Figure 4 As shown, the electric vehicle includes:

[0100] The acquisition module 42 is used to acquire first image data in the driving direction of the two-wheeled vehicle and first state data of the two-wheeled vehicle when the two-wheeled vehicle is in a driving state, wherein the first image data and the first state data are time-aligned.

[0101] Compensation module 44 is used to perform a region compensation operation on the first image data to obtain second image data. The region compensation operation is used to perform non-uniform compensation on the road surface mapping region within an image data. The road surface mapping region is the mapping region of the road surface area in the driving direction within the image data.

[0102] The risk determination module 46 is used to identify road surface anomalies in the driving direction based on the second image data, identify collision targets of the two-wheeled vehicle in the driving direction based on the second image data, and determine the bump risk of the road surface anomalies to the two-wheeled vehicle and the collision risk of the collision targets to the two-wheeled vehicle based on the first state data.

[0103] The risk warning module 48 is used to determine the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk, and to provide risk warnings based on the driving risk.

[0104] According to the embodiments provided in this application, when a two-wheeled vehicle is in motion, first image data in the direction of travel of the two-wheeled vehicle and first state data of the two-wheeled vehicle that are time-aligned with the first image data are acquired. Then, a region compensation operation is performed on the first image data to obtain second image data. The region compensation operation is used to perform non-uniform compensation on the road surface mapping area within one image data. The road surface mapping area is the mapping area of ​​the road surface area in the direction of travel within the one image data. Furthermore, based on the second image data, road surface anomalies in the direction of travel are identified. Based on the second image data, collision targets of the two-wheeled vehicle in the direction of travel are identified. Based on the first state data, the bump risk of the road surface anomaly to the two-wheeled vehicle and the collision risk of the collision target to the two-wheeled vehicle are determined respectively. Based on the bump risk and the collision risk, the driving risk of the two-wheeled vehicle is determined, and a risk warning is given based on the driving risk. By adopting the above scheme, road surface anomalies and collision targets can be identified from the non-uniformly compensated image data. Combined with the driving status data of the two-wheeled vehicle that is time-aligned with the image data, the bump risk and collision risk in the driving direction of the two-wheeled vehicle can be accurately determined and a risk warning can be issued. This solves the problem of inaccurate warnings of driving risks during the driving process of two-wheeled vehicles in related technologies.

[0105] In some exemplary embodiments, the compensation module 44 is further configured to cut out a first image region from the first image data according to a preset shape, wherein the first image region is the road surface mapping region within the first image data; to perform vertical partitioning on the first image region, and to perform a compensation operation on each vertical region according to the distance of each obtained vertical region relative to the target edge of the first image data, thereby obtaining a compensated first image region, wherein the compensation operation includes at least one of the following: scale magnification, contrast magnification; and to fill the compensated first image region with an outer rectangle to obtain the second image data.

[0106] In some exemplary embodiments, such as Figure 5As shown, the risk determination module 46 further includes a first identification unit 462, used to input the second image data into a first target detection model, and obtain at least one candidate road surface anomaly output by the first target detection model, and the longitudinal position corresponding to each candidate road surface anomaly among the at least one candidate road surface anomaly; determine a first confidence level of each candidate road surface anomaly according to the longitudinal region where each candidate road surface anomaly is located, wherein the road surface mapping region in the first image data is divided into multiple longitudinal regions, and the longitudinal region where each candidate road surface anomaly is located is the longitudinal region to which the longitudinal position corresponding to each candidate road surface anomaly belongs among the multiple longitudinal regions; and determine the candidate road surface anomaly as the road surface anomaly if the first confidence level is higher than the confidence level threshold corresponding to the longitudinal region where it is located, wherein one of the multiple longitudinal regions corresponds to a confidence level threshold.

[0107] In some exemplary embodiments, the plurality of longitudinal regions include a first longitudinal region and a second longitudinal region, wherein the distance of the first longitudinal region relative to the target edge of the first image data is less than the distance of the second longitudinal region relative to the target edge; the first identification unit 462 is further configured to perform the following determination operation on each candidate road surface anomaly as a current candidate road surface anomaly to obtain a first confidence level of each candidate road surface anomaly: when the longitudinal region where the current candidate road surface anomaly is located is the first longitudinal region, the second confidence level output by the first target detection model for the current candidate road surface anomaly is determined as the first confidence level of the current candidate road surface anomaly; when the longitudinal region where the current candidate road surface anomaly is located is the second longitudinal region, the cumulative confidence level of the current candidate road surface anomaly in the image data sequence is determined as the first confidence level of the current candidate road surface anomaly; wherein, the image data sequence is obtained by sequentially performing the region compensation operation on each image data in the two-wheeled vehicle's driving direction, and the image data sequence includes the second image data.

[0108] In some exemplary embodiments, such as Figure 5 As shown, the risk determination module 46 further includes a second identification unit 464, which is used to input the second image data into the second target detection model and obtain at least one candidate collision target output by the second target detection model; and to determine the candidate collision target whose detection box center point is located in the second image area among the at least one candidate collision target as the collision target, wherein the second image area is: the mapping area of ​​the driving channel area in the road surface area in the driving direction in the second image data.

[0109] In some exemplary embodiments, the risk determination module 46 is further configured to determine the initial bump risk of the road surface anomaly to the two-wheeled vehicle by using the confidence level of the identified road surface anomaly, the area coefficient corresponding to the road surface anomaly, and the type coefficient corresponding to the road surface anomaly. The area coefficient is used to indicate the proportional relationship between the detection frame parameter of the road surface anomaly and a first distance; the type coefficient is determined by the type of the road surface anomaly; the first distance is the distance of the road surface anomaly relative to the two-wheeled vehicle, determined by the detection frame parameter of the road surface anomaly; and the confidence level of the road surface anomaly, the detection frame parameter of the road surface anomaly, and the type coefficient of the road surface anomaly are further defined. The model is determined by the first target detection model and the second image data; the initial bump risk is corrected by a correction coefficient to obtain the bump risk of the road surface anomaly to the two-wheeled vehicle; wherein, the correction coefficient includes at least one of the following: a first coefficient corresponding to the first state data; a second coefficient corresponding to the offset, wherein the offset is the offset of the position of the road surface anomaly mapped in the second image data relative to the second image region, the second image region being: the mapping region of the driving channel region in the road surface region in the driving direction in the second image data; and a third coefficient corresponding to the bump state of the vehicle traveling in front of the two-wheeled vehicle.

[0110] In some exemplary embodiments, the risk determination module 46 is further configured to determine the estimated arrival time of the two-wheeled vehicle to the collision target by using the identified second distance of the collision target relative to the two-wheeled vehicle and the first state data, wherein the second distance is determined by the detection frame parameters of the collision target; the detection frame parameters of the collision target are determined by the second target detection model and the second image data; and the collision risk is determined by the estimated arrival time, wherein the collision risk is negatively correlated with the estimated arrival time.

[0111] In some exemplary embodiments, the risk warning module 48 is further configured to determine a target collision risk threshold that matches the collision risk from a preset set of collision risk thresholds, and determine a first risk level corresponding to the target collision risk threshold, wherein one collision risk threshold in the set of collision risk thresholds corresponds to one collision risk level; determine a target bump risk threshold that matches the bump risk from a preset set of bump risk thresholds, and determine a second risk level corresponding to the target bump risk threshold, wherein one bump risk threshold in the set of bump risk thresholds corresponds to one bump risk level; and determine the driving risk of the two-wheeled vehicle through the first risk level and the second risk level.

[0112] In some exemplary embodiments, the acquisition module 42 is further configured to acquire third image data through an image acquisition component, wherein the image acquisition component is deployed on the two-wheeled vehicle; acquire second state data through a display component or control component of the two-wheeled vehicle; and perform timestamp alignment on the third image data and the second state data to obtain the first image data and the first state data.

[0113] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0114] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for alerting driving risks, characterized in that, include: When the two-wheeled vehicle is in motion, first image data in the direction of travel of the two-wheeled vehicle and first state data of the two-wheeled vehicle are acquired, wherein the first image data and the first state data are time-aligned. A region compensation operation is performed on the first image data to obtain the second image data. The region compensation operation is used to perform non-uniform compensation on the road surface mapping region within one image data. The road surface mapping region is the mapping region of the road surface area in the driving direction within the one image data. Based on the second image data, road surface anomalies in the driving direction are identified; based on the second image data, collision targets of the two-wheeled vehicle in the driving direction are identified; and based on the first state data, the bump risk of the road surface anomalies to the two-wheeled vehicle and the collision risk of the collision targets to the two-wheeled vehicle are determined respectively. Based on the bump risk and the collision risk, the driving risk of the two-wheeled vehicle is determined, and a risk warning is given based on the driving risk.

2. The method according to claim 1, characterized in that, The step of performing region compensation on the first image data to obtain the second image data includes: A first image region is cut out from the first image data according to a preset shape, wherein the first image region is the road surface mapping region within the first image data; The first image region is divided into vertical partitions, and a compensation operation is performed on each vertical partition according to the distance of each obtained vertical partition relative to the target edge of the first image data to obtain the compensated first image region. The compensation operation includes at least one of the following: scale magnification and contrast magnification. The second image data is obtained by filling the bounding rectangle of the compensated first image region.

3. The method according to claim 1, characterized in that, The step of identifying road surface anomalies in the driving direction based on the second image data includes: The second image data is input into the first target detection model, and at least one candidate road surface anomaly output by the first target detection model is obtained, as well as the longitudinal position corresponding to each candidate road surface anomaly in the at least one candidate road surface anomaly. According to the longitudinal region where each candidate road surface anomaly is located, a first confidence level of each candidate road surface anomaly is determined, wherein the road surface mapping region in the first image data is divided into multiple longitudinal regions, and the longitudinal region where each candidate road surface anomaly is located is the longitudinal region to which the longitudinal position corresponding to each candidate road surface anomaly belongs among the multiple longitudinal regions. Candidate road surface anomalies whose first confidence level is higher than the confidence threshold corresponding to their respective longitudinal regions are identified as road surface anomalies, wherein one of the multiple longitudinal regions corresponds to a confidence threshold.

4. The method according to claim 3, characterized in that, The plurality of vertical regions include a first vertical region and a second vertical region, wherein the distance of the first vertical region relative to the target edge of the first image data is less than the distance of the second vertical region relative to the target edge; The step of determining the first confidence level of each candidate road surface anomaly according to the longitudinal region where each candidate road surface anomaly is located includes: For each candidate road surface anomaly, the following determination operation is performed as the current candidate road surface anomaly to obtain the first confidence level of each candidate road surface anomaly: When the longitudinal region where the current candidate road surface anomaly is located is the first longitudinal region, the second confidence level output by the first target detection model for the current candidate road surface anomaly is determined as the first confidence level of the current candidate road surface anomaly; If the longitudinal region where the current candidate road surface anomaly is located is the second longitudinal region, the cumulative confidence of the current candidate road surface anomaly in the image data sequence is determined as the first confidence of the current candidate road surface anomaly. The image data sequence is obtained by sequentially performing the region compensation operation on each image data in the driving direction of the two-wheeled vehicle, and the image data sequence includes the second image data.

5. The method according to claim 1, characterized in that, The step of identifying the collision target of the two-wheeled vehicle in the direction of travel based on the second image data includes: The second image data is input into the second target detection model, and at least one candidate collision target is obtained from the output of the second target detection model; Among the at least one candidate collision targets, the candidate collision target whose detection box center point is located within the second image region is determined as the collision target, wherein the second image region is the mapping region of the driving channel region within the road surface region in the driving direction within the second image data.

6. The method according to claim 1, characterized in that, The step of determining the risk of bumpiness to the two-wheeled vehicle due to the road surface anomaly based on the first state data includes: The initial bump risk of the road surface anomaly to the two-wheeled vehicle is determined by the confidence level of the identified road surface anomaly, the area coefficient corresponding to the road surface anomaly, and the type coefficient corresponding to the road surface anomaly. The area coefficient indicates the proportional relationship between the detection box parameters of the road surface anomaly and a first distance. The type coefficient is determined by the type of the road surface anomaly. The first distance is the distance of the road surface anomaly relative to the two-wheeled vehicle, determined by the detection box parameters of the road surface anomaly. The confidence level, detection box parameters, and type of the road surface anomaly are all determined by a first target detection model and the second image data. The initial bump risk is corrected by a correction factor to obtain the bump risk of the two-wheeled vehicle caused by the road surface anomaly. The correction coefficient includes at least one of the following: The first coefficient corresponding to the first state data; The second coefficient corresponding to the offset, wherein the offset is the offset of the position of the road surface anomaly mapped in the second image data relative to the second image region, and the second image region is: the mapping region of the driving channel region in the road surface region in the driving direction in the second image data; The third coefficient corresponds to the bumpy state of the vehicle traveling in front of the two-wheeled vehicle.

7. The method according to claim 1, characterized in that, The step of determining the collision risk of the collision target to the two-wheeled vehicle based on the first state data includes: The estimated arrival time of the two-wheeled vehicle to the collision target is determined by the second distance of the identified collision target relative to the two-wheeled vehicle and the first state data, wherein the second distance is determined by the detection box parameters of the collision target; the detection box parameters of the collision target are determined by the second target detection model and the second image data. The collision risk is determined by the estimated arrival time, wherein the collision risk is negatively correlated with the estimated arrival time.

8. The method according to claim 1, characterized in that, Determining the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk includes: A target collision risk threshold that matches the collision risk is determined from a preset set of collision risk thresholds, and a first risk level corresponding to the target collision risk threshold is determined, wherein one of the collision risk thresholds in the set of collision risk thresholds corresponds to one collision risk level; A target turbulence risk threshold that matches the turbulence risk is determined from a preset set of turbulence risk thresholds, and a second risk level corresponding to the target turbulence risk threshold is determined, wherein one turbulence risk threshold in the set of turbulence risk thresholds corresponds to one turbulence risk level. The driving risk of the two-wheeled vehicle is determined by the first risk level and the second risk level.

9. The method according to claim 1, characterized in that, The acquisition of the first image data of the two-wheeled vehicle's driving direction and the first state data of the two-wheeled vehicle includes: A third image data is acquired by an image acquisition component, wherein the image acquisition component is deployed on the two-wheeled vehicle; The second state data is obtained through the display component or control component of the two-wheeled vehicle; The third image data and the second state data are timestamped to obtain the first image data and the first state data.

10. A driving risk warning device, characterized in that, include: The acquisition module is used to acquire first image data in the driving direction of the two-wheeled vehicle and first state data of the two-wheeled vehicle when the two-wheeled vehicle is in a driving state, wherein the first image data and the first state data are time-aligned. The compensation module is used to perform a region compensation operation on the first image data to obtain the second image data. The region compensation operation is used to perform non-uniform compensation on the road surface mapping area within one image data. The road surface mapping area is the mapping area of ​​the road surface area in the driving direction within the one image data. The risk determination module is used to identify road surface anomalies in the driving direction based on the second image data, identify collision targets of the two-wheeled vehicle in the driving direction based on the second image data, and determine the bump risk of the road surface anomalies to the two-wheeled vehicle and the collision risk of the collision targets to the two-wheeled vehicle based on the first state data. The risk warning module is used to determine the driving risk of the two-wheeled vehicle based on the bump risk and the collision risk, and to provide risk warnings based on the driving risk.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 9.

Images