Road surface perception method and system based on multi-source data fusion

Abstract

Embodiments of the present specification relate to the field of information technology, in particular to a road surface perception method and system based on multi-source data fusion. The method comprises the steps of: dividing the perception area of the road, setting the preparation area in front of the perception area; receiving multi-source data, the multi-source data including multiple types of roadside camera data, laser radar data and RSU data; identifying a plurality of attribute information of the vehicle in the preparation area through the roadside camera data and the RSU data, the attribute information at least including the license plate number; distinguishing the vehicle in the perception area according to the attribute information, tracking the position information of the vehicle in the perception area, and obtaining the driving event of the vehicle according to the position information; establishing the driving behavior characteristics of the vehicle according to the historical driving event of the vehicle corresponding to the license plate number; and predicting the probability of driving accidents and the type of accidents in the current perception area according to the response of the driving behavior characteristics to the pre-configured driving accident prediction model.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the field of information technology, in particular to a road surface perception method and system based on multi-source data fusion. BACKGROUND

[0002] With the continuous development of intelligent transportation systems, improving road safety and traffic efficiency has become a research focus. Traditional road monitoring systems often rely on a single data source, such as roadside cameras or radar devices. These methods face many challenges in complex urban environments, such as visual obstacles, adverse weather conditions, and insufficient response to dynamically changing traffic conditions. In order to more accurately perceive road conditions and predict potential traffic accidents, multi-source data fusion technology has gradually received attention. However, how to efficiently integrate information from multiple data sources and use this information for real-time road perception and accident prediction is currently lacking in corresponding technology. SUMMARY

[0003] Embodiments of the present specification describe a road surface perception method and system based on multi-source data fusion.

[0004] In a first aspect, the embodiments of the present specification provide a road surface perception method based on multi-source data fusion, comprising the steps of:

[0005] Dividing the perception area of the road, setting a preparation area in front of the perception area;

[0006] Receiving multi-source data covering the preparation area and the perception area, the multi-source data including multiple types of roadside camera data, laser radar data and RSU data;

[0007] Identifying several attribute information of the vehicle in the preparation area through the roadside camera data and the RSU data, the attribute information at least including the license plate number;

[0008] According to the attribute information, distinguishing the vehicles in the perception area, and tracking the position information of the vehicle in the perception area through the laser radar data, obtaining the driving event of the vehicle according to the position information;

[0009] According to the historical driving event of the vehicle corresponding to the license plate number, establishing the driving behavior characteristics of the vehicle;

[0010] According to the response of the driving behavior characteristics to the pre-configured driving accident prediction model, predicting the probability and type of driving accidents occurring in the current perception area.

[0011] In a second aspect, the embodiments of the present specification provide a road surface perception system based on multi-source data fusion, comprising:

[0012] The division module divides a sensing area of the road surface, and a preparation area is arranged in front of the sensing area;

[0013] The receiving module receives multi-source data covering the preparation area and the sensing area, and the multi-source data includes multiple types of road-side camera data, laser radar data and RSU data;

[0014] The identification module identifies a plurality of attribute information of the vehicle in the preparation area through the road-side camera data and the RSU data, and the attribute information at least includes a license plate number;

[0015] The sensing module distinguishes the vehicle in the sensing area according to the attribute information, and tracks the position information of the vehicle in the sensing area through the laser radar data, and obtains a driving event of the vehicle according to the position information;

[0016] The feature module establishes a driving behavior feature of the vehicle according to the historical driving event of the vehicle corresponding to the license plate number;

[0017] The prediction module predicts a probability and a type of a driving accident occurring in the current sensing area according to a response of the driving behavior feature to a pre-configured driving accident prediction model.

[0018] In a third aspect, an electronic device is provided, including a processor and a memory;

[0019] The processor is connected with the memory;

[0020] The memory is used for storing executable program codes;

[0021] The processor runs a program corresponding to the executable program codes by reading the executable program codes stored in the memory, so as to execute the method in any of the above aspects.

[0022] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method in any of the above aspects.

[0023] The technical solutions provided by some embodiments of the present specification have at least the following beneficial effects:

[0024] In the embodiments of the present specification, a road surface perception method and system based on multi-source data fusion are provided. By integrating data from various sources such as roadside camera data, laser radar data, and RSU data, high-precision real-time monitoring and analysis of road conditions are achieved, effectively distinguishing between different vehicles and improving the accuracy of vehicle identification. By tracking the location information of vehicles in the perception area and combining historical driving events to establish a driving behavior feature model, driving habits are analyzed in depth, providing a data basis for personalized driving behavior evaluation. By predicting the probability and type of traffic accidents, this helps to take preventive measures in a timely manner and reduce the occurrence of traffic accidents.

[0025] Other features and advantages of the embodiments of the present specification will be further disclosed in the following specific embodiments and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present specification, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present specification, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0027] Figure 1 A road surface perception schematic diagram is provided for the embodiments of the present specification.

[0028] Figure 2 A road surface perception method flowchart is provided for the embodiments of the present specification.

[0029] Figure 3 A driving scheme schematic diagram is provided for the embodiments of the present specification.

[0030] Figure 4 A road surface perception system schematic diagram is provided for the embodiments of the present specification.

[0031] Figure 5 An electronic device schematic diagram is provided for the embodiments of the present specification. DETAILED DESCRIPTION

[0032] The technical solutions of the embodiments of the present specification will be explained and described below in combination with the drawings of the embodiments of the present specification. However, the following embodiments are only preferred embodiments of the present specification, not all. Based on the embodiments in the embodiments, other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present specification.

[0033] The terms "first", "second", "third", and the like in the description and claims of this specification and the above-mentioned drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0034] In the following description, the occurrence of terms such as "inner", "outer", "upper", "lower", "left", "right", etc. only for the convenience of describing the embodiments and simplifying the description, and not to indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present specification.

[0035] The data involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data comply with relevant laws, regulations and standards of relevant countries and regions.

[0036] Before introducing the technical solutions recorded in the present specification, the application scenarios and related technologies of the technical solutions are introduced.

[0037] With the increasing complexity of urban transportation systems and the rising demand for intelligentization, real-time and accurate perception of road conditions has become a key to ensuring traffic safety and efficiency. Road monitoring, as an important part of intelligent transportation systems, not only helps traffic management departments to timely grasp road conditions, but also provides safe driving suggestions for drivers, thereby effectively reducing traffic accidents and improving road capacity and service level.

[0038] Current road perception methods mainly rely on single sensors such as cameras 22 or geomagnetic sensors, etc. These methods have exposed problems such as limited coverage, poor environmental adaptability, and low data reliability in actual application. For example, cameras 22 cannot clearly capture vehicle information in adverse weather conditions (such as heavy rain and heavy fog); while geomagnetic sensors can accurately detect the presence of vehicles, but it is difficult to provide detailed information about vehicle type, speed and driving trajectory. Please refer to the accompanying drawings Figure 1The road monitoring technology provided in the present application gradually develops towards the direction of multi-source data fusion. By integrating the data of various sensors such as roadside cameras 22, laser radars, RSUs (roadside units), etc., not only can the monitoring coverage be expanded, but also the accuracy and environmental adaptability of the data can be significantly improved. The multi-source data fusion technology provided in the present application can realize omnidirectional and high-precision perception of the road surface conditions, realize vehicle recognition, driving trajectory tracking, and driving behavior analysis, etc. In addition, with the help of machine learning algorithms, potential traffic accidents can also be predicted, further enhancing road safety.

[0039] The present specification first provides a road surface perception method based on multi-source data fusion, please refer to the accompanying drawings Figure 2 , comprising the steps of:

[0040] Step S1) dividing the perception area 12 of the road, and setting a preparation area 11 in front of the perception area 12.

[0041] In the present application, the perception area 12 refers to the road area that needs to be monitored. For example, important intersections with heavy traffic, traffic complex sections prone to accidents, or sections with special monitoring needs, etc. can be selected. Multiple factors can be considered, such as road type (such as expressway, urban trunk road), traffic flow characteristics, historical traffic accident records, etc. By reasonably dividing the perception area 12, sensors (such as cameras 22, laser radars, RSU modules 12, etc.) can be deployed in a targeted manner to ensure coverage of all vehicles and their dynamics in the area.

[0042] The preparation area 11 is set in front of the perception area 12, and its main purpose is to collect vehicle information entering the perception area 12 in advance, so as to more accurately identify and distinguish these vehicles.

[0043] The design of the preparation area 11 should take into account the time interval between the vehicle entering the preparation area 11 and completely entering the perception area 12, which can be used for preliminary data collection and processing. For example, in the preparation area 11, the basic attribute information of the vehicle such as license plate number, color, length, and vehicle type, etc. can be obtained through roadside cameras 22 and RSU data.

[0044] The preparation area 11 does not need to be tracked in real time, and can use computationally intensive algorithms. For example, multiple attempts can be made to identify the license plate number. The stored related information of the vehicle corresponding to the license plate number can also be pre-read.

[0045] The perception zone 12 needs to track in real time, and it is not suitable to use high delay or to consume a large amount of computing power. Therefore, in the perception zone 12, the vehicle is mainly distinguished by identifying the color, length, model, etc. of the vehicle, combined with the information identified in the preparation zone 11, to quickly realize the differentiation of the vehicle and the association of the corresponding license plate number. For example, if the preparation zone 11 identifies that the color of the vehicle with license plate number A is yellow, and it is expected that the vehicle will be within the range of the perception zone 12, and there is no other yellow vehicle, then the vehicle can be distinguished from other vehicles only by color.

[0046] Step S2) receiving multi-source data covering the preparation zone 11 and the perception zone 12, the multi-source data including multiple types of roadside camera 22 data, laser radar data and RSU data.

[0047] The roadside camera 22 data can provide high-resolution visual information, which can be used to identify the color, length, model and license plate number of the vehicle. The driving track and dynamic behavior of the vehicle are monitored through the video stream. Laser radar data: using laser ranging technology to generate a three-dimensional point cloud map of the surrounding environment, which can accurately measure the position, speed and distance of the vehicle relative to other objects, and is used to track the position of the vehicle within the perception zone 12. RSU module 12, i.e. Road Side Unit, is one of the key components in the intelligent transportation system, mainly used to realize the communication between the vehicle and the infrastructure (V2I, Vehicle-to-Infrastructure). RSU is usually installed near the roadside or specific traffic facilities, such as signal lights, signs, etc. position, through wireless communication technology to exchange data with on-board unit (On-Board Unit, OBU). RSU can communicate with on-board unit (OBU) to obtain vehicle type, speed, direction, etc. information. It has a direct effect on enhancing the identification ability of specific vehicles. RSU can send real-time road conditions, traffic flow, accident warning, etc. information to nearby vehicles, helping drivers make safer and more effective driving decisions. In addition to sending information, RSU can also receive data from vehicles, including speed, direction of travel, vehicle type, etc. In advanced driver assistance systems (ADAS) and autonomous driving technologies, the accurate position information and real-time road condition updates provided by RSU are crucial for improving driving safety and optimizing path planning.

[0048] In this embodiment, when the finally derived probability of occurrence of the driving accident 33 is higher than the preset threshold, the RSU can be used to notify the on-board unit to issue a warning. The warning includes the driving scheme 32 and the type of accident that may occur.

[0049] Step S3) identifying several attribute information of the vehicle in the preparation zone 11 through the roadside camera 22 data and the RSU data, the attribute information at least including the license plate number.

[0050] The attribute information also includes color, length and vehicle type,

[0051] The method for identifying several attribute information of the vehicle in the preparation area 11 through the roadside camera 22 data and the RSU data includes:

[0052] The license plate number, color and length are obtained by using image recognition technology to identify the roadside camera 22 data;

[0053] The vehicle type is obtained by using a pre-configured vehicle type template to match the roadside camera 22 data when the vehicle type is not obtained through the RSU data.

[0054] By using advanced image recognition technology to process the data captured by the roadside camera 22, the system can automatically identify the license plate number, color and approximate length of the vehicle. Optical character recognition (OCR) technology is used to extract and identify the license plate number from the camera 22 image. Based on color analysis algorithm, the system can determine the main color or color scheme of the vehicle. By analyzing the pixel size of the vehicle in the image and combining with the known distance parameters, the approximate length of the vehicle is estimated.

[0055] When obtaining the vehicle type, the system attempts to obtain the vehicle type information through the communication between the RSU and the vehicle OBU. If successful, the required information is directly obtained. The vehicle type refers to the type of the car, and the vehicle type includes sedan 31, pickup, SUV, van, small truck, sports car, bus and public bus.

[0056] When the RSU fails to provide the vehicle type information, the system will turn to use a pre-configured vehicle type template library to match and analyze the roadside camera 22 data. These templates contain the characteristic profiles and some predetermined features of various common vehicle types. By comparing the actual photographed vehicle image with the records in the template database, the system can infer the most likely vehicle type. For example, a sedan 31 usually has a low body height and smooth and flowing lines, a pickup has a obvious cargo box area behind the cab, and an SUV or off-road vehicle often has a high ground clearance and a large wheel arch. The bus and public bus have a closed compartment and a long length, and the public bus has a lamp plate displaying the bus line in the front part.

[0057] Step S4) Differentiate the vehicles in the perception area 12 according to the attribute information, and track the position information of the vehicles in the perception area 12 through the laser radar data, and obtain the driving event of the vehicle according to the position information.

[0058] The attribute information also includes color, length and vehicle type,

[0059] The method for distinguishing vehicles in the perception zone 12 according to the attribute information comprises:

[0060] Frame images of the road-side camera 22 data in the perception zone 12 are extracted at preset time intervals to obtain a sequence of frame images;

[0061] The vehicles appearing in the frame images are marked, and at least one attribute of the vehicles, such as color, length, or vehicle type, is identified, and a corresponding license plate number of the identified vehicle is obtained according to the attribute, for distinguishing the vehicles.

[0062] Frame images in the perception zone 12 are extracted from the data obtained by the road-side camera 22 at preset time intervals (for example, 2 frames per second) to form a sequence of continuous frame images. In each frame image, all appearing vehicles need to be marked first. This process can be realized by a target detection algorithm in computer vision technology, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), etc. For each marked vehicle, the system will further analyze and identify at least one attribute (color, length, or vehicle type) thereof. For example, a color identification algorithm is used to determine the color of the vehicle; a known distance parameter is combined with the pixel size of the vehicle in the image to estimate the approximate length of the vehicle; and a specific type of vehicle is identified by pre-configured vehicle type template matching. After obtaining the attribute information of the color, length, or vehicle type of the vehicle, the information is associated with the license plate number obtained by the image recognition technology.

[0063] The method for distinguishing vehicles in the perception zone 12 according to the attribute information comprises:

[0064] According to the attribute information of the vehicles in the perception zone 12, a set of identified attributes of each vehicle is obtained;

[0065] An image recognition technology is used to identify the color of the vehicle appearing in the frame image;

[0066] A set of identified attributes including only the color is obtained, and a license plate number is matched according to the identified color of the vehicle;

[0067] If there are still vehicles without matched license plate numbers, the length of the vehicle is obtained according to the pixel area occupied by the color of the vehicle;

[0068] A set of identified attributes including only the color and the length is obtained, and a license plate number is matched according to the identified color and length of the vehicle;

[0069] If there are still vehicles whose license plate numbers are not matched, the pre-configured vehicle model template is used to match the roadside camera 22 data to obtain the vehicle model of the vehicle;

[0070] The license plate number is matched according to the color, length and model of the identified vehicle.

[0071] The set of identified attributes is the minimum set of attributes that can identify and distinguish vehicles. For example, a busy intersection is installed with a roadside camera 22 and a laser radar system, and the method is implemented. At a certain time point, there are four vehicles in the perception area 12 at the same time: a white small car 31, a blue SUV, a white passenger car and a white truck. The result of identifying and distinguishing the four vehicles is to obtain the license plate number, which provides the condition for the next step of driving behavior analysis.

[0072] All vehicles in the perception area 12 are preliminarily identified by image recognition to obtain the basic attribute information set of each vehicle, including color (blue, white) and preliminary estimated vehicle model outline (small car 31, passenger car, SUV, truck). Using image recognition technology, the color of each vehicle can be quickly identified.

[0073] Based on the color information, try to match the known license plate number. If the license plate number of a vehicle has been recorded before and only one vehicle of that color exists within a predetermined time period, it can be directly matched successfully. For example, the blue vehicle can directly match the license plate number of the vehicle identified in the preparation area 11.

[0074] The predetermined time period is based on the time period that a vehicle stays in the perception area 12 at a normal speed. It means that the front of the vehicle will be in the perception area 12 at the same time. That is, to determine whether there is only one vehicle of a certain color within a predetermined time period, it is necessary to determine whether there is a vehicle of the same color before that time period.

[0075] When all vehicles cannot be matched according to the color, the length of the vehicle is identified. When a vehicle has the same color, only the color cannot be matched (for example, there are three white vehicles), and the approximate length of the vehicle is estimated according to the size of the pixel area occupied by the vehicle in the frame image.

[0076] Taking the white vehicles as an example, one of the vehicles is obviously longer than the other two, so it can be inferred that the shorter one is a small car 31 or a truck, and the longer one is a passenger car. Combined with the color and length information, the license plate number is tried to match again. Since the two white vehicles in the perception area 12 have obviously different lengths, they can be quickly distinguished by the length of the vehicle.

[0077] For vehicles that cannot be identified by color and length, i.e. the white sedan 31 and the white van, pre-configured vehicle model templates are used for matching. These templates are trained based on samples and can recognize the features of various common vehicle models.

[0078] In this embodiment, after the vehicle models of the remaining two vehicles are identified, the correct license plate numbers are finally matched in combination with the previous color information, length, and vehicle model (sedan 31 and van).

[0079] Step S5) Establish the driving behavior characteristics of the vehicle according to the historical driving events of the vehicle corresponding to the license plate number.

[0080] The method for obtaining the driving events of the vehicle according to the position information comprises:

[0081] Obtain the speed curve of the vehicle according to the position information, and obtain the sudden braking event of the vehicle according to the speed curve;

[0082] Obtain the lane changing event of the vehicle according to the position information, and the lane changing event comprises lane changing span, the closest distance to nearby vehicles during lane changing, and the speed change value at the beginning and end of lane changing;

[0083] Obtain the tailgating event of the vehicle according to the position information, and the tailgating event is an event in which the following distance is less than a preset threshold;

[0084] Obtain the collision event and the scraping event of the vehicle according to the position information.

[0085] Using the high-precision distance measurement capability provided by the laser radar, the accurate position of the vehicle within the perception area 12 can be tracked in real time. According to the position changes of the vehicle at different times, the driving speed of the vehicle is calculated, and then abnormal behaviors such as sudden braking are found. The sudden braking event is identified by analyzing the sharp deceleration section in the speed curve. If the speed of the vehicle suddenly and significantly decreases at a certain time (for example, the deceleration exceeds a preset threshold), it is considered that a sudden braking event has occurred.

[0086] It is monitored whether the vehicle moves across the lane line, and details such as lane changing span, the closest distance to nearby vehicles during lane changing, and the speed change value at the beginning and end of lane changing are recorded. The tailgating event is triggered when the distance between two vehicles is detected to be less than a set safety threshold.

[0087] In combination with the position information and the speed change, it is judged whether there is a collision or scraping situation. When the position coordinates of two vehicles show that they overlap or are in close contact, in combination with the change of the speed curve (such as sudden speed drop), it can be inferred that a collision or scraping event has occurred. The vehicle collision event is an event when two vehicles are in the same lane, and the vehicle scraping event is an event when two vehicles are in different lanes.

[0088] Step S6) predicting the probability and type of the driving accident 33 in the current perception area 12 according to the response of the driving behavior feature to the pre-configured driving accident 33 prediction model.

[0089] The method for establishing the driving behavior feature of the vehicle according to the historical driving events of the vehicle corresponding to the license plate number comprises:

[0090] The method for establishing the driving behavior feature according to the historical driving events comprises:

[0091] reading the historical driving events of each vehicle;

[0092] The input of the driving behavior feature is the vehicle emergency braking event, the vehicle following too close event, the vehicle lane changing event, the vehicle collision event and the vehicle scratching event, and the output is the braking reaction distance, the following habit distance, the lane changing aggressiveness and the lane changing safety.

[0093] receiving the braking reaction distance generated according to the vehicle emergency braking event, wherein the braking reaction distance is the average braking distance in the vehicle emergency braking event corresponding to the license plate number, the braking distance is the distance of the vehicle forward during the process of starting to decelerate to the speed lower than the preset threshold value, and the deceleration acceleration is greater than the preset threshold value;

[0094] receiving the following habit distance generated according to the vehicle following too close event, calculating the product of the number of vehicle following too close events and a preset coefficient, and the following habit distance is the difference between the preset constant value and the product;

[0095] receiving the lane changing aggressiveness generated according to the vehicle lane changing event, calculating the weighted sum of the total number and the total span of the vehicle lane changing events according to the statistics of the total number and the total span of the vehicle lane changing events, and taking the weighted sum after normalization as the lane changing aggressiveness;

[0096] receiving the lane changing safety generated according to the vehicle lane changing event, the vehicle collision event and the vehicle scratching event, generating a first coefficient according to the quotient of the nearest distance to the nearby vehicle during the lane changing process and the preset reference distance, generating a second coefficient according to the reciprocal of the lane changing start and end speed change value, generating a third coefficient according to the reciprocal of the total number of vehicle collision events and vehicle scratching events, and using the product of the preset initial value and the first coefficient, the second coefficient and the third coefficient as the lane changing safety.

[0097] The driving events are collected to establish the driving behavior feature according to the historical driving events of the vehicle corresponding to the license plate number. The historical driving event records of each vehicle are read, including the emergency braking event, the following too close event, the lane changing event, the collision event and the scratching event. The output of the driving behavior feature model is a series of indexes describing the driving behavior, including the braking reaction distance, the following habit distance, the lane changing aggressiveness and the lane changing safety, etc.

[0098] The brake reaction distance is quantified based on the average braking distance in the vehicle emergency braking event, i.e. the distance the vehicle travels from the start of deceleration to the speed being lower than a preset threshold. The following car habit distance, which reflects the minimum safety distance the driver tends to maintain, is calculated by counting the number of vehicle following too close events and applying a preset coefficient. The lane changing aggressiveness, which embodies the driver's preference for the frequency and amplitude of lane changing, is obtained by weighting and normalizing the total number of lane changing events and their span. The lane changing safety is generated by a specific algorithm considering factors such as the shortest distance to surrounding vehicles, speed change value and whether a collision occurs during the lane changing process, and represents the safety level of the lane changing operation.

[0099] The brake reaction distance, following car habit distance, lane changing aggressiveness and lane changing safety should all be normalized to convert them into dimensionless values for subsequent operations.

[0100] The detailed driving behavior characteristics of each vehicle are constructed by the brake reaction distance, following car habit distance, lane changing aggressiveness and lane changing safety, which not only helps to depict the habits and risk tendencies of the driver, but also provides a basis for predicting potential traffic accidents.

[0101] The method for predicting the probability and type of driving accidents 33 in the current perception area 12 domain according to the response of the pre-configured driving accident 33 prediction model to the driving behavior characteristics includes:

[0102] The driving accident 33 prediction model includes a driving behavior simulation module and an accident monitoring module;

[0103] The driving behavior simulation module generates a number of possible driving scenarios 32 for each vehicle according to the position information and current speed of the vehicles in the perception area 12, and the driving scenario 32 includes driving route, speed curve and probability;

[0104] The combinations of all possible driving scenarios 32 of all vehicles are listed, and the vehicles are simulated to pass through the perception area 12 according to the combinations of driving scenarios 32. During the simulation, the accident monitoring module determines whether a driving accident 33 occurs according to preset distance judgment conditions;

[0105] When a driving accident 33 occurs, the type of accident is obtained according to the corresponding distance judgment condition, and the probability of the driving accident 33 is calculated according to the probability corresponding to the driving scenario 32 of the vehicle that caused the accident.

[0106] The driving behavior simulation module is used to predict the possible behavior path of each vehicle in a future period of time according to real-time perception data. The accident monitoring module is used to monitor the position relationship between vehicles in real time during the simulation process, to determine whether the preset accident triggering condition is met, and to identify the type of accident and calculate the probability.

[0107] The driving behavior simulation module generates several possible driving schemes 32 based on the vehicle's current location and speed, combined with its historical driving behavior characteristics (such as lane-changing initiative, following distance preference, braking response, etc.), as shown in the appendix. Figure 3 As shown. The driving route represents the vehicle's possible trajectory, such as maintaining the current lane, changing lanes left / right, slowing down to yield, or making an emergency maneuver. The speed curve shows the vehicle's speed over time under this scenario. The probability indicates the likelihood of driving scenario 32 occurring, determined by both driving behavior characteristics and the current traffic environment. For example, a vehicle with a "high lane-changing initiative" is more likely to choose to change lanes when traffic is slow.

[0108] For example, an SUV is driving in a straight lane with a slower vehicle ahead. The simulation module may generate three driving options 32: maintain straight and follow the vehicle ahead (60% probability); change lanes to the left to overtake (30% probability); and brake suddenly to avoid the vehicle ahead (10% probability). All combinations of driving options 32 are enumerated and simulated. Since there are multiple vehicles within the perception zone 12, the system performs a Cartesian product of all possible driving options 32 for each vehicle, generating all possible traffic scenario combinations. For example, if there are 3 vehicles, each with 3 possible driving options 32, a total of 27 combinations of driving options 32 are generated. Dynamic traffic simulation is performed for each scenario combination, simulating the vehicle's movement within the perception zone 12, with a time step of 2 seconds or less.

[0109] During the simulation, the accident monitoring module continuously monitors the spatial relationship between vehicles and determines whether a driving accident has occurred based on preset distance judgment conditions.

[0110] If the minimum distance between the two vehicles is ≤0, a collision is determined (e.g., rear-end collision, side collision). If the minimum distance between the two vehicles is >0 but less than a safety threshold (e.g., 1 meter) and their trajectories intersect, a scrape is determined. If a vehicle deviates from its lane line and is too close to a vehicle in the adjacent lane, a scrape is determined (lane change scrape). When any of these conditions are met during the simulation, the accident monitoring module determines that an accident has occurred. For each of the 32 driving scheme combinations that lead to an accident, the total probability of its occurrence is the product of the probabilities of the schemes selected by each vehicle.

[0111] For example, the combination: car A selects the "emergency braking" scheme, the probability is 10%; car B selects the "too close to the car" scheme, the probability is 20%. The combination leads to a rear-end collision accident, and the probability of occurrence of the driving accident 33 is: P = 0.1 x 0.2 = 0.02. The total probability of the driving accident 33 occurring in the current perception area 12, including various types of accidents (such as rear-end collision, scratching, side collision, etc.) and their probabilities, is the road surface perception result. The scheme of the embodiment generates multiple possibility paths through the driving behavior simulation module, and performs dynamic risk identification in combination with the accident monitoring module, realizes the probabilistic and typological prediction of the driving accident 33 in the complex traffic environment, and is a key technical scheme of the intelligent traffic system to active safety and forward-looking prediction.

[0112] On the other hand, the present specification provides a road surface perception system based on multi-source data fusion, please refer to the attached Figure 4 , comprising:

[0113] The division module 100 divides the perception area 12 of the road surface, and sets the preparation area 11 in front of the perception area 12;

[0114] The receiving module 200 receives multi-source data covering the preparation area 11 and the perception area 12, and the multi-source data includes at least one of the roadside camera 22 data, the laser radar data and the RSU data;

[0115] The identification module 300 identifies a plurality of attribute information of the vehicle in the preparation area 11 through the roadside camera 22 data and the RSU data, and the attribute information at least includes the license plate number;

[0116] The perception module 400 distinguishes the vehicle in the perception area 12 according to the attribute information, and tracks the position information of the vehicle in the perception area 12 through the laser radar data, and obtains the driving event of the vehicle according to the position information;

[0117] The feature module 500 establishes the driving behavior characteristics of the vehicle according to the historical driving event of the vehicle corresponding to the license plate number;

[0118] The prediction module 600 predicts the probability and type of the driving accident 33 occurring in the current perception area 12 according to the response of the driving behavior characteristics to the pre-configured driving accident 33 prediction model.

[0119] Please refer to Figure 5 The structure schematic diagram of the electronic equipment provided by the embodiment of the present specification is shown.

[0120] As Figure 5As shown, the electronic device 1100 can include at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 can be used to realize the connection and communication of the above-mentioned components. The user interface 1103 can include a key, and the optional user interface can also include a standard wired interface, a wireless interface. The network interface 1104 can include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc. The processor 1101 can include one or more processing cores. The processor 1101 connects various parts in the entire electronic device 1100 through various interfaces and lines, executes various functions of the routing device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1105, and calling data stored in the memory 1105. Optionally, the processor 1101 can be realized by at least one of DSP, FPGA, PLA. The processor 1101 can integrate CPU, GPU and modem, etc. The CPU is mainly used to process operating systems, user interfaces and application programs, etc. The GPU is used to render and draw the content required to be displayed on the display screen. The modem is used to process wireless communication.

[0121] It can be understood that the above-mentioned modem can also not be integrated into the processor 1101, but be realized by a separate chip.

[0122] The memory 1105 can include RAM and ROM. Optionally, the memory 1105 includes a non-transitory computer readable medium. The memory 1105 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1105 can include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 1105 can also be at least one storage device located away from the above-mentioned processor 1101. The memory 1105 as a computer storage medium can include an operating system, a network communication module, a user interface module and an application program. The processor 1101 can be used to call the application program stored in the memory 1105 and execute the method in the above-mentioned multiple embodiments.

[0123] The embodiments of the present specification also provide a computer-readable storage medium, which stores instructions, when the instructions are executed on a computer or a processor, cause the computer or the processor to perform the steps of the above-mentioned embodiments. The constituent modules of the above-mentioned electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in the computer-readable storage medium.

[0124] The embodiments of the present specification also provide a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned embodiments.

[0125] In the case of no conflict, the technical features in the embodiments and the implementation forms can be combined arbitrarily.

[0126] In the above-mentioned embodiments, all or part of the embodiments can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product. The computer program product comprises a plurality of computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with a plurality of available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)) and the like.

[0127] When implemented by hardware or firmware, the foregoing method flow is programmed into a hardware circuit to obtain a corresponding hardware circuit structure, and the corresponding function is realized. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is an integrated circuit, and the logic function thereof is determined by the user programming the device. A digital system is "integrated" on a PLD by the designer programming it by himself / herself, and a chip manufacturer does not need to be invited to design and manufacture a special integrated circuit chip. Moreover, nowadays, instead of manually manufacturing an integrated circuit chip, this programming is mostly implemented by using a "logic compiler" software, which is similar to a software compiler used when a program is developed and written, and the original code before the compilation also needs to be written in a specific programming language, which is called a hardware description language (HDL), and there are many kinds of HDLs. It should be clear to those skilled in the art that the method flow only needs to be logically programmed in the above-mentioned several hardware description languages and programmed into an integrated circuit, and a hardware circuit that realizes the logical method flow can be easily obtained.

[0128] The above-described embodiments are merely preferred embodiments of the present specification, and do not limit the scope of the present specification. Various changes and modifications to the technical solutions of the present specification made by those skilled in the art without departing from the design spirit of the present specification shall fall within the protection scope of the claims of the present specification.

Claims

1. A road surface perception method based on multi-source data fusion, characterized in that, Including the following steps: The road surface is divided into sensing zones, and a preparation zone is set up in front of the sensing zones; Receive multi-source data covering the preparation area and the perception area, the multi-source data including various types of roadside camera data, lidar data and RSU data; Using roadside camera data and RSU data, several attribute information of vehicles in the preparation area are identified, including at least the license plate number. Vehicles within the perception area are distinguished based on the attribute information, and the vehicle's position information within the perception area is tracked using LiDAR data. The vehicle's driving events are obtained based on the position information. Based on the historical driving events of the vehicle corresponding to the license plate number, establish the vehicle's driving behavior characteristics; Based on the response of the driving behavior characteristics to the pre-configured driving accident prediction model, the probability and type of driving accident in the current perception area are predicted.

2. The road perception method based on multi-source data fusion according to claim 1, characterized in that, The attribute information also includes color, length, and vehicle model. Methods for identifying several attribute information of vehicles in the preparation area using roadside camera data and RSU data include: Image recognition technology is used to identify the data from the roadside camera to obtain the license plate number, color, and length; An attempt is made to identify the vehicle model using the RSU data. If the vehicle model cannot be obtained using the RSU data, a pre-configured model template is used to match the roadside camera data to obtain the vehicle model.

3. The road perception method based on multi-source data fusion according to claim 1, characterized in that, The attribute information also includes color, length, and vehicle model. The method for distinguishing vehicles within the perception area based on the attribute information includes: Frame images of the roadside camera data within the sensing area are extracted at preset time intervals to obtain a frame image sequence; Vehicles appearing in the marked frame image are identified, and at least one attribute of the vehicle, such as color, length, or model, is determined. The license plate number corresponding to the identified vehicle is obtained based on the attribute to distinguish the vehicle.

4. The road perception method based on multi-source data fusion according to claim 3, characterized in that, The method for distinguishing vehicles includes: identifying vehicles appearing in a labeled frame image and recognizing at least one attribute of the vehicle's color, length, or model; obtaining a matching license plate number based on the attribute; and using the method to distinguish vehicles. Based on the attribute information of the vehicles within the perception area, obtain the identification attribute set for each vehicle; Using image recognition technology, the colors of the vehicles appearing in the frame image are identified; Obtain a set of recognition attributes that includes only color, and match the license plate number based on the color of the recognized vehicle; If there are still vehicles with unmatched license plate numbers, the length of the vehicle is obtained based on the pixel area occupied by the color of the vehicle. Obtain a set of recognition attributes that includes only color and length, and match the license plate number based on the color and length of the recognized vehicle; If there are still vehicles whose license plate numbers are not matched, the vehicle model is obtained by matching the roadside camera data with the pre-configured vehicle model template. The license plate number is matched based on the color, length, and model of the identified vehicle.

5. The road surface perception method based on multi-source data fusion according to any one of claims 1 to 4, characterized in that, Methods for obtaining vehicle driving events based on the location information include: The vehicle speed curve is obtained based on the location information, and the vehicle emergency braking event is obtained based on the speed curve. The vehicle lane change event is obtained based on the location information. The vehicle lane change event includes the lane change span, the closest distance to the nearest vehicle during the lane change, and the change speed at the start and end of the lane change. Based on the location information, a vehicle following too closely event is obtained, wherein the following too closely event is an event in which the following distance is less than a preset threshold. Vehicle collision events and vehicle scraping events are obtained based on the location information.

6. The road perception method based on multi-source data fusion according to claim 5, characterized in that, Methods for establishing vehicle driving behavior characteristics based on the historical driving events corresponding to the license plate number include: Read the historical driving events for each vehicle; The inputs to the driving behavior characteristics are vehicle emergency braking events, vehicle following too closely events, vehicle lane changing events, vehicle collision events, and vehicle scraping events, and the outputs are braking reaction distance, following habit distance, lane changing initiative, and lane changing safety. Receive the braking reaction distance generated based on the vehicle emergency braking event. The braking reaction distance is the average braking distance in the emergency braking event of the vehicle corresponding to the license plate number. The braking distance is the distance the vehicle travels during the process of deceleration from the beginning to the speed below a preset threshold, and the deceleration acceleration is greater than the preset threshold. Receive the following habit distance generated based on the vehicle following too closely event, calculate the product of the number of vehicle following too closely events and a preset coefficient, wherein the following habit distance is a preset constant value minus the difference of the product; Receive lane change initiative generated based on vehicle lane change events, calculate a weighted sum of the total number of lane change events and the total span based on the total number of lane change events and the total span, and normalize the weighted sum as the lane change initiative; The system receives lane change stability data generated based on vehicle lane change events, vehicle collision events, and vehicle scrape events. It generates a first coefficient based on the quotient of the closest distance to a nearby vehicle during the lane change process and a preset reference distance, generates a second coefficient based on the reciprocal of the change speed at the start and end of the lane change, and generates a third coefficient based on the reciprocal of the total number of vehicle collision events and vehicle scrape events. The system uses a preset initial value and the product of the first, second, and third coefficients as the lane change stability data.

7. The road surface perception method based on multi-source data fusion according to claim 6, characterized in that, The method for predicting the probability and type of a driving accident in the current perception area based on the response of a pre-configured driving accident prediction model to the driving behavior features includes: The driving accident prediction model includes a driving behavior simulation module and an accident monitoring module; The driving behavior simulation module generates several possible driving plans for each vehicle based on the vehicle's location information and current speed within the perception area. The driving plan includes a driving route, speed curve, and probability. List all possible combinations of driving schemes for all vehicles, simulate the vehicle passing through the perception zone according to the combination of driving schemes, and during the process, the accident monitoring module determines whether a driving accident has occurred based on preset distance judgment conditions; When a driving accident occurs, the accident type is determined based on the corresponding distance conditions, and the probability of the driving accident is calculated based on the probability corresponding to the driving plan of the vehicle involved in the accident.

8. A road surface perception system based on multi-source data fusion, characterized in that, include: The module divides the road surface into sensing zones, and a preparation zone is set in front of the sensing zones; The receiving module receives multi-source data covering the preparation area and the sensing area, including multiple types of data such as roadside camera data, lidar data, and RSU data. The identification module identifies several attribute information of vehicles in the preparation area through roadside camera data and RSU data, and the attribute information includes at least the license plate number. The perception module distinguishes vehicles within the perception area based on the attribute information, tracks the vehicle's position information within the perception area using lidar data, and obtains the vehicle's driving events based on the position information. The feature module establishes the driving behavior features of a vehicle based on its historical driving events corresponding to the license plate number. The prediction module predicts the probability and type of a driving accident in the current perception area based on the response of the driving behavior characteristics to a pre-configured driving accident prediction model.

9. An electronic device, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images