Multi-sensor target data fusion detection method and system for intelligent driving
Through the multi-sensor target data fusion detection method, the location and driving conditions of other vehicles on the road are identified and analyzed, and reminder information is provided, which solves the problem that existing intelligent driving technology cannot accurately remind, and improves the intelligence and practicality of intelligent driving.
Patent Information
- Application Number
- CN202410930166.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-07-11
AI Technical Summary
The existing intelligent driving technology cannot accurately provide reminder information in manual driving vehicles, which causes car owners to pay close attention to the lane changes of other vehicles when driving on complex roads, resulting in insufficient driving and insufficient intelligent driving.
Through the multi-sensor target data fusion detection method, road pictures are obtained, other vehicles are identified, location information is obtained, data fusion processing is performed, direction labels and prediction labels are output, split distances and relative speeds are calculated, driving judgment information is analyzed, and reminder information is provided.
It improves the intelligence and efficiency of lane change situations of other vehicles, predicts potential collision risks, judges the impact of lane change on driving, and outputs driving judgment information used to remind drivers to improve the practicality and intelligence of intelligent driving.
Smart Images

Figure CN118468233B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent driving data fusion, and specifically to a multi-sensor target data fusion detection method and system for intelligent driving. Background Art
[0002] Intelligent driving essentially involves cognitive engineering of attention attraction and attention distraction, and mainly includes three aspects: network navigation, autonomous driving and manual intervention. Intelligent driving can obtain relevant visual and auditory signals and information through sensors on the vehicle, and output reminder information through calculation to remind the driver.
[0003] Existing improvements for intelligent driving usually analyze all environmental information, make driving decisions based on the analysis results, and control vehicle driving to improve the safety of intelligent driving in autonomous driving scenarios. For example, a Chinese patent application with publication number CN117333746A discloses a multi-sensor data fusion method, device, medium and equipment for intelligent driving. This solution can solve the problem of inaccurate target detection results in perception technology based on multimodal solutions in the prior art, thereby achieving the purpose of improving the safety and reliability of intelligent driving. However, the existing autonomous driving technology is not mature enough, and there are still many car owners who drive their vehicles manually. At this time, multi-sensors can be used for target data fusion detection, thereby analyzing the lane change situation of other vehicles on the road, outputting reminder information, and realizing an intelligent driving method for assisting car owners in driving. In view of this, it is necessary to improve the existing intelligent driving. Summary of the invention
[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent. By improving the intelligent driving, it is used to solve the problem in the prior art that when the car owner chooses to drive the vehicle manually, the intelligent driving technology cannot accurately provide the car owner with reminder information to assist the car owner in driving, which leads to the car owner needing to continue to pay close attention to the lane changes of other vehicles on the road when driving on complex roads, resulting in the car owner's driving not being easy enough and the intelligent driving not being smart enough.
[0005] To achieve the above objectives, in a first aspect, the present invention provides a multi-sensor target data fusion detection method for intelligent driving, comprising:
[0006] Obtain road images and identify other vehicles in the road images; obtain the location information of other vehicles and the vehicle itself, perform data fusion processing on the location information and road images, and output direction labels;
[0007] Perform driving prediction processing on other vehicles and output prediction labels; calculate the position information and output the split distance;
[0008] The split distance is calculated, and the displacement distance and relative speed are output based on the calculation results; the direction label and prediction label are analyzed based on the displacement distance and relative speed, and the driving judgment information is output.
[0009] Furthermore, obtaining a road picture and identifying other vehicles in the road picture; obtaining position information of other vehicles and the vehicle includes:
[0010] At each interval, a camera is used to obtain a road image in front of the vehicle, and the top and bottom sides of the road image are defined as the length of the road image;
[0011] Binarize the road image to obtain the outline of each other vehicle in the road image; and use a rectangular frame to mark other vehicles in the road image;
[0012] At each first interval, the laser radar is used to detect the position information of other vehicles in front of the vehicle and the vehicle, wherein the position information includes the straight-line distance between the other vehicles and the vehicle and the angle between the other vehicles and the vehicle;
[0013] The location information is sorted from left to right in ascending order of straight-line distance and numbered, and marked as a location sequence.
[0014] Furthermore, data fusion processing of the location information and the road picture includes:
[0015] Set the number i to an initial value of 1;
[0016] Performing raw data fusion processing, the raw data fusion processing comprising:
[0017] Set the dividing line segment with the length of the road image, starting from the bottom of the road and moving along the width of the road image to the top of the road image;
[0018] It is stipulated that the bounding box of each other vehicle shall be numbered only when it intersects the dividing line segment for the first time;
[0019] When the dividing line segment intersects with the border of any other vehicle for the first time, the number of the other vehicle is set to i, and i is increased by 1;
[0020] The original data fusion process is performed again until the segmentation line moves to the top of the road image;
[0021] Perform digital-graphic data fusion processing.
[0022] Furthermore, the digital image data fusion processing includes:
[0023] Associate all other vehicles at the same time with their location information by number;
[0024] Connect the bottom midpoint and the top midpoint of the road image to get the center axis; add right lane labels to other vehicles whose borders are completely on the right side of the center axis; add the same lane labels to other vehicles whose borders intersect the center axis; add left lane labels to other vehicles whose borders are completely on the left side of the center axis;
[0025] Position information and direction labels are displayed in the bounding box of each other vehicle in the road image, and the direction labels include a left lane label, a right lane label, and a same lane label.
[0026] Furthermore, the prediction labels include static labels, left turn labels, and right turn labels, and the driving prediction processing of other vehicles is performed to output the prediction labels, including:
[0027] Using image processing technology to identify other vehicles in the road picture, the driving prediction process includes:
[0028] Identify the taillight conditions of other vehicles, where the taillight conditions include whether the left turn light is flashing, the right turn light is flashing, or not flashing;
[0029] When the taillight condition is not flashing, set a static tag for other vehicles;
[0030] When the taillight condition is that the left turn signal is flashing, set the left turn tag for other vehicles;
[0031] When the taillight condition is that the right turn signal is flashing, set a right turn tag for other vehicles;
[0032] The predicted labels are displayed in the borders of each other vehicle in the road picture, and the road picture is cached; the predicted labels include static labels, left turn labels, and right turn labels.
[0033] Furthermore, the split distance includes a horizontal distance and a vertical distance, and the position information is calculated and the split distance is output, including:
[0034] Use the mapping calculation formula to calculate the position information of each other vehicle and output the lateral distance and vertical distance;
[0035] The mapping calculation formula is configured as: ; Among them, H is the horizontal distance, L is the vertical distance, A is the angle, and zx is the straight-line distance.
[0036] Furthermore, the displacement distance includes a vertical displacement distance and a lateral displacement distance, and the split distance is calculated, and the displacement distance and the relative speed are output based on the calculation result, including:
[0037] Get the vehicle's speed;
[0038] Get the road picture before the first time and mark it as a historical picture;
[0039] Calculate the difference between the vertical distance of each identical other vehicle in the road image and the vertical distance of other vehicles in the historical image, and mark it as the vertical displacement distance;
[0040] Calculate the ratio of the vertical displacement distance to the first time, marked as relative velocity;
[0041] The difference between the lateral distance of each identical other vehicle in the road image and the lateral distance of other vehicles in the historical image is calculated and marked as the lateral displacement distance.
[0042] Furthermore, the driving judgment information includes normal driving information, deceleration driving information and waiting deceleration information. The direction label and the prediction label are analyzed based on the displacement distance and the relative speed, and the output driving judgment information includes:
[0043] Analyze other vehicles that have the same lane label and left turn label combination or the same lane label and right turn label combination;
[0044] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, outputting normal driving information;
[0045] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information;
[0046] When the lateral displacement distance is less than the first displacement distance and the relative speed is less than 0, the waiting deceleration information is output;
[0047] When the lateral displacement distance is less than the first displacement distance and the relative speed is greater than or equal to 0, normal driving information is output.
[0048] Furthermore, the direction label and the prediction label are analyzed based on the displacement distance and the relative speed, and the output driving judgment information also includes:
[0049] Analyze other vehicles that have a combination of a left lane label and a right turn label or a combination of a right lane label and a left turn label;
[0050] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, outputting normal driving information;
[0051] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information;
[0052] When the lateral displacement distance is less than the first displacement distance, normal driving information is output.
[0053] In a second aspect, the present invention also provides a multi-sensor target data fusion detection system for intelligent driving, including an information acquisition module, an information labeling module, and a driving judgment module;
[0054] The information acquisition module is used to acquire road pictures, identify other vehicles in the road pictures, and acquire location information of other vehicles and the vehicle itself;
[0055] The information labeling module is used to perform data fusion processing on the location information and the road picture, and output the direction label; perform driving prediction processing on other vehicles, and output the prediction label; calculate the location information, and output the split distance;
[0056] The driving judgment module is used to calculate the split distance, and output the displacement distance and relative speed based on the calculation results; analyze the direction label and the prediction label based on the displacement distance and the relative speed, and output the driving judgment information.
[0057] Beneficial effects of the present invention: The present invention obtains a road picture, uses image processing technology to obtain the frame of each other vehicle in the road picture, and then obtains the position information of each other vehicle and the vehicle. Through data fusion processing, the position information can be matched with other vehicles one by one; then the lanes where other vehicles are located are analyzed, and a direction label is marked for each other vehicle; the advantage of this is that the lane change conditions of other vehicles on different lanes can be classified, and the label combination that will affect the driving of the vehicle can be screened out, and the specific label combination can be analyzed, and the non-specific label combination does not need to be analyzed, thereby improving the intelligence and efficiency of lane change analysis;
[0058] The present invention also calculates the lateral distance and relative speed between each other vehicle and the vehicle, analyzes the lane change situation based on the lateral distance and relative speed, and finally outputs driving judgment information. The advantage of this is that the relative speed can be used to predict whether there is a risk of collision between the vehicle and other vehicles in the subsequent driving process, and the change in the lateral distance can be used to determine whether the other vehicle is about to change lanes or is changing lanes; by combining the change in the lateral distance and the relative speed, it is finally determined whether the lane change of other vehicles has an impact on the driving of the vehicle, and the driving judgment information used to remind the driver is output, thereby improving the practicality and intelligence of intelligent driving. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 is a functional block diagram of the system of the present invention;
[0060] Figure 2 is a flow chart of the steps of the method of the present invention;
[0061] Figure 3It is a three-dimensional schematic diagram of the position information of the vehicle and other vehicles of the present invention. DETAILED DESCRIPTION
[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] Embodiment 1, first aspect, the present invention provides refer to Figure 1 As shown, the present invention provides a multi-sensor target data fusion detection system for intelligent driving, including an information acquisition module, an information labeling module and a driving judgment module;
[0064] The information acquisition module is used to acquire road images, identify other vehicles in the road images, and acquire the location information of other vehicles and the vehicle itself;
[0065] The information acquisition module is configured with an image processing strategy, which includes:
[0066] At each interval, the camera is used to obtain a road image in front of the vehicle, and the top and bottom sides of the road image are defined as the length of the road image;
[0067] Binarize the road image to obtain the outline of each other vehicle in the road image; use a regular rectangular frame to mark other vehicles in the road image;
[0068] At each interval, the laser radar is used to detect the position information of other vehicles in front of the vehicle and the vehicle. The position information includes the straight-line distance between the other vehicles and the vehicle and the angle between the other vehicles and the vehicle.
[0069] In specific implementation, the first time is set to 0.5s. In theory, the smaller the first time is set, the more timely the analysis of other vehicles will be. However, at this time, the operating burden of the system is large, and the system needs a certain amount of time to perform the analysis. Considering the comprehensive timeliness, system burden and system operation efficiency, the first time is set to 0.5s.
[0070] The location information is sorted from left to right in ascending order of straight-line distance and numbered, and marked as a location sequence.
[0071] The information annotation module is used to perform data fusion processing on the location information and road pictures and output the direction label; perform driving prediction processing on other vehicles and output the prediction label; calculate the location information and output the split distance;
[0072] The information annotation module is configured with a data fusion strategy, which includes:
[0073] Set the number i to an initial value of 1;
[0074] Perform raw data fusion processing, which includes:
[0075] Set the dividing line segment with the length of the road image, starting from the bottom of the road and moving along the width of the road image to the top of the road image;
[0076] It is stipulated that the bounding box of each other vehicle shall be numbered only when it intersects the dividing line segment for the first time;
[0077] When the dividing line segment intersects with the border of any other vehicle for the first time, the number of the other vehicle is set to i, and i is increased by 1;
[0078] The original data fusion process is performed again until the segmentation line moves to the top of the road image;
[0079] Perform digital and image data fusion processing;
[0080] Associate all other vehicles at the same time with their location information by number;
[0081] Connect the bottom midpoint and the top midpoint of the road image to get the center axis; add right lane labels to other vehicles whose borders are completely on the right side of the center axis; add the same lane labels to other vehicles whose borders intersect the center axis; add left lane labels to other vehicles whose borders are completely on the left side of the center axis;
[0082] The position information and direction labels are displayed in the bounding box of each other vehicle in the road image. The direction labels include left lane labels, right lane labels, and same lane labels.
[0083] The information annotation module is also equipped with a driving prediction strategy, which includes:
[0084] Image processing technology is used to identify other vehicles in road pictures and perform driving prediction processing. Driving prediction processing includes:
[0085] Identify the taillight conditions of other vehicles, including whether the left turn signal is flashing, the right turn signal is flashing, or not flashing;
[0086] When the taillight condition is not flashing, set a static tag for other vehicles;
[0087] When the taillight condition is that the left turn signal is flashing, set the left turn tag for other vehicles;
[0088] When the taillight condition is that the right turn signal is flashing, set a right turn tag for other vehicles;
[0089] Display the predicted label in the bounding box of each other vehicle in the road image and cache the road image; the predicted label includes static label, left turn label and right turn label;
[0090] In specific implementation, the RGB value of each pixel in the border is analyzed through image processing technology, and the area corresponding to the red hue is identified, so as to obtain the area where the taillights of other vehicles are located; by calculating the distance between the taillight area and the leftmost or rightmost side of the border, it is determined by distance whether each taillight area corresponds to the left taillight or the right taillight; finally, the RGB value in each taillight area is analyzed to determine whether there is an RGB value with an orange-yellow hue, so as to determine whether the taillight is shining; when the left taillight is shining, it means that the left turn light is shining; when the right taillight is shining, it means that the right turn light is shining.
[0091] The information annotation module is also configured with a distance calculation strategy, which includes:
[0092] Use the mapping calculation formula to calculate the position information of each other vehicle and output the lateral distance and vertical distance;
[0093] The mapping calculation formula is configured as: ; Where H is the horizontal distance, L is the vertical distance, A is the angle, and zx is the straight-line distance;
[0094] like Figure 3 As shown, Figure 3 Display the position information of other vehicles and the vehicle in the southwest isometric perspective; Figure 3 In the figure, T1 is other vehicles, T2 is the own vehicle, T3 is the straight-line distance, T4 is the angle, T5 is the vertical distance, and T6 is the lateral distance;
[0095] If the straight-line distance is 6m and the angle is 30°, then the horizontal distance H=Sin30°×6, that is, 3m, and the vertical distance L=Cos30°×6, that is, 5.2m.
[0096] The driving judgment module is used to calculate the split distance, and output the displacement distance and relative speed based on the calculation results; analyze the direction label and the prediction label based on the displacement distance and relative speed, and output the driving judgment information;
[0097] The driving judgment module is equipped with a motion analysis strategy, which includes:
[0098] Get the vehicle's speed;
[0099] Get the road picture before the first time and mark it as a historical picture;
[0100] Calculate the difference between the vertical distance of each identical other vehicle in the road image and the vertical distance of other vehicles in the historical image, and mark it as the vertical displacement distance;
[0101] Calculate the ratio of the vertical displacement distance to the first time, marked as relative velocity;
[0102] It should be noted that when the relative speed is greater than 0, it means that the speed of other vehicles is greater than that of the vehicle itself; when the relative speed is less than 0, it means that the speed of other vehicles is less than that of the vehicle itself; when the relative speed is equal to 0, it means that other vehicles remain relatively stationary with the vehicle itself;
[0103] The difference between the lateral distance of each identical other vehicle in the road image and the lateral distance of other vehicles in the historical image is calculated and marked as the lateral displacement distance.
[0104] The driving judgment module is also configured with a driving judgment strategy, which includes:
[0105] Analyze other vehicles that have the same lane label and left turn label combination or the same lane label and right turn label combination;
[0106] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, outputting normal driving information;
[0107] In specific implementation, the first displacement distance is set to 0.2m. Taking a car with an average length of 1.6m as an example, in extreme cases, the car needs to move 1.6m laterally to change lanes. If other vehicles change lanes at a speed of 0.2m / s in the lateral direction, it takes 8s for the car to change lanes. By setting the first displacement distance to 0.2m, the driving judgment information can be output when the car has 7s left to complete the lane change, providing the driver with sufficient reaction time to make driving decisions.
[0108] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information;
[0109] When the lateral displacement distance is less than the first displacement distance and the relative speed is less than 0, the waiting deceleration information is output;
[0110] When the lateral displacement distance is less than the first displacement distance and the relative speed is greater than or equal to 0, normal driving information is output;
[0111] Analyze other vehicles that have a combination of a left lane label and a right turn label or a combination of a right lane label and a left turn label;
[0112] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, outputting normal driving information;
[0113] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information;
[0114] When the lateral displacement distance is less than the first displacement distance, normal driving information is output.
[0115] It should be noted that, taking other vehicles having the same lane label and left turn label combination or the same lane label and right turn label combination as examples:
[0116] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, it indicates that other vehicles are changing lanes at a speed greater than or equal to the driving speed of the vehicle. At this time, there is no impact on the driving of the vehicle, and normal driving information is output;
[0117] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, it means that the other vehicle is changing lanes at a speed lower than the speed of the vehicle. At this time, the two vehicles may be too close to each other, so the deceleration driving information is output;
[0118] When the lateral displacement distance is less than the first displacement distance and the relative speed is less than 0, it means that other vehicles are slowing down to prepare for lane change. At this time, they need to slow down to avoid collision, and the waiting deceleration information is output;
[0119] When the lateral displacement distance is less than the first displacement distance and the relative speed is greater than or equal to 0, it means that other vehicles are preparing to change lanes at a speed greater than the driving speed of the vehicle, which has no effect on the driving of the vehicle, and normal driving information is output;
[0120] Take other vehicles with a left lane label and a right turn label combination or a right lane label and a left turn label combination as examples:
[0121] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, it indicates that other vehicles are changing lanes at a speed greater than the driving speed of the vehicle, which has no impact on the driving of the vehicle, and outputs normal driving information;
[0122] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, it means that other vehicles in other lanes are changing lanes at a speed lower than the speed of the vehicle and entering the lane where the vehicle is located. At this time, it is necessary to slow down to avoid collision, and the deceleration driving information is output;
[0123] When the lateral displacement distance is less than the first displacement distance, it means that other vehicles are preparing to change lanes but have not yet implemented to enter the lane where the vehicle is located, which has no impact on the driving of the vehicle, and normal driving information is output.
[0124] Example 2, second aspect, please refer to Figure 2 As shown, the present invention also provides a multi-sensor target data fusion detection method for intelligent driving, comprising:
[0125] Step S1, obtaining a road picture, identifying other vehicles in the road picture; obtaining the position information of other vehicles and the vehicle, performing data fusion processing on the position information and the road picture, and outputting a direction label; Step S1 also includes the following sub-steps:
[0126] Step S101, using a camera to obtain a road picture in front of the vehicle at a first interval, and setting the top and bottom sides of the road picture as the length of the road picture;
[0127] Step S102, binarizing the road image to obtain the outline of each other vehicle in the road image; and marking the other vehicles in the road image with a regular rectangular frame;
[0128] Step S103, using a laser radar to detect position information of other vehicles in front of the vehicle and the vehicle at a first interval, wherein the position information includes a straight-line distance between the other vehicles and the vehicle and an angle between the other vehicles and the vehicle;
[0129] Step S104: sort the position information from left to right in ascending order of straight-line distance and number them, marking them as a position sequence.
[0130] Step S105, setting the number i with an initial value of 1;
[0131] Performing raw data fusion processing, the raw data fusion processing comprising:
[0132] Set the dividing line segment with the length of the road image, starting from the bottom of the road and moving along the width of the road image to the top of the road image;
[0133] It is stipulated that the bounding box of each other vehicle shall be numbered only when it intersects the dividing line segment for the first time;
[0134] When the dividing line segment intersects with the border of any other vehicle for the first time, the number of the other vehicle is set to i, and i is increased by 1;
[0135] The original data fusion process is performed again until the segmentation line moves to the top of the road image;
[0136] Step S106, performing digital-graphic data fusion processing, the digital-graphic data fusion processing includes:
[0137] Associate all other vehicles at the same time with their location information by number;
[0138] Connect the bottom midpoint and the top midpoint of the road image to get the center axis; add right lane labels to other vehicles whose borders are completely on the right side of the center axis; add the same lane labels to other vehicles whose borders intersect the center axis; add left lane labels to other vehicles whose borders are completely on the left side of the center axis;
[0139] Position information and direction labels are displayed in the bounding box of each other vehicle in the road image, and the direction labels include a left lane label, a right lane label, and a same lane label.
[0140] Step S2, performing driving prediction processing on other vehicles and outputting prediction labels; calculating the position information and outputting the split distance; Step S2 also includes the following sub-steps:
[0141] Using image processing technology to identify other vehicles in the road picture, the driving prediction process includes:
[0142] Step S201, identifying the taillight conditions of other vehicles, wherein the taillight conditions include the left turn light flashing, the right turn light flashing, and the taillight not flashing;
[0143] Step S202, when the taillight is not flashing, set a static label for other vehicles; when the taillight is flashing the left turn signal, set a left turn label for other vehicles; when the taillight is flashing the right turn signal, set a right turn label for other vehicles;
[0144] Step S203, displaying a predicted label in the frame of each other vehicle in the road image, and caching the road image; the predicted label includes a static label, a left turn label, and a right turn label;
[0145] Step S204, calculating the position information of each other vehicle using a mapping calculation formula, and outputting a lateral distance and a vertical distance;
[0146] The mapping calculation formula is configured as: ; Among them, H is the horizontal distance, L is the vertical distance, A is the angle, and zx is the straight-line distance.
[0147] Step S3, calculating the split distance, outputting the displacement distance and relative speed based on the calculation results; analyzing the direction label and the prediction label based on the displacement distance and the relative speed, and outputting the driving judgment information; Step S3 also includes the following sub-steps:
[0148] Step S3011, obtaining a road picture before a first time and marking it as a historical picture;
[0149] Step S3012, calculating the difference between the vertical distance of each identical other vehicle in the road image and the vertical distance of other vehicles in the historical image, and marking it as the vertical displacement distance;
[0150] Step S3013, calculating the ratio of the vertical displacement distance to the first time, marked as relative speed;
[0151] Step S3014, calculating the difference between the lateral distance of each identical other vehicle in the road image and the lateral distance of other vehicles in the historical image, and marking it as the lateral displacement distance.
[0152] Step S3021, analyzing other vehicles that have the same lane label and left turn label combination or the same lane label and right turn label combination;
[0153] Step S3022: when the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, output normal driving information;
[0154] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information;
[0155] When the lateral displacement distance is less than the first displacement distance and the relative speed is less than 0, the waiting deceleration information is output;
[0156] When the lateral displacement distance is less than the first displacement distance and the relative speed is greater than or equal to 0, normal driving information is output.
[0157] Step S3031, analyzing other vehicles having both a left lane label and a right turn label combination or both a right lane label and a left turn label combination;
[0158] Step S3032: when the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, output normal driving information;
[0159] When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information;
[0160] When the lateral displacement distance is less than the first displacement distance, normal driving information is output.
[0161] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0162] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0163] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
Claims
1. Multi-sensor target data fusion detection method for intelligent driving, It is characterized in that include: Obtain road images and identify other vehicles in the road images; Obtain the location information of other vehicles and the vehicle itself, perform data fusion processing on the location information and road images, and output direction labels; Perform driving prediction processing on other vehicles and output prediction labels; Calculate the position information and output the split distance; Calculate the split distance, and output the displacement distance and relative speed based on the calculation results; analyze the direction label and prediction label based on the displacement distance and relative speed, and output driving judgment information; Obtain road images and identify other vehicles in the road images; Obtaining the location information of other vehicles and the vehicle includes: At each interval, the camera is used to obtain a road image in front of the vehicle, and the top and bottom sides of the road image are defined as the length of the road image; Binarize the road image to obtain the outline of each other vehicle in the road image; and use a rectangular frame to mark other vehicles in the road image; At each first interval, the laser radar is used to detect the position information of other vehicles in front of the vehicle and the vehicle, wherein the position information includes the straight-line distance between the other vehicles and the vehicle and the angle between the other vehicles and the vehicle; The position information is sorted from left to right in ascending order of straight-line distance and numbered, and marked as a position sequence; Data fusion processing of location information and road pictures includes: Set the number i to an initial value of 1; Performing raw data fusion processing, the raw data fusion processing comprising: Set the dividing line segment with the length of the road image, starting from the bottom of the road and moving along the width of the road image to the top of the road image; It is stipulated that the bounding box of each other vehicle shall be numbered only when it intersects the dividing line segment for the first time; When the dividing line segment intersects with the border of any other vehicle for the first time, the number of the other vehicle is set to i, and i is increased by 1; The original data fusion process is performed again until the segmentation line moves to the top of the road image; Perform digital and image data fusion processing; The digital image data fusion processing includes: Associate all other vehicles at the same time with their location information by number; Connect the bottom midpoint and the top midpoint of the road image to get the center axis; add right lane labels to other vehicles whose borders are completely on the right side of the center axis; add the same lane labels to other vehicles whose borders intersect the center axis; add left lane labels to other vehicles whose borders are completely on the left side of the center axis; Displaying position information and direction labels in the frame of each other vehicle in the road image, wherein the direction labels include a left lane label, a right lane label, and a same lane label; The prediction labels include static labels, left turn labels, and right turn labels. The prediction processing is performed on other vehicles, and the prediction labels are output, including: Using image processing technology to identify other vehicles in the road picture, the driving prediction process includes: Identify the taillight conditions of other vehicles, where the taillight conditions include whether the left turn light is flashing, the right turn light is flashing, or not flashing; When the taillight condition is not flashing, set a static tag for other vehicles; When the taillight condition is that the left turn signal is flashing, set the left turn tag for other vehicles; When the taillight condition is that the right turn signal is flashing, set a right turn tag for other vehicles; Displaying a predicted label in the border of each other vehicle in the road image, and caching the road image; the predicted label includes a static label, a left turn label, and a right turn label; The split distance includes a horizontal distance and a vertical distance. The position information is calculated and the split distance is output, including: Use the mapping calculation formula to calculate the position information of each other vehicle and output the lateral distance and vertical distance; The mapping calculation formula is configured as: ; Among them, H is the horizontal distance, L is the vertical distance, A is the angle, and zx is the straight-line distance.
2. The multi-sensor target data fusion detection method for intelligent driving according to claim 1, It is characterized in that The displacement distance includes a vertical displacement distance and a lateral displacement distance. The split distance is calculated, and the displacement distance and relative speed are output based on the calculation results, including: Get the vehicle's speed; Get the road picture before the first time and mark it as a historical picture; Calculate the difference between the vertical distance of each identical other vehicle in the road image and the vertical distance of other vehicles in the historical image, and mark it as the vertical displacement distance; Calculate the ratio of the vertical displacement distance to the first time, marked as relative velocity; The difference between the lateral distance of each identical other vehicle in the road image and the lateral distance of other vehicles in the historical image is calculated and marked as the lateral displacement distance.
3. The multi-sensor target data fusion detection method for intelligent driving according to claim 2, It is characterized in that The driving judgment information includes normal driving information, deceleration driving information and waiting deceleration information. The direction label and the prediction label are analyzed based on the displacement distance and the relative speed. The output driving judgment information includes: Analyze other vehicles that have the same lane label and left turn label combination or the same lane label and right turn label combination; When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, outputting normal driving information; When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information; When the lateral displacement distance is less than the first displacement distance and the relative speed is less than 0, the waiting deceleration information is output; When the lateral displacement distance is less than the first displacement distance and the relative speed is greater than or equal to 0, normal driving information is output.
4. The multi-sensor target data fusion detection method for intelligent driving according to claim 3, It is characterized in that Based on the displacement distance and relative speed, the direction label and the prediction label are analyzed, and the output driving judgment information also includes: Analyze other vehicles that have a combination of a left lane label and a right turn label or a combination of a right lane label and a left turn label; When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is greater than or equal to 0, outputting normal driving information; When the lateral displacement distance is greater than or equal to the first displacement distance and the relative speed is less than 0, outputting deceleration driving information; When the lateral displacement distance is less than the first displacement distance, normal driving information is output.
5. A multi-sensor target data fusion detection system for intelligent driving, used to implement the multi-sensor target data fusion detection method for intelligent driving according to any one of claims 1 to 4, It is characterized in that It includes an information acquisition module, an information marking module and a driving judgment module; The information acquisition module is used to acquire road pictures, identify other vehicles in the road pictures, and acquire location information of other vehicles and the vehicle; The information labeling module is used to perform data fusion processing on the location information and the road picture and output the direction label; perform driving prediction processing on other vehicles and output the prediction label; Calculate the position information and output the split distance; The driving judgment module is used to calculate the split distance, and output the displacement distance and relative speed based on the calculation results; analyze the direction label and the prediction label based on the displacement distance and the relative speed, and output the driving judgment information.
Citation Information
Patent Citations
Multi-sensor data fusion method and device for intelligent driving, medium and equipment
CN117333746A
Device and method for detection of violation of front vehicle and early warning of violation of vehicle on road
CN106652468A
Vehicle driving auto-control method and device
CN108528432A
Intelligent vehicle lane changing collision probability evaluation method based on track prediction
CN112258893A
Safety early warning method and system based on multi-lane vehicle distance detection
CN112365741A
