Vehicle collision detection method, electronic device, vehicle, computer medium and product
By combining vehicle environmental images and sensor parameters, ultrasonic radar and inertial measurement units can be used to position and track target obstacles, which solves the problem that users in the vehicle have difficulty in detecting obstacle collisions in a timely manner, and improves the accuracy and real-timeness of collision detection.
Patent Information
- Application Number
- CN202510206794.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-24
- Publication Date
- 2025-08-19
AI Technical Summary
In the prior art, it is difficult for users in vehicles to pay attention to the small collision between external obstacles and vehicles in a timely manner, resulting in the inability to deal with collision abnormalities in time and harming the interests of users.
By combining the vehicle's environmental image and sensor parameters, ultrasonic radar and inertia measurement units can be used to position and track target obstacles, determine the collision between obstacles and vehicles, and improve the accuracy of collision detection.
It effectively reduces vehicle blind spots, improves the accuracy of detection of small bumps and high-speed collisions, the algorithm is simple and easy to deploy, and has good real-time detection, which improves the accuracy of collision detection.
Smart Images
Figure CN120507758A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and more specifically, to a vehicle collision detection method, an electronic device, a vehicle, a computer-readable storage medium, and a computer program product. Background Art
[0002] With the rapid development of the automotive industry, vehicles have become an essential part of modern transportation. Early vehicle designs primarily focused on mechanical performance and driving experience, but with the increasing number of vehicles and the increasing complexity of road conditions, traffic safety issues have become increasingly prominent. While inside a vehicle, users may not notice minor collisions with external obstacles, resulting in inability to promptly address these incidents, potentially harming their safety. Therefore, implementing vehicle collision detection is a pressing issue in related technologies. Summary of the Invention
[0003] The present application provides a vehicle collision detection method, an electronic device, a vehicle, a computer-readable storage medium, and a computer program product.
[0004] The present application provides a vehicle collision detection method, which includes:
[0005] Determine target obstacles in the vehicle's environment based on the vehicle's environment image;
[0006] A collision condition between the target obstacle and the vehicle is determined according to sensor parameters of the target obstacle and a vehicle sensor, wherein the vehicle sensor includes an ultrasonic radar and / or an inertial measurement unit.
[0007] In this way, in the vehicle collision detection method, electronic device, vehicle, computer-readable storage medium and computer program device of the embodiments of the present application, the positioning and tracking of the target obstacle are achieved by combining the vehicle's environmental image, and combining sensor parameters to determine whether the target obstacle is likely to collide with the vehicle, so as to achieve vehicle collision detection by combining sensor parameters with environmental images, thereby improving the accuracy of collision detection.
[0008] In some embodiments, determining a target obstacle in the environment of the vehicle based on the vehicle's environment image includes:
[0009] Performing target recognition on the environmental image to determine target obstacles;
[0010] The target obstacle is tracked according to the target obstacle and the previous obstacle trajectory.
[0011] In some embodiments, performing target recognition on the environment image and determining target obstacles includes:
[0012] Based on the pre-trained target recognition model, target recognition is performed on the environment image to determine target obstacles.
[0013] In some embodiments, tracking the target obstacle based on the target obstacle and the previous obstacle trajectory includes:
[0014] The target obstacle is matched with the previous obstacle trajectory to achieve target tracking.
[0015] In some embodiments, the vehicle collision detection method further includes:
[0016] Classifying the target obstacle into a high confidence class and a low confidence class according to the detection score of the target obstacle;
[0017] Performing secondary matching on the target obstacle that failed to be matched and the previous obstacle trajectory that failed to be matched in the low confidence class;
[0018] The target obstacles that failed to be matched in the high confidence class are saved as new obstacle tracks.
[0019] In some embodiments, determining a collision condition between the target obstacle and the vehicle based on sensor parameters of the target obstacle and a vehicle sensor includes:
[0020] Determining whether to enter a determination phase based on an overlap ratio between a detection frame of the target obstacle and a preset area of the vehicle;
[0021] In the determination stage, a collision condition between the target obstacle and the vehicle is determined based on the sensor parameters.
[0022] In some embodiments, determining to enter the determination phase based on the overlap ratio between the detection frame of the target obstacle and the preset area of the vehicle further includes:
[0023] determining a first distance between the target obstacle and the vehicle according to a detection frame of the target obstacle;
[0024] When the first distance is less than a first preset distance, it is determined to enter the determination phase according to an overlap rate between the detection frame of the target obstacle and a preset area of the vehicle.
[0025] In some embodiments, determining a collision condition between the target obstacle and the vehicle based on the sensor parameters includes:
[0026] Determine the collision condition between the target obstacle and the vehicle according to the acceleration parameter of the vehicle and / or the second distance between the vehicle and the target obstacle; wherein,
[0027] The acceleration parameter is determined by an inertial measurement unit of the vehicle, and the second distance is obtained by collecting information using an ultrasonic radar of the vehicle.
[0028] In some embodiments, determining a collision condition between the target obstacle and the vehicle based on an acceleration parameter of the vehicle and / or a second distance between the vehicle and the target obstacle includes:
[0029] When the acceleration parameter is greater than a preset acceleration threshold, determining that the vehicle collides with the target obstacle; or
[0030] When the second distance is less than a second preset distance, it is determined that the vehicle collides with the target obstacle.
[0031] In some embodiments, determining to enter the determination phase based on an overlap ratio between the detection frame of the target obstacle and a preset area of the vehicle includes:
[0032] When the current overlap ratio is greater than the preset overlap ratio, the count value is increased by one;
[0033] When the current overlap ratio is greater than the preset overlap ratio, the count value is reduced by one;
[0034] When the count value is greater than a first preset threshold, it is determined to enter the determination stage; wherein the current overlap rate is the overlap rate between the detection frame of the current target obstacle in the current environment image and the preset area of the vehicle.
[0035] In some embodiments, the vehicle collision detection method further includes:
[0036] In the determination phase, when the count value is less than a second preset threshold, it is determined to exit the determination phase.
[0037] An embodiment of the present application provides an electronic device, which includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the method in any of the above embodiments are implemented.
[0038] An embodiment of the present application provides a vehicle, which includes the electronic device according to the above embodiment.
[0039] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method of any of the above embodiments are implemented.
[0040] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of any of the above embodiments when the computer program is executed by a processor.
[0041] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0043] Figure 1 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0044] Figure 2 is a schematic diagram of target recognition and target tracking processing results of an environment image in certain embodiments of the present application;
[0045] Figure 3 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0046] Figure 4 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0047] Figure 5 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0048] Figure 6 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0049] Figure 7 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0050] Figure 8 is a schematic diagram of target recognition and target tracking processing results of an environment image in certain embodiments of the present application;
[0051] Figure 9 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0052] Figure 10 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0053] Figure 11 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0054] Figure 12is a schematic diagram of target recognition and target tracking processing results of an environment image in certain embodiments of the present application;
[0055] Figure 13 is a flow chart of a vehicle collision detection method according to certain embodiments of the present application;
[0056] Figure 14 It is a flowchart of a vehicle collision detection method according to certain embodiments of the present application. DETAILED DESCRIPTION
[0057] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.
[0058] With the rapid development of the automotive industry, vehicles have become an essential part of modern transportation. Early vehicle designs primarily focused on mechanical performance and driving experience, but with the increasing number of vehicles and the increasing complexity of road conditions, traffic safety issues have become increasingly prominent. While inside a vehicle, users may not notice minor collisions with external obstacles, resulting in inability to promptly address these incidents, potentially harming their safety. Therefore, implementing vehicle collision detection is a pressing issue in related technologies.
[0059] Based on the above possible problems, please refer to Figure 1 and Figure 2 , an embodiment of the present application provides a vehicle collision detection method, the vehicle collision detection method comprising:
[0060] 01: Determine the target obstacle 10 in the vehicle's environment based on the vehicle's environment image;
[0061] 02: Determine a collision condition between the target obstacle 10 and the vehicle based on sensor parameters of the target obstacle 10 and the vehicle sensor, where the vehicle sensor includes an ultrasonic radar and / or an inertial measurement unit.
[0062] Embodiments of the present application provide an electronic device comprising one or more processors and a memory, wherein the memory stores a computer program executable by the processor. The processor may be configured to: determine a target obstacle 10 in the vehicle's environment based on an image of the vehicle's environment; and determine a collision condition between the target obstacle 10 and the vehicle based on sensor parameters of the target obstacle 10 and the vehicle's sensors, wherein the vehicle sensors may include an ultrasonic radar and / or an inertial measurement unit.
[0063] Embodiments of the present application provide a collision detection device, comprising a first determination module and a second determination module. The first determination module can be configured to determine a target obstacle 10 in the vehicle's environment based on an image of the vehicle's environment; the second determination module can be configured to determine a collision between the target obstacle 10 and the vehicle based on sensor parameters of the target obstacle 10 and the vehicle's sensors, wherein the vehicle sensors include ultrasonic radar and / or an inertial measurement unit.
[0064] Specifically, the vehicle's environmental image can be a frame image from a video captured by the vehicle's camera. The camera can be located on the front, rear, left, or right side of the vehicle, and is used to capture environmental images in the corresponding direction of the vehicle. The environmental image includes environmental objects in the vehicle's environment. For example, the environmental image may include pedestrians, buildings, trees, vegetation, traffic signs, streetlights, and other elements.
[0065] Target obstacles 10 are objects that could potentially collide with the vehicle. Examples include objects within a predetermined area 20 of the vehicle, objects relatively close to the vehicle, and objects rapidly approaching the vehicle. By identifying and analyzing the vehicle's surrounding image, target obstacles 10 within the vehicle's environment can be identified.
[0066] Vehicle sensors include ultrasonic radar (Ultrasonic Sensor System, USS) and / or inertial measurement unit (IMU). USS can be used to obtain distance information between the target obstacle 10 and the vehicle, with good reliability and accuracy within 3m. The vehicle's wide-angle camera has a blind spot in the area of 0m to 3m near the vehicle. However, when an obstacle collides with the vehicle, the distance to the vehicle is extremely close. Therefore, if only a camera is used for collision detection, the accuracy is low. Combining USS with a camera for collision detection can minimize the vehicle's blind spot.
[0067] The IMU can capture the vehicle's acceleration. Preliminary experiments established the normal range of IMU acceleration in all directions during normal driving. A collision is detected when the IMU value exceeds this range. Integrating the IMU for collision detection can, to a certain extent, address the shortcomings of high-speed collision detection with USS and cameras alone.
[0068] Therefore, the embodiments of this application combine cameras, IMUs, and USSs for collision detection, resulting in a smaller blind spot than when using only cameras, IMUs, or USSs. Furthermore, they provide relatively stable detection results for both minor bumps and high-speed collisions. The algorithm is relatively simple and easy to deploy on vehicles, providing real-time detection and effectively improving the accuracy of collision detection for the vehicle itself.
[0069] In this way, in the vehicle collision detection method, electronic device, vehicle, computer-readable storage medium and computer program device of the embodiments of the present application, the positioning and tracking of the target obstacle 10 are achieved by combining the vehicle's environmental image, and combining sensor parameters to determine whether the target obstacle 10 is likely to collide with the vehicle, so as to achieve vehicle collision detection by combining sensor parameters with environmental images, thereby improving the accuracy of collision detection.
[0070] See also Figure 2 and Figure 3 In some embodiments, step 01, determining a target obstacle 10 in the environment of the vehicle based on the vehicle's environment image, includes:
[0071] 011: Perform target recognition on the environmental image and determine the target obstacle 10;
[0072] 012: According to the target obstacle 10 and the previous obstacle trajectory, the target obstacle 10 is tracked.
[0073] In some embodiments, the processor may be configured to: perform target recognition on the environment image to determine the target obstacle 10; and perform target tracking on the target obstacle 10 based on the target obstacle 10 and a previous obstacle trajectory.
[0074] In some embodiments, the first determination module includes a recognition submodule and a tracking submodule. The recognition submodule can be used to perform target recognition on the environment image to determine the target obstacle 10; the tracking submodule can be used to track the target obstacle 10 based on the target obstacle 10 and the previous obstacle trajectory.
[0075] Specifically, target recognition is performed on the elements included in the environmental target to determine the target obstacle 10. The target obstacle 10 can be other vehicles, pedestrians, buildings, traffic devices, etc.
[0076] Based on the target obstacle 10 and the previous obstacle trajectory, target obstacle 10 can be tracked. The previous obstacle trajectory refers to the target obstacle 10 identified in the previous frame of the environment image. For example, by matching the target obstacle 10 with the previous obstacle trajectory, if a match is found, it can be determined that the target trajectory also appeared in the previous frame. If the target obstacle 10 appears in multiple frames, it can be considered that the target obstacle 10 has been accurately identified.
[0077] Combining target recognition and tracking to process environmental images enhances the ability to detect obscured objects. The target tracking phase improves the algorithm's recognition rate for partially obscured obstacles and moving objects. It also stabilizes the detection frame, facilitating subsequent obstacle distance estimation.
[0078] For example, if target obstacle 10 is a pedestrian, the pedestrian is identified in the first frame of the environmental image; in the second frame, the pedestrian is obscured by vegetation and cannot be identified; in the third frame, the pedestrian exits the obscured area and can be identified. In this case, if only target recognition is performed, the pedestrian may not be accurately identified in the third frame. However, if a solution combining target recognition and target tracking is adopted, the pedestrian's detection frame will be retained even if the pedestrian is obscured in the second frame. When the pedestrian reappears in the third frame, it can be matched with the previously detected frame, allowing for quick and accurate identification, thereby improving the accuracy of detecting target obstacle 10.
[0079] In this way, by combining the target recognition and target tracking solutions, the accuracy of detecting the target obstacle 10 can be improved, thereby further improving the accuracy of collision detection.
[0080] See also Figure 4 In some embodiments, step 011, performing target recognition on the environment image to determine the target obstacle 10, includes:
[0081] 0111: Based on the pre-trained target recognition model, perform target recognition on the environmental image and determine the target obstacle 10.
[0082] In some embodiments, the processor may be configured to perform target recognition on the environment image based on a pre-trained target recognition model to determine the target obstacle 10 .
[0083] In some embodiments, the recognition submodule includes a recognition unit that can be used to perform target recognition on the environment image based on a pre-trained target recognition model to determine the target obstacle 10 .
[0084] Specifically, the target recognition model can be a YOLO model, an SSD model, an R-CNN model, etc., which can be selected according to actual needs and is not limited here. Due to the lightweight and high real-time performance of YOLOv7-Tiny, in this embodiment, the target recognition model is the YOLOv7-Tiny model.
[0085] The environmental images captured by the camera undergo preprocessing, including image correction and noise reduction, and are then fed into a pretrained YOLOv7-Tiny model for object recognition. The YOLOv7-Tiny model is trained using a training dataset to refine the pretrained weights. Traffic signs such as pedestrians, vehicles, water barriers, and cones are pre-labeled in the training dataset. The trained YOLOv7-Tiny model becomes more familiar with common obstacles in the driving scene, achieving higher efficiency and accuracy in object recognition.
[0086] In this way, by performing target recognition on the environmental image based on the pre-trained target recognition model, the target obstacle 10 in the environmental image can be recognized more efficiently and accurately.
[0087] See also Figure 5 In some embodiments, step 012, tracking the target obstacle 10 according to the target obstacle 10 and the previous obstacle trajectory, includes:
[0088] 0121: Match the target obstacle 10 with the previous obstacle trajectory to achieve target tracking.
[0089] In some embodiments, the processor may be configured to match target obstacle 10 with previous obstacle trajectories to achieve target tracking.
[0090] In some embodiments, the tracking submodule includes a matching unit that can be used to match the target obstacle 10 with a previous obstacle trajectory to achieve target tracking.
[0091] Specifically, the prior obstacle trajectory is the detection box of the target obstacle 10 identified in the previous frame of the environmental image. The target obstacle 10 identified in the current frame of the environmental image is matched with the detection box of the prior obstacle trajectory. During the matching process, the similarity between the detection box of the target obstacle 10 and the Kalman filter estimation result of the prior obstacle trajectory can be used as a metric to measure the degree of matching. Intersection over Union (IoU) or Re-ID feature distance can be used as similarity metrics.
[0092] In one embodiment, the overlap ratio between the detection frame of the target obstacle 10 and the Kalman filter estimation result of the previous obstacle trajectory is calculated as the similarity between the target obstacle 10 and the Kalman filter estimation result of the previous obstacle trajectory, and then the Hungarian algorithm is sampled for matching.
[0093] In this way, by matching the target obstacle 10 with the previous obstacle trajectory, target tracking of the target obstacle 10 can be achieved.
[0094] See also Figure 6 In some embodiments, the vehicle collision detection method further includes:
[0095] 03: According to the detection score of target obstacle 10, target obstacle 10 is classified into high confidence class and low confidence class;
[0096] 04: Perform secondary matching on the target obstacle 10 that failed to match in the low confidence class and the previous obstacle trajectory that failed to match;
[0097] 05: Save the target obstacle 10 that failed to be matched in the high confidence class as a new obstacle trajectory.
[0098] In some embodiments, the processor can be used to: classify the target obstacle 10 into a high-confidence class and a low-confidence class based on the detection score of the target obstacle 10; perform a secondary matching on the target obstacle 10 that failed to match in the low-confidence class and the previous obstacle trajectory that failed to match; and save the target obstacle 10 that failed to match in the high-confidence class as a new obstacle trajectory.
[0099] In certain embodiments, the collision detection device further includes a classification module, a matching module, and a storage module. The classification module may be configured to classify the target obstacle 10 into a high-confidence class or a low-confidence class based on the detection score of the target obstacle 10; the matching module may be configured to perform a secondary matching between the target obstacle 10 in the low-confidence class that failed to match and the previous obstacle trajectory that failed to match; and the storage module may be configured to save the target obstacle 10 in the high-confidence class that failed to match as a new obstacle trajectory.
[0100] Specifically, the ByteTrack algorithm can be used to track the target obstacle 10. First, the target obstacle 10 is classified based on the detection score obtained when identifying the target obstacle 10. The detection score includes accuracy, recall, F1 score, average precision, etc. For example, if the detection score is within a first set score range, the target obstacle 10 is classified as a high-confidence class; if the detection score is within a second set score range, the target obstacle 10 is classified as a low-confidence class. The first set score range and the second set score range can be set according to actual circumstances.
[0101] After matching the target obstacle 10 with the previous obstacle trajectory, retain the high-confidence detection boxes that did not match the trajectory, as well as the previous obstacle trajectory that did not match the target obstacle 10. The high-confidence target obstacle 10 that did not match the trajectory is saved as a new obstacle trajectory. A second matching is performed on the previous obstacle trajectory that failed to match the detection box of the low-confidence target obstacle 10.
[0102] If the secondary matching of the target obstacle 10 of the low confidence class still fails, the detection frame of the target obstacle 10 is considered to be a background that does not contain any object and is deleted.
[0103] In one embodiment, a detector is used to obtain a detection frame and a corresponding detection score of the target obstacle 10, and the detection frame of the target obstacle 10 is classified. If the score is higher than a first threshold value T_high, the detection frame is classified as a high-confidence class. If the score is lower than the first threshold value T_high and higher than a second threshold value T_low, the detection frame of the target obstacle 10 is classified as a low-confidence class.
[0104] For those prior obstacle trajectories and low-confidence target obstacle 10 detection boxes that failed the first match, retain the prior obstacle trajectories that still did not match the bounding box after the second match. Delete the low-confidence target obstacle 10 for which no corresponding target obstacle 10 trajectory was found after the second match. Finally, save the high-confidence target obstacle 10 that did not match the corresponding target obstacle 10 trajectory in the first match as the newly appeared trajectory.
[0105] In this way, by classifying the target obstacles 10 and performing secondary matching on the target obstacles 10 that failed to match the low-confidence class, the target obstacles 10 that failed to match the high-confidence class are retained as new obstacle trajectories, thereby achieving target tracking of the target obstacles 10.
[0106] See also Figure 7 and Figure 8 In some embodiments, step 02, determining a collision condition between the target obstacle 10 and the vehicle based on sensor parameters of the target obstacle 10 and the vehicle sensor, includes:
[0107] 021: Determine to enter the judgment phase based on the overlap rate between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle;
[0108] 022: In the determination phase, the collision condition between the target obstacle 10 and the vehicle is determined based on the sensor parameters.
[0109] In some embodiments, the processor may be configured to: determine entry into a determination phase based on an overlap ratio between a detection frame of the target obstacle 10 and a preset area 20 of the vehicle; and, in the determination phase, determine a collision condition between the target obstacle 10 and the vehicle based on sensor parameters.
[0110] In some embodiments, the second determination module may be used in both the first determination submodule and the second determination submodule. The first determination submodule may be used to determine whether to enter the determination phase based on the overlap ratio between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle. The second determination submodule may be used to determine the collision between the target obstacle 10 and the vehicle based on sensor parameters during the determination phase.
[0111] Specifically, the decision phase is entered based on the overlap between the detection frame of the target obstacle 10 and the vehicle's preset area 20. The preset area 20 is a pre-calibrated area around the vehicle, which can be pre-calibrated based on the vehicle's dimensions, camera, and vehicle sensor parameters.
[0112] When the overlap rate between the preset area 20 of the vehicle and the detection frame of the target obstacle 10 is large, it can be considered that the target obstacle 10 is located in the preset area 20 of the vehicle, and the determination stage can be entered at this time.
[0113] In the determination phase, it is determined whether the target obstacle 10 collides with the vehicle based on the sensor parameters transmitted from the USS or the IMU.
[0114] Because vehicle collision detection based on vehicle sensor parameters requires significant computing power and consumes significant computing resources, maintaining the determination phase results in high computational costs. If the target obstacle 10 is absent from the preset area 20, the probability of a collision is likely low, and therefore the determination phase can be skipped. The determination phase is only entered when the target obstacle 10 persists within the preset area 20, thus saving computing power. Using camera-captured environmental images to trigger the determination phase, followed by a USS or IMU for early warning and arbitration, can improve the reliability of collision detection.
[0115] In one embodiment, all detection frames of target obstacles 10 with distance information are traversed. If a detection frame exists with a distance less than 6 meters and an IoU greater than 0.5 between the detection frame and the preset area 20, it is considered that a target obstacle 10 with a potential collision exists. If a target obstacle 10 with a potential collision exists in several consecutive frames of the environment image, a judgment state is entered.
[0116] In this way, by determining the overlap rate between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle, entering the judgment stage, and determining the collision situation between the target obstacle 10 and the vehicle based on the sensor parameters, multiple sensors can be used for collision detection when the probability of collision is high, reducing computing power consumption.
[0117] See also Figure 9 In some embodiments, step 021, determining to enter the determination phase based on the overlap rate between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle, further includes:
[0118] 0211: Determine a first distance between the target obstacle 10 and the vehicle based on the detection frame of the target obstacle 10;
[0119] 0212: When the first distance is less than the first preset distance, it is determined to enter the judgment phase according to the overlap rate between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle.
[0120] In some embodiments, the processor may be configured to: determine a first distance between the target obstacle 10 and the vehicle based on a detection frame of the target obstacle 10; and determine to enter a determination phase based on an overlap ratio between the detection frame of the target obstacle 10 and a preset area 20 of the vehicle when the first distance is less than a first preset distance.
[0121] In some embodiments, the first determination submodule includes a first determination unit and a second determination unit. The first determination unit may be configured to determine a first distance between the target obstacle 10 and the vehicle based on a detection frame of the target obstacle 10; and the second determination unit may be configured to determine whether to enter the determination phase based on an overlap ratio between the detection frame of the target obstacle 10 and a preset area 20 of the vehicle when the first distance is less than a first preset distance.
[0122] Specifically, a first distance between the target obstacle 10 and the vehicle can be determined based on the environmental image to detect the distance of the target obstacle 10 and screen for target obstacles 10 that are closer to the vehicle. This first distance can be measured using a monocular ranging method. For example, forward-looking monocular ranging can effectively screen out target obstacles 10 with a high collision risk within a range of 3 to 6 meters, thereby increasing collision detection accuracy.
[0123] If the first distance is less than the first preset distance, target obstacle 10 is relatively close to the vehicle. In this case, the overlap ratio between the detection frame of target obstacle 10 and the vehicle's preset area 20 is used to determine whether target obstacle 10 is within preset area 20, thereby entering the determination phase. The first preset distance is a preset distance value that can be set based on actual circumstances. For example, the first preset distance can be 5m, 5.5m, 6m, 6.5m, 7m, etc., without limitation.
[0124] In one embodiment, a pinhole imaging model is used to estimate the distance between the target obstacle 10 and the vehicle based on the bottom edge of the detection frame of the target obstacle 10. The pinhole imaging model is one of the fundamental theoretical models in monocular ranging. By leveraging the principle of similar triangles and relevant camera parameters, the actual distance of an object from the camera can be inferred based on the pixel size of the object in the image. For example, relevant camera parameters include focal length and sensor size. Experimental analysis shows that the error accuracy of monocular ranging is less than 8% within the range of 3m to 13m.
[0125] It should be noted that when calculating the first distance, it is necessary to pay attention to the impact of factors such as camera distortion, lighting conditions, and object recognition and positioning on the ranging accuracy.
[0126] In this way, when the first distance is less than the first preset distance, the entry into the determination phase is determined based on the overlap rate between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle. This can improve the success rate of detecting the target obstacle 10 within the preset area 20, thereby improving the accuracy of the judgment on entering the determination phase.
[0127] See also Figure 10 In some embodiments, step 022, determining a collision condition between the target obstacle 10 and the vehicle based on sensor parameters, includes:
[0128] 0221: Determine a collision condition between the target obstacle 10 and the vehicle based on the acceleration parameter of the vehicle and / or the second distance between the vehicle and the target obstacle 10; wherein,
[0129] The acceleration parameter is determined by the vehicle's inertial measurement unit, and the second distance is obtained based on the vehicle's ultrasonic radar.
[0130] In some embodiments, the processor may be configured to determine a collision condition between the target obstacle 10 and the vehicle based on an acceleration parameter of the vehicle and / or a second distance between the vehicle and the target obstacle 10 .
[0131] In some embodiments, the second determining submodule includes a third determining unit configured to determine a collision condition between the target obstacle 10 and the vehicle based on an acceleration parameter of the vehicle and / or a second distance between the vehicle and the target obstacle 10 .
[0132] Specifically, during the determination phase, the parameters of the IMU and USS are monitored in real time.
[0133] For the USS, the direct and indirect distances obtained from all USSs on the vehicle are aggregated to select the most reliable distance values. Among these highly reliable distance values, the minimum value of the second distance between the vehicle and the target obstacle 10 since entering the determination phase is determined as the second distance uss_dis_xmin.
[0134] For the IMU, the acceleration in the IMU information on the vehicle is summarized, and the maximum acceleration of the vehicle after entering the judgment phase is maintained as the acceleration parameter imu_acc_xmax.
[0135] It should be noted that the actual IMU and USS values are noisy. To reduce the impact of data jitter on detection, the sensor parameters and their historical data can be smoothed before determining their respective extreme values. For example, after performing a sliding window average smoothing on the IMU and USS parameters, the second distance uss_dis_xmin and acceleration parameter imu_acc_xmax can be determined.
[0136] Whether the target obstacle 10 and the vehicle have collided can be determined based on the vehicle's acceleration parameter and / or the second distance between the vehicle and the target obstacle 10. For example, if the acceleration parameter is large, it can be determined that the target obstacle 10 and the vehicle have collided. Alternatively, if the second distance is small, it can be determined that the target obstacle 10 and the vehicle have collided.
[0137] In this way, according to the acceleration parameter of the vehicle and / or the second distance between the vehicle and the target obstacle 10 , the collision condition between the target obstacle 10 and the vehicle can be determined, thereby realizing collision detection of the vehicle.
[0138] See also Figure 11 and Figure 12 In some embodiments, step 0221, determining a collision condition between the target obstacle 10 and the vehicle based on an acceleration parameter of the vehicle and / or a second distance between the vehicle and the target obstacle 10, includes:
[0139] 02211: When the acceleration parameter is greater than a preset acceleration threshold, it is determined that the vehicle collides with the target obstacle 10; or,
[0140] 02212: When the second distance is less than the second preset distance, it is determined that the vehicle collides with the target obstacle 10.
[0141] In some embodiments, the processor may be configured to: determine that the vehicle has collided with the target obstacle 10 when the acceleration parameter is greater than a preset acceleration threshold; and determine that the vehicle has collided with the target obstacle 10 when the second distance is less than a second preset distance.
[0142] In some embodiments, the third determining unit includes a first determining subunit and a second determining subunit. The first determining subunit may be configured to determine that the vehicle has collided with the target obstacle 10 when the acceleration parameter is greater than a preset acceleration threshold, and the second determining subunit may be configured to determine that the vehicle has collided with the target obstacle 10 when the second distance is less than a second preset distance.
[0143] Specifically, when the acceleration parameter imu_acc_xmax is greater than a preset acceleration threshold, the acceleration of the vehicle exceeds the maximum acceleration that the vehicle can reach. At this time, it can be considered that the vehicle has shaken violently, that is, the vehicle has collided.
[0144] When the second distance uss_dis_xmin is less than the second preset distance, the target obstacle 10 is very close to the vehicle, and it can be determined that the vehicle collides with the target obstacle 10 .
[0145] In one embodiment, it is determined whether the acceleration parameter imu_acc_xmax is greater than a preset acceleration threshold T1 and whether the second distance uss_dis_xmin is less than a second preset distance T2. If one of the above conditions is met, it is considered that the vehicle has collided and a result of the vehicle collision is output.
[0146] In one embodiment, see Figure 2 、 Figure 8 and Figure 12 The middle box is the preset area 20, and different colors can be used to indicate different states. For example, green indicates the trigger phase, during which the camera recognizes the target obstacle 10. Yellow indicates the judgment phase. Red indicates a collision between the target obstacle 10 and the vehicle.
[0147] In another embodiment, the closest detection distance of the USS used is 0.2 meters, so the second preset distance T2 is set to 0.2. After the sudden braking and accelerator pedaling experiments, the maximum acceleration of the IMU when the vehicle is driving normally is 5m / s. 2 , so the preset acceleration threshold T1 is 5m / s 2 When the vehicle acceleration is greater than the preset acceleration threshold, the vehicle acceleration exceeds the acceleration that can be generated by normal driving, and it can be considered that a collision has occurred.
[0148] In addition, when the acceleration parameter is less than or equal to the preset acceleration threshold and the second distance is greater than or equal to the second preset distance, it is determined that the vehicle and the target obstacle 10 have not collided.
[0149] In this manner, based on the comparison results of the acceleration parameter and the preset acceleration threshold, and the comparison results of the second distance and the second preset distance, a collision between the vehicle and the target obstacle 10 can be determined. Furthermore, if the acceleration parameter is greater than the preset acceleration threshold, or if the second distance is less than the second preset distance, a collision between the vehicle and the target obstacle 10 is determined, thereby achieving a vehicle collision determination.
[0150] See also Figure 13 In some embodiments, in step 021 or step 0212, the process of entering the determination phase is determined based on the overlap rate between the detection frame of the target obstacle 10 and the preset area 20 of the vehicle, including:
[0151] 0213: When the current overlap ratio is greater than the preset overlap ratio, the count value is increased by one;
[0152] 0214: When the current overlap ratio is greater than the preset overlap ratio, the count value is reduced by one;
[0153] 0215: When the count value is greater than the first preset threshold, it is determined to enter the judgment stage; wherein the current overlap rate is the overlap rate between the detection frame of the current target obstacle 10 in the current environment image and the preset area 20 of the vehicle.
[0154] In some embodiments, the processor may be configured to: increase the count value by one when the current overlap rate is greater than the preset overlap rate; decrease the count value by one when the current overlap rate is greater than the preset overlap rate; and determine to enter the determination phase when the count value is greater than a first preset threshold.
[0155] In some embodiments, the first determination submodule includes a first counting unit, a second counting unit, and a fourth determining unit. The first counting unit may be configured to increment a count value by one when the current overlap ratio is greater than a preset overlap ratio; the second counting unit may be configured to decrement a count value by one when the current overlap ratio is greater than the preset overlap ratio; and the fourth determining unit may be configured to determine that a determination phase is entered when the count value is greater than a first preset threshold.
[0156] Specifically, the current overlap ratio is the overlap ratio between the detection frame of the current target obstacle 10 in the current environment image and the preset area 20 of the vehicle. The overlap ratio is calculated for each frame of the environment image and compared with the preset overlap ratio. The count value is processed based on the comparison result to count the presence of the target obstacle 10 in the preset area 20 in consecutive frames.
[0157] If the current overlap ratio is greater than the preset overlap ratio, the target obstacle 10 is present in the preset area 20, and the count value is incremented by one. If the current overlap ratio is less than the preset overlap ratio, the target obstacle 10 is not present in the preset area 20, and the count value is decremented by one. By processing the count value, it is possible to determine whether the target obstacle 10 is present in the preset area 20 in the continuous frame environment image based on the count value.
[0158] If the count value is greater than a first preset threshold, and if the target obstacle 10 is present within the preset area 20 for several consecutive frames, the process enters a determination phase to determine the vehicle's collision condition based on the sensor parameters. The first preset threshold is a preset value that can be set based on actual needs.
[0159] In one embodiment, the preset area 20 of the vehicle is calibrated, such as Figure 3 As shown in . The detection frames of target obstacle 10 with distance information are traversed. If a target frame exists within 6 meters of the vehicle and its IoU with the vehicle's preset area 20 is greater than 0.5, the count value count is incremented by 1; otherwise, the count value count is decremented by 1. The count value count ranges from 0 to 60. The first preset threshold is 10. If the count value count>10, the decision phase begins. The camera acquisition frame rate is 30Hz. If the target obstacle 10 is stably present for approximately 0.3 consecutive times, the decision phase begins.
[0160] In this manner, based on the comparison result between the current overlap ratio and the preset overlap ratio, the count value is processed to determine whether the target obstacle 10 exists within the preset region 20 in the consecutive frame images. Furthermore, if the count value is greater than a first preset threshold, it is determined that the target obstacle 10 exists within the preset region 20 in all consecutive frames, and the determination phase is entered, thereby triggering the entry into the determination phase.
[0161] See also Figure 14In some embodiments, the vehicle collision detection method further includes:
[0162] 06: In the determination phase, if the count value is less than the second preset threshold, it is determined to exit the determination phase.
[0163] In some embodiments, the processor may be configured to determine to exit the determination phase when the process is in the determination phase and the count value is less than a second preset threshold.
[0164] In some embodiments, the configuration detection device includes a third determination module. The third determination module can be configured to determine to exit the determination phase when the count value is less than a second preset threshold value during the determination phase.
[0165] Specifically, during the determination phase, the count value is compared with a second preset threshold. If the count value is less than the second preset threshold, it can be assumed that the target obstacle 10 has not been within the vehicle's preset area 20 for several consecutive frames. No collision detection by the sensor is required, and the determination phase can be exited. The second preset threshold is a pre-set value that can be adjusted based on actual needs.
[0166] In one embodiment, during the determination phase, it is additionally determined whether the count value count is less than 1. If the count value count<1, the determination phase is exited.
[0167] In this way, when in the determination phase and the count value is less than the second preset threshold, there is no need to perform collision detection based on the sensor parameters, and it can be determined to exit the determination phase, thereby triggering the exit from the determination phase.
[0168] An embodiment of the present application provides a vehicle, which includes the electronic device according to the above embodiment.
[0169] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method of any of the above embodiments are implemented.
[0170] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of any of the above embodiments when the computer program is executed by a processor.
[0171] In the description of this specification, the reference terms "one embodiment," "some embodiments," "illustrative embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.
[0172] Furthermore, the term "connection" should be interpreted broadly. For example, it can include fixed connection, detachable connection, or integral connection; it can include direct connection, indirect connection through an intermediate medium, and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.
[0173] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.
[0174] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, as should be understood by those skilled in the art to which the embodiments of the present application pertain.
[0175] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A vehicle collision detection method, characterized in that: The vehicle collision detection method comprises: Determine target obstacles in the vehicle's environment based on the vehicle's environment image; A collision condition between the target obstacle and the vehicle is determined according to sensor parameters of the target obstacle and a vehicle sensor, wherein the vehicle sensor includes an ultrasonic radar and / or an inertial measurement unit.
2. The vehicle collision detection method according to claim 1, characterized in that: The determining of a target obstacle in the environment of the vehicle according to the environment image of the vehicle includes: Performing target recognition on the environmental image to determine target obstacles; The target obstacle is tracked according to the target obstacle and the previous obstacle trajectory.
3. The vehicle collision detection method according to claim 2, characterized in that: The performing target recognition on the environment image and determining target obstacles includes: Based on the pre-trained target recognition model, target recognition is performed on the environment image to determine target obstacles.
4. The vehicle collision detection method according to claim 2, characterized in that: Tracking the target obstacle according to the target obstacle and the previous obstacle trajectory includes: The target obstacle is matched with the previous obstacle trajectory to achieve target tracking.
5. The vehicle collision detection method according to claim 4, characterized in that: The vehicle collision detection method further includes: Classifying the target obstacle into a high confidence class and a low confidence class according to the detection score of the target obstacle; Performing secondary matching on the target obstacle that failed to be matched and the previous obstacle trajectory that failed to be matched in the low confidence class; The target obstacles that failed to be matched in the high confidence class are saved as new obstacle tracks.
6. The vehicle collision detection method according to claim 1, characterized in that: The determining, based on the target obstacle and sensor parameters of the vehicle sensor, a collision condition between the target obstacle and the vehicle, includes: Determining whether to enter a determination phase based on an overlap ratio between a detection frame of the target obstacle and a preset area of the vehicle; In the determination stage, a collision condition between the target obstacle and the vehicle is determined based on the sensor parameters.
7. The vehicle collision detection method according to claim 6, characterized in that: The step of determining to enter the determination phase based on the overlap ratio between the detection frame of the target obstacle and the preset area of the vehicle further includes: determining a first distance between the target obstacle and the vehicle according to a detection frame of the target obstacle; When the first distance is less than a first preset distance, it is determined to enter the determination phase according to an overlap rate between the detection frame of the target obstacle and a preset area of the vehicle.
8. The vehicle collision detection method according to claim 6, characterized in that: The determining, based on the sensor parameters, a collision condition between the target obstacle and the vehicle includes: Determine the collision condition between the target obstacle and the vehicle according to the acceleration parameter of the vehicle and / or the second distance between the vehicle and the target obstacle; wherein, The acceleration parameter is determined by an inertial measurement unit of the vehicle, and the second distance is obtained by collecting information using an ultrasonic radar of the vehicle.
9. The vehicle collision detection method according to claim 8, characterized in that: The determining, based on the acceleration parameter of the vehicle and / or the second distance between the vehicle and the target obstacle, a collision condition between the target obstacle and the vehicle includes: When the acceleration parameter is greater than a preset acceleration threshold, determining that the vehicle collides with the target obstacle; or When the second distance is less than a second preset distance, it is determined that the vehicle collides with the target obstacle.
10. The vehicle collision detection method according to claim 6 or 7, characterized in that: The step of determining whether to enter the determination phase based on the overlap rate between the detection frame of the target obstacle and the preset area of the vehicle includes: When the current overlap ratio is greater than the preset overlap ratio, the count value is increased by one; When the current overlap ratio is greater than the preset overlap ratio, the count value is reduced by one; When the count value is greater than a first preset threshold, it is determined to enter the determination stage; wherein the current overlap rate is the overlap rate between the detection frame of the current target obstacle in the current environment image and the preset area of the vehicle.
11. The vehicle collision detection method according to claim 10, characterized in that: The vehicle collision detection method further includes: In the determination phase, when the count value is less than a second preset threshold, it is determined to exit the determination phase.
12. An electronic device, characterized in that: The electronic device includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 11 are implemented.
13. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 12.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.