An obstacle prompting method, system, vehicle and storage medium

Abstract

The application provides an obstacle prompting method, system, vehicle and storage medium. The vehicle is provided with a first sensor and a second sensor, and the sensing range of the second sensor covers the sensing blind area of the first sensor. When a trigger condition is triggered, sensing data of the surrounding environment of the vehicle sensed by the second sensor is obtained. Then, based on the sensing data, information of the obstacle in the sensing blind area can be determined. Finally, based on the obstacle information, prompting information can be output to prompt the driver. Since the second sensor can sense the obstacle in the sensing blind area of the first sensor, the driver can be timely reminded of the obstacle in the blind area, omnidirectional obstacle detection outside the vehicle is realized, and the driving safety of the vehicle is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to an obstacle warning method, system, vehicle, and storage medium. Background Technology

[0002] With the increasing prevalence of automobiles, vehicle-related safety incidents are attracting more and more attention. To improve driving safety, vehicles are typically equipped with various sensors for obstacle detection. However, these sensors often have blind spots, failing to detect obstacles in all directions around the vehicle, thus posing a safety hazard. Summary of the Invention

[0003] This application provides an obstacle warning method, system, vehicle, and storage medium to improve vehicle driving safety.

[0004] According to a first aspect of this application, an obstacle warning method is provided, applied to a vehicle, the vehicle being equipped with a first sensor and a second sensor, the sensing range of the second sensor covering the sensing blind zone of the first sensor; the method includes:

[0005] In response to the triggering condition being triggered, the sensing information obtained by the second sensor sensing the surrounding environment of the vehicle is acquired;

[0006] Based on the sensing information, obstacle information within the sensing blind zone is determined;

[0007] Based on the obstacle information, a prompt message is output.

[0008] In some examples, the vehicle is equipped with an independently powered signal detection device; the triggering conditions include:

[0009] Before the vehicle is powered on, the signal detection device detects the signal emitted by the vehicle's control components.

[0010] The control component is used to control the vehicle to unlock and / or power on.

[0011] In some examples, the triggering condition also includes:

[0012] Based on the detected signal, it is determined that the distance between the control component and the vehicle is less than a preset first distance threshold.

[0013] In some examples, the process of acquiring sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle in response to a triggering condition includes:

[0014] In response to the triggering condition being triggered, the second sensor is powered on, and sensing information obtained by the second sensor sensing the surrounding environment of the vehicle after being powered on is acquired.

[0015] In some examples, the triggering conditions include:

[0016] The distance between the target object and the vehicle is less than a preset second distance threshold, and the vehicle's driving information meets preset conditions;

[0017] The target object includes obstacles and / or other vehicles that are at least partially within the sensing range of the first sensor; the vehicle's driving information meets preset conditions, including one or more of the following:

[0018] The road information currently being traveled by the vehicle indicates that the passable width of the road is less than a preset width threshold;

[0019] The vehicle's speed is less than a preset speed threshold;

[0020] The vehicle's current gear is the driving gear.

[0021] In some examples, the triggering condition includes: the predicted position of the moving object within the sensing blind zone during a preset time period;

[0022] The predicted position is determined based on the distance between the moving object and the vehicle, and the moving speed of the moving object.

[0023] In some examples, the output of prompt information based on the obstacle information includes:

[0024] The target warning level is determined based on the obstacle information;

[0025] Output corresponding prompt information based on the target prompt level; wherein, the prompt information corresponding to different prompt levels includes different prompt content and / or different output parameters.

[0026] In some examples, determining the target cue level based on the obstacle information includes one or more of the following:

[0027] Based on the distance between the obstacle and the vehicle and / or the speed of the obstacle as indicated by the obstacle information, the target warning level is determined;

[0028] Predict the movement trajectory of the obstacle based on the obstacle information; determine the target indication level based on the movement trajectory and the vehicle's driving trajectory.

[0029] In some examples, the vehicle's driving modes include manual driving mode and automatic driving mode. If the vehicle's current driving mode is manual driving mode, the method further includes:

[0030] Switch the vehicle's driving mode to automatic driving mode.

[0031] According to a second aspect of this application, a vehicle is provided, the vehicle comprising:

[0032] A first sensor and a second sensor; the sensing range of the second sensor covers the sensing blind zone of the first sensor;

[0033] processor;

[0034] Memory used to store processor-executable instructions;

[0035] Wherein, when the processor invokes the executable instructions, it implements the operation of any of the methods described in the first aspect.

[0036] According to a third aspect of this application, an obstacle warning system is provided, mounted on a vehicle, the system comprising:

[0037] A perception controller, connected to a first sensor and a second sensor, is configured to acquire sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle in response to a triggering condition; and to determine obstacle information within the sensing blind zone based on the sensing information; wherein the sensing range of the second sensor covers the sensing blind zone of the first sensor.

[0038] An output device, connected to the perception controller, is used to output prompt information based on the obstacle information.

[0039] According to a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a plurality of computer instructions, which, when executed, perform any of the methods described in the first aspect.

[0040] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0041] This application provides an obstacle warning method, system, vehicle, and storage medium. The vehicle is equipped with a first sensor and a second sensor, and the sensing range of the second sensor covers the blind spot of the first sensor. When a trigger condition is met, sensing data obtained by the second sensor from sensing the vehicle's surrounding environment is acquired. Then, based on the sensing data, information about obstacles in the blind spot can be determined. Finally, based on the obstacle information, a warning message can be output to alert the driver. Because the second sensor can detect obstacles in the blind spot of the first sensor, it can promptly remind the driver of obstacles in the blind spot, achieving omnidirectional obstacle detection outside the vehicle and improving vehicle driving safety.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this application, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] Figure 1A This is a schematic diagram of the sensing range and blind spot of vehicle-mounted sensors in related technologies.

[0045] Figure 1B This is a schematic diagram of the sensing range and blind spot of vehicle-mounted sensors in related technologies.

[0046] Figure 2 This is a flowchart illustrating an obstacle warning method according to an embodiment of this application.

[0047] Figure 3 This is a flowchart illustrating an obstacle warning method according to another embodiment of this application.

[0048] Figure 4 This is a flowchart illustrating an obstacle warning method according to another embodiment of this application.

[0049] Figure 5 This is a hardware structure diagram of an obstacle warning device according to an embodiment of this application.

[0050] Figure 6 This is a hardware structure diagram of a vehicle according to an embodiment of this application.

[0051] Figure 7A This is a structural block diagram of an obstacle warning system according to an embodiment of this application.

[0052] Figure 7B This is a structural block diagram of an obstacle warning system according to another embodiment of this application.

[0053] Figure 8 This is a structural block diagram of an obstacle warning device according to an embodiment of this application. Detailed Implementation

[0054] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0055] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0056] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0057] With the increasing prevalence of automobiles, vehicle-related safety incidents are attracting more and more attention. To improve driving safety, vehicles are typically equipped with various sensors for obstacle detection. These sensors can include, but are not limited to, image acquisition devices, LiDAR, millimeter-wave radar, and ultrasonic sensors. Image acquisition devices can include, but are not limited to, monocular cameras, binocular cameras, and Time-of-Flight (TOF) cameras. However, vehicle sensors often have blind spots. For example, ultrasonic sensors... Figure 1A-Figure 1B As shown, vehicles typically have ultrasonic sensors mounted on all four sides: front, rear, left, and right. For example, four ultrasonic sensors are mounted on the front and rear of the vehicle, and two on each of the left and right sides. Due to the limited field of view (FOV) of the ultrasonic sensors, i.e., the sensing angle, there are areas around the vehicle that the ultrasonic sensors cannot detect, known as blind spots.

[0058] like Figure 1A As shown, the ultrasonic sensors mounted at the front and rear of the vehicle have a sensing range 111. Due to the limited vertical field of view (FOV) of the ultrasonic sensors, the areas near the underside of the vehicle at the front and rear are sensing blind spots 121. Figure 1B As shown, the ultrasonic sensors mounted on both sides of the vehicle have a sensing range of 112. Due to the limited horizontal field of view (FOV) of the ultrasonic sensors, there are blind spots 122 on both sides of the vehicle. Therefore, the sensors cannot perform omnidirectional obstacle detection outside the vehicle due to these blind spots, posing a safety hazard.

[0059] To address the aforementioned technical problems, this application provides an obstacle warning method applied to a vehicle. The vehicle is equipped with a first sensor and a second sensor. The sensing range of the second sensor covers the blind spot of the first sensor. The method includes, as follows: Figure 2 The steps shown are as follows:

[0060] Step 210: In response to the triggering condition being triggered, acquire the sensing information obtained by the second sensor sensing the surrounding environment of the vehicle;

[0061] Step 220: Determine obstacle information within the sensing blind zone based on the sensing information;

[0062] Step 230: Output prompt information based on the obstacle information.

[0063] The first and second sensors on the vehicle can be any of the sensors mentioned above for obstacle detection. For example, the first sensor could be an ultrasonic sensor, and the second sensor could be an image acquisition device.

[0064] The sensing range of the second sensor covers the sensing blind zone of the first sensor. For example, the sensing range of the second sensor includes the sensing blind zone of the first sensor, but does not include the sensing range of the first sensor. Alternatively, the second sensor may include both the sensing blind zone and the sensing range of the first sensor.

[0065] In some embodiments, the field of view (FOV) of the second sensor is greater than that of the first sensor. However, correspondingly, the sensing depth of the second sensor is less than that of the first sensor. That is, the second sensor can detect obstacles within a range with a large angle but a small distance; while the first sensor can detect obstacles within a range with a small angle but a large distance. For example, the first sensor can be an ultrasonic sensor, and the second sensor can be a wide-angle camera, such as a fisheye camera. Fisheye cameras have extremely short focal lengths, but their FOV can reach 180-220°.

[0066] In some embodiments, the second sensor can be a fisheye camera in a vehicle-mounted Around View Monitor (AVM). The AVM captures images using multiple (typically four) ultra-wide-angle fisheye cameras, and stitching these images together creates a panoramic image of the vehicle's surroundings. These multiple fisheye cameras, for example four, can be mounted on the front, rear, left, and right sides of the vehicle. Since the field of view (FOV) of a fisheye camera can reach 180-220°, the union of the sensing ranges of the four fisheye cameras can surround the exterior of the vehicle, achieving 360° omnidirectional coverage.

[0067] The AVM (Autonomous Parking Monitor) in a vehicle is typically used to implement automatic parking functionality. The fisheye camera within the AVM is used for parking space detection. Unlike the vehicle's sensors used for obstacle detection, the fisheye camera in the AVM is not activated when obstacle detection is required. It is only when the automatic parking function is triggered that the AVM system is activated, and the fisheye camera is used for parking space detection. Therefore, in related technologies, the fisheye camera in the AVM does not participate in obstacle detection.

[0068] In this embodiment, the second sensor can be a fisheye camera in the AVM, allowing the fisheye camera in the AVM to also participate in obstacle detection. By reusing the fisheye camera, the functions that the AVM can achieve are enriched; on the other hand, there is no need to add or change the vehicle's hardware, so that the vehicle's functions are expanded without increasing the vehicle's manufacturing cost.

[0069] This application provides an obstacle warning method. When a triggering condition is met, sensing data obtained by a second sensor sensing the vehicle's surrounding environment is acquired. Based on this sensing data, information about obstacles in the blind spot can be determined. Finally, based on the obstacle information, a warning message can be output to alert the driver. Because the second sensor can detect obstacles in the blind spot of the first sensor, it can promptly remind the driver of obstacles in the blind spot, achieving omnidirectional obstacle detection outside the vehicle and improving driving safety.

[0070] Relevant regulations require drivers to walk around the vehicle before getting in, checking the surroundings and under the vehicle to ensure there are no people, animals, or other obstacles that could affect the safe starting of the vehicle. However, current technologies often fail to address this issue. Firstly, the onboard obstacle detection sensors do not activate after the vehicle is powered on and before it begins to move, forcing the driver to actively observe the vehicle's surroundings. Secondly, even if the sensors activate, they may not detect children or animals in blind spots, posing a significant safety hazard.

[0071] Thus, the obstacle warning method provided in this application can be applied in vehicle startup scenarios. In a vehicle startup scenario, the triggering condition may include: detecting a signal emitted by the vehicle's control components by a signal detection device before the vehicle is powered on.

[0072] In this embodiment, the vehicle may be equipped with an independently powered signal detection device. Independent power supply means that the power source supplying the signal detection device is independently configured. For example, in an electric vehicle, this power source is different from the power battery used to provide electrical energy to the power system. Because the signal detection device is independently powered, its operation is not affected by the vehicle being powered off. That is, the signal detection device can still function normally and receive signals transmitted by the control components after the vehicle is powered off.

[0073] In addition, the vehicle's control components are used to control vehicle unlocking and / or power-on. These control components may include, but are not limited to, dedicated vehicle remote controls, mobile phones with vehicle control software installed, personal computers (PCs), tablets, or wearable devices. The control components may have physical buttons and / or virtual controls, which users can trigger to unlock and / or power on the vehicle. The control components may periodically transmit signals; if a signal detection device detects a signal transmitted by the control components before the vehicle is powered on, the triggering condition is considered to have been met.

[0074] Based on the above embodiments, the triggering condition further includes: determining that the distance between the control component and the vehicle is less than a preset first distance threshold based on the signal detected by the signal detection device.

[0075] In this embodiment, the signal detection device may include a connected signal receiver and a processor. The signal receiver can send the received signal to the processor for processing. For example, the processor can determine whether the distance between the control component and the vehicle is less than a first distance threshold based on the signal strength. As another example, the signal emitted by the control component can carry the control component's location information. The processor can calculate the distance between the control component and the vehicle based on the control component's location information and the vehicle's location information, and then determine whether the distance is less than the first distance threshold. The vehicle's location information can be obtained based on a separately powered positioning device mounted on the vehicle.

[0076] When the signal detection device detects a signal emitted by the control component and / or the distance between the control component and the vehicle is less than a first distance threshold, it controls the second sensor to power on and acquires the sensing information obtained by the second sensor after powering on the vehicle's surrounding environment.

[0077] Vehicle power-on can be divided into low-voltage system power-on and high-voltage system power-on. Low-voltage system power-on includes powering on the vehicle's controller, interactive devices, and various sensors. High-voltage system power-on includes powering on the power unit. Therefore, as an example, when the triggering condition is met, the low-voltage system can be powered on, thus powering on the first and second sensors.

[0078] In some embodiments, the second sensor can be a fisheye camera in the AVM (Autonomous Vehicle Monitor). Multiple fisheye cameras can be included, and the union of their sensing ranges surrounds the exterior of the vehicle. In the vehicle startup scenario, the fisheye cameras in the AVM are invoked for obstacle detection, fully utilizing the onboard sensors for obstacle detection. Furthermore, to address the need for surround-view detection in the vehicle startup scenario, the large field of view of the fisheye camera is utilized to achieve surround-view detection of the vehicle, ensuring the safety of vehicle startup.

[0079] Furthermore, considering that fisheye cameras have a large field of view but a small sensing depth, obstacles detected by fisheye cameras tend to be closer to the vehicle. Therefore, when the vehicle speed is high, other sensors with greater sensing depth can be used for obstacle detection. Thus, in some embodiments, when the detected vehicle speed exceeds a preset speed threshold, obstacle detection by the second sensor, such as the fisheye camera, can be disabled. The speed threshold is positively correlated with the sensing depth of the second sensor; that is, the greater the sensing depth of the second sensor, the larger the speed threshold can be set. For example, the speed threshold could be 5 km / h.

[0080] In vehicle startup scenarios, on the one hand, since sensor startup takes a certain amount of time, when the signal detection device detects a signal emitted by the control component and / or the distance between the control component and the vehicle is less than a first distance threshold, it controls the second sensor to power on, enabling the second sensor to start quickly. Once the driver enters the vehicle, the second sensor can detect whether there are obstacles around the vehicle. On the other hand, the second sensor automatically detects obstacles after the vehicle starts, covering the blind spot of the first sensor. This allows for safe starting when there are no children, animals, or other obstacles around the vehicle, greatly reducing safety hazards during vehicle startup.

[0081] Vehicles often encounter narrow passage scenarios while driving, such as encountering oncoming vehicles on a narrow road, or navigating a narrow section of road. In these scenarios, drivers often need to rely on experience to navigate slowly or utilize the vehicle's narrow-passage assist function. This function uses sensors to detect the distance between obstacles and the vehicle. However, sensors also have blind spots. If an obstacle is partially within the sensor's blind spot, the distance between the obstacle and the vehicle cannot be accurately calculated, potentially leading to a safety hazard.

[0082] For example, if an electric scooter with pedals is traveling towards a vehicle, although the vehicle can detect the distance between them using its onboard sensors, it won't be able to detect the distance if the pedals are in the sensor's blind spot. Since the pedals extend outwards from the scooter's body, the vehicle's body may be scratched by them. Therefore, even with narrow-lane assist features, a collision can still occur if an obstacle is partially within the sensor's blind spot.

[0083] Thus, the obstacle warning method provided in this application can be applied to narrow passage scenarios. In narrow passage scenarios, the triggering conditions may include: the distance between the target object and the vehicle is less than a preset second distance threshold, and the vehicle's driving information meets preset conditions.

[0084] The target object may include an obstacle that is at least partially within the sensing range of the first sensor. That is, the obstacle may be partially within the sensing range of the first sensor, and partially within its blind spot. Since the sensing range of the second sensor covers the blind spot of the first sensor, both the first and second sensors can detect the obstacle. Alternatively, the obstacle may be entirely within the sensing range of the first sensor. In this case, the first sensor can detect the obstacle. The distance between the obstacle and the vehicle can be determined based on the sensing information from the first sensor.

[0085] Furthermore, the target object can also include other vehicles, such as other vehicles traveling in the opposite direction. For example, the distance between other vehicles and this vehicle can be determined using sensing information from the first sensor. As another example, if other vehicles can communicate wirelessly with this vehicle, then the distance between them can be determined based on location information obtained from the other vehicles.

[0086] For example, other vehicles can be equipped with V2X (Vehicle to X) communication devices. V2X stands for vehicle to everything, referring to communication between the vehicle and surrounding vehicles, equipment, and base stations to obtain real-time traffic information such as road conditions, pedestrian information, etc. V2X can include vehicle-to-vehicle (V2V) communication, vehicle-to-infrastructure (V2I) communication, vehicle-to-pedestrian (V2P) communication, vehicle-to-network (V2N) communication, and so on.

[0087] In this embodiment, both the vehicle and other vehicles are equipped with V2X communication devices, enabling communication between them. Other vehicles can send their location information to the vehicle. Based on the location information of both vehicles, the vehicle can calculate the distance between itself and the other vehicles.

[0088] Vehicle driving information can include one or more of the following: road information, vehicle speed, and current gear. The vehicle driving information meeting preset conditions can include one or more of the following: road information indicating that the passable road width is less than a preset width threshold; vehicle speed less than a preset speed threshold; and the vehicle's current gear being a driving gear. For example, the driving gear could be D (Drive). The passable road width can be measured based on onboard sensors or determined based on prior map information stored in the vehicle.

[0089] In this embodiment, an obstacle warning method is applied in a narrow passage scenario. The vehicle first needs to determine if it is currently in a narrow passage scenario. If the vehicle detects a target object (obstacle or other vehicle) in front of it using sensors mounted on the front and / or sides of the vehicle, such as a first sensor, and determines that the distance between the target object and the vehicle is less than a second distance threshold based on the sensor information, and simultaneously the passable width of the road the vehicle is currently traveling on is less than a preset width threshold, and / or the vehicle's speed is less than a preset speed threshold, and / or the vehicle's current gear is a driving gear (e.g., D gear), then it can be determined that the vehicle is currently in a narrow passage scenario.

[0090] Alternatively, if a vehicle obtains location information sent by other vehicles via wireless communication technology, and determines that the distance between the vehicle and other vehicles is less than a second distance threshold based on the location information of the other vehicles and its own location information, and at the same time, the passable width of the road the vehicle is currently traveling on is less than a preset width threshold, and / or the vehicle's speed is less than a preset speed threshold, and / or the vehicle's current gear is a driving gear (such as D gear), then it can be determined that the vehicle is currently in a narrow passage scenario.

[0091] When a vehicle determines that it is currently in a narrow passage scenario, it can activate a second sensor to acquire sensing information obtained from sensing the vehicle's surrounding environment. Based on this sensing information, it can identify obstacle information within the sensing blind spot to prompt the driver to avoid obstacles and / or other vehicles.

[0092] In some embodiments, when the vehicle's speed is detected to be greater than a preset speed threshold, obstacle detection by the second sensor can be disabled. The speed threshold is positively correlated with the sensing depth of the second sensor; that is, the greater the sensing depth of the second sensor, the higher the speed threshold can be set. For example, the speed threshold could be 5 km / h. In other words, the obstacle detection function of the second sensor is stopped as long as the vehicle's speed exceeds the speed threshold.

[0093] In other embodiments, obstacle detection of the second sensor is turned off when detection shutdown conditions are met. Detection shutdown conditions include one or more of the following: the distance between the target object and the vehicle is greater than a second distance threshold; road information indicates that the passable width of the road the vehicle is currently traveling on is greater than a width threshold; the vehicle's speed is greater than a speed threshold; the vehicle is not in a driving gear, such as in park or reverse.

[0094] When a vehicle meets any of the above detection shutdown conditions, it can be assumed that the vehicle is not in a narrow passage scenario, and the obstacle detection of the second sensor can be turned off.

[0095] In this embodiment, when the vehicle detects that it is currently in a narrow passage scenario, it calls on the second sensor to participate in obstacle detection. Since the second sensor covers the sensing blind spot of the first sensor, areas that the original narrow passage assist function could not detect can also be detected by the second sensor, which plays a supplementary role to the vehicle's narrow passage assist function and greatly improves the safety of the vehicle in narrow passage scenarios.

[0096] Regarding the triggering conditions, in some embodiments, the triggering conditions may include: the predicted position of the moving object within a preset time period is within the sensing blind zone of the first sensor. The moving object refers to any movable object around the vehicle, including but not limited to other vehicles, pedestrians, animals, etc. Based on the sensing information collected by the vehicle-mounted sensors, the movement trajectory of the moving object within a future time period can be predicted. If the predicted position of the moving object within the preset time period is within the sensing blind zone of the first sensor, a second sensor can be invoked to detect the moving object that has moved into the sensing blind zone. The predicted position of the moving object can be determined based on the distance between the moving object and the vehicle, and the moving speed of the moving object. The distance between the moving object and the vehicle can be obtained based on the embodiments described above. The moving speed of the moving object can be calculated based on multiple frames of sensing information continuously collected by the sensor. For example, the sensor can be an image acquisition device, and the moving speed of the moving object can be calculated using multiple frames of images continuously collected by the image acquisition device over a period of time. Related calculation methods are described in the relevant art and will not be elaborated upon here.

[0097] In some embodiments, if the second sensor does not detect an obstacle within the aforementioned preset time period, obstacle detection of the second sensor can be turned off.

[0098] In this embodiment, if it is predicted that a moving object will enter the sensing blind zone of the first sensor, the second sensor is activated in advance to participate in obstacle detection. This allows the second sensor to detect the moving object immediately after it enters the sensing blind zone, improving vehicle driving safety.

[0099] Furthermore, in some embodiments, the second sensor comprises multiple sensors, and the union of the sensing ranges of these multiple second sensors surrounds the exterior of the vehicle. That is, the union of the sensing ranges covers a 360° area outside the vehicle, achieving omnidirectional sensing of the vehicle. As an example, the second sensor can be a fisheye camera, with a field of view (FOV) of 180-220°. By mounting fisheye cameras on the front, rear, left, and right sides of the vehicle, the union of the FOVs of the four fisheye cameras covers a 360° area outside the vehicle.

[0100] In some embodiments, the second sensor can be an image acquisition device, which may include, but is not limited to, a monocular camera, a binocular camera, a TOF (Time of Flight) camera, etc. Thus, the obstacle information determination process in step 220 above may include, for example... Figure 3 The steps shown are as follows:

[0101] Step 221: Input the multiple frames of images acquired by the image acquisition device within a preset time period into the trained model for obstacle detection;

[0102] Step 222: Obtain the obstacle information based on the output of the model.

[0103] The trained model can be an end-to-end framework, also known as BEVFormer, that integrates multi-camera and temporal features based on a deformable attention mechanism. Multiple consecutive frames of images captured by an image acquisition device over a period of time are input into BEVFormer, and the model can directly output the position, size, velocity, and confidence score of the detected obstacles. Based on the model's output, one or more of the obstacle's position, size, type, velocity, and confidence score can be obtained.

[0104] In some embodiments, if the image acquisition device is a fisheye camera, the spherical image acquired by the fisheye camera suffers from severe distortion. Therefore, distortion correction processing can be performed on the spherical image before inputting it into the model. For example, distortion correction processing can be performed on multiple frames of images, such as spherical images, based on pre-calibrated distortion coefficients from the image acquisition device, such as a fisheye camera. After distortion correction, the spherical image can be corrected into a planar rectangular image.

[0105] In some embodiments, the first sensor may also perform obstacle detection simultaneously while the second sensor is detecting obstacles. For example, the obstacle information acquired by the first and second sensors can be fused together, and a prompt message can be output based on the fused obstacle information.

[0106] Based on any of the above embodiments, after acquiring the sensing information from the second sensor and determining the obstacle information within the sensing blind zone based on the sensing information, a prompt message can be output based on the obstacle information. In some embodiments, the process of outputting the prompt message in step 230 above may include, for example: Figure 4 The steps shown are as follows:

[0107] Step 231: Determine the target cue level based on the obstacle information;

[0108] Step 232: Output the corresponding prompt information based on the target prompt level.

[0109] The different prompt levels correspond to different prompt content and / or different output parameters.

[0110] There are various ways to output prompts, including but not limited to audio output, visual output, and motion-sensing output. Audio output can play the prompts through the vehicle's in-vehicle speakers; motion-sensing output can include tightening the driver's seatbelt; and visual output can display the prompts through an in-vehicle display interface. This display interface can include, but is not limited to, the interactive interface in a human-machine interface device, or the windshield used to implement a head-up display (HUD). In visual output, detected obstacles can be displayed in a virtual model, which can also display obstacle type information, distance information, speed information, etc. If the second sensor is an image acquisition device, the display interface can also be switched to the real-time image from the image acquisition device that detected the obstacle.

[0111] Warning messages can be categorized into multiple warning levels. In some cases, different warning levels may correspond to different warning messages. For example, a low warning level might correspond to the message "Obstacle on side X of vehicle, please be careful." A high warning level might correspond to the message "Pedestrian in blind spot on side X of vehicle, please brake," and so on.

[0112] In some cases, the output parameters of the prompt information differ depending on the prompt level. These output parameters can include audio output parameters, visual output parameters, and haptic output parameters. Different audio output parameters can manifest as variations in pitch, volume, speech rate, and playback frequency intervals. Different visual output parameters can manifest as differences in display brightness, color, and the area occupied by the displayed content within the display area. Different haptic output parameters can be reflected in the varying tightening force of the driver's seatbelt.

[0113] In some examples, different warning levels correspond to different warning messages, including different warning content and output parameters. Thus, after determining the target warning level based on obstacle information, the warning content and output parameters corresponding to the target warning level can be obtained. The warning message corresponding to the target warning level can then be generated based on the warning content and output parameters, and output using the determined output parameters.

[0114] For example, the alert levels can include three levels. Taking the second sensor as the image acquisition device as an example, in the first level, the vehicle's interior display can be switched to the real-time image from the obstacle-detecting image acquisition device, highlighting the detected obstacle in green, with no audible warning. In the second level, the vehicle's interior display can be switched to the real-time image from the obstacle-detecting image acquisition device, highlighting the detected obstacle in yellow, and the vehicle's speakers will broadcast the alert at a preset interval. In the third level, the vehicle's interior display can be switched to the real-time image from the obstacle-detecting image acquisition device, highlighting the detected obstacle in red, the vehicle's speakers will continuously broadcast the alert, and the driver's seatbelt will be tightened.

[0115] Regarding determining the target warning level based on obstacle information, in some embodiments, the target warning level can be determined based on the distance between the obstacle and the vehicle and / or the speed of the obstacle as indicated by the obstacle information. For example, the smaller the distance between the obstacle and the vehicle, the higher the warning level; the greater the speed of the obstacle, the higher the warning level.

[0116] In other embodiments, the movement trajectory of an obstacle can be predicted based on obstacle information, and then the target alert level can be determined based on the obstacle's movement trajectory and the vehicle's driving trajectory. Specifically, the obstacle's movement trajectory over a future period can be predicted based on the obstacle's direction and speed indicated by the obstacle information. Similarly, the vehicle's driving trajectory over a future period can be predicted based on its current driving direction and speed, or determined based on the autonomous driving route planned by the autonomous driving decision-making system. Based on the obstacle's movement trajectory and the vehicle's driving trajectory, the distance between the obstacle and the vehicle at a future moment can be determined, and the target alert level can be determined based on this distance. For example, the smaller the distance, the higher the alert level.

[0117] In some embodiments, the vehicle's driving modes may include a manual driving mode and an autonomous driving mode. If the vehicle's current driving mode is manual, then when the second sensor detects an obstacle, in addition to outputting a warning message, the vehicle's driving mode can be switched from manual driving mode to autonomous driving mode to actively intervene in vehicle control and avoid collisions and accidents.

[0118] As an example, after switching to autonomous driving mode, the autonomous driving decision module can make autonomous driving decisions such as braking, steering, and acceleration based on obstacle information detected by the second sensor. For example, if an obstacle is detected in front of the vehicle, a braking command is output to bring the vehicle to a stop. For example, if an obstacle is detected on the side of the vehicle, and there are no obstacles or other vehicles on the other side, a steering command is output to avoid a collision with the obstacle. As another example, if an obstacle is detected behind the vehicle, and there are no obstacles or other vehicles in front of the vehicle, an acceleration command is output to avoid a collision with the obstacle behind the vehicle.

[0119] Based on the obstacle warning method described in any of the above embodiments, this application also provides, as well as... Figure 5 The diagram shows the structure of an obstacle warning device. Figure 5 At the hardware level, the device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to implement the obstacle warning method described in any of the above embodiments.

[0120] Based on the obstacle warning method described in any of the above embodiments, this application also provides, as well as... Figure 6 The diagram shows the structure of a vehicle. Figure 6At the hardware level, the vehicle includes a first sensor, a second sensor, a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The second sensor's sensing range covers the blind spot of the first sensor. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the obstacle warning method described in any of the above embodiments.

[0121] Based on the obstacle warning method described in any of the above embodiments, this application also provides, as well as... Figure 7A The diagram shows a block diagram of an obstacle warning system 700. This obstacle warning system 700 is mounted on a vehicle. Figure 7A As shown, the obstacle warning system 700 includes:

[0122] A perception controller 710, connected to a first sensor 721 and a second sensor 722, is used to acquire sensing information obtained by the second sensor 722 sensing the surrounding environment of the vehicle in response to a triggering condition; and to determine obstacle information in the sensing blind zone based on the sensing information; wherein the sensing range of the second sensor 722 covers the sensing blind zone of the first sensor 721.

[0123] The output device 730, connected to the sensing controller 710, is used to output prompt information based on obstacle information.

[0124] The output device 730 may include a display screen and speakers inside the vehicle. The perception controller 710, the first sensor 721, the second sensor 722 and the output device 730 can be connected via CAN (Controller Area Network) or via in-vehicle Ethernet.

[0125] In some embodiments, when the distance between the target object and the vehicle is less than a preset second distance threshold, and the vehicle's driving information meets preset conditions, the perception controller 710 calls the second sensor 722 to sense the environment around the vehicle.

[0126] In some embodiments, when a moving object is located within the sensing blind zone of the first sensor 721 within a preset time period, the perception controller 710 invokes the second sensor 722 to sense the environment around the vehicle.

[0127] In some embodiments, such as Figure 7BAs shown, the obstacle warning system 700 also includes a vehicle controller 740. The perception controller 710, the first sensor 721, the second sensor 722, the output device 730, and the vehicle controller 740 can communicate via CAN (Controller Area Network) or via in-vehicle Ethernet. The vehicle also has an independently powered signal detection device (not shown), which can communicate with the vehicle controller. Before the vehicle is powered on, if the independently powered signal detection device detects a signal emitted by the vehicle's control components, or determines based on the signal that the distance between the control components and the vehicle is less than a first distance threshold, the vehicle controller 740 can be activated. The vehicle controller 740 can control the power-on of the vehicle's low-voltage system, including the perception controller 710, the first sensor 721, the second sensor 722, and the output device 730. Subsequently, the perception controller can acquire the sensing information after the second sensor is powered on.

[0128] In some embodiments, if the second sensor 722 is an image sensor, multiple frames of images acquired by the second sensor 722 within a preset time period can be sent to the perception controller 710. The perception controller 710 can input these multiple frames of images into a trained model for obstacle detection and obtain obstacle information based on the model's output.

[0129] In some embodiments, the perception controller 710 can determine the target prompt level based on obstacle information and send an information output command to the output device 730. In response to the information output command, the output device 730 outputs prompt information corresponding to the target prompt level.

[0130] Based on the obstacle warning method described in any of the above embodiments, this application also provides a computer program product, including a computer program, which, when executed by a processor, can be used to perform the obstacle warning method described in any of the above embodiments.

[0131] In addition, this application also provides an obstacle warning device, such as Figure 8 As shown, an obstacle warning device 800 includes:

[0132] The acquisition module 810 is used to acquire sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle in response to the triggering condition.

[0133] The determining module 820 is used to determine obstacle information within the sensing blind zone based on the sensing information;

[0134] The output module 830 is used to output prompt information based on the obstacle information.

[0135] The specific implementation methods of each of the above functional modules are described in the above embodiments, and will not be repeated here.

[0136] Based on the obstacle prompting method described in any of the above embodiments, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, can be used to perform the method described in any of the above embodiments.

[0137] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0138] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention filed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

Claims

1. An obstacle warning method, applied to a vehicle, characterized in that, The vehicle is equipped with a first sensor and a second sensor, wherein the sensing range of the second sensor covers the sensing blind zone of the first sensor; the method includes: In response to the triggering condition being triggered, the sensing information obtained by the second sensor sensing the surrounding environment of the vehicle is acquired; Based on the sensing information, obstacle information within the sensing blind zone is determined; Output prompt information based on the obstacle information; The triggering condition includes at least the following: the predicted position of the moving object within a preset time period is within the sensing blind zone; when the triggering condition is that the predicted position of the moving object within a preset time period is within the sensing blind zone, the step of acquiring the sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle includes: acquiring the sensing information obtained by the second sensor from sensing the moving object that has moved into the sensing blind zone.

2. The method according to claim 1, characterized in that, The vehicle is equipped with an independently powered signal detection device; the triggering conditions include: Before the vehicle is powered on, the signal detection device detects the signal emitted by the vehicle's control components. The control component is used to control the vehicle to unlock and / or power on.

3. The method according to claim 2, characterized in that, The triggering conditions also include: Based on the detected signal, it is determined that the distance between the control component and the vehicle is less than a preset first distance threshold.

4. The method according to any one of claims 2-3, characterized in that, The step of acquiring sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle in response to the triggering condition includes: In response to the triggering condition being triggered, the second sensor is powered on, and sensing information obtained by the second sensor sensing the surrounding environment of the vehicle after being powered on is acquired.

5. The method according to claim 1, characterized in that, The triggering conditions include: The distance between the target object and the vehicle is less than a preset second distance threshold, and the vehicle's driving information meets preset conditions; The target object includes obstacles and / or other vehicles that are at least partially within the sensing range of the first sensor; the vehicle's driving information meets preset conditions, including one or more of the following: The road information currently being traveled by the vehicle indicates that the passable width of the road is less than a preset width threshold; The vehicle's speed is less than a preset speed threshold; The vehicle is currently in drive gear.

6. The method according to claim 1, characterized in that, The predicted position is determined based on the distance between the moving object and the vehicle, and the moving speed of the moving object.

7. The method according to claim 1, characterized in that, The step of outputting prompt information based on the obstacle information includes: The target warning level is determined based on the obstacle information; Output corresponding prompt information based on the target prompt level; wherein, the prompt information corresponding to different prompt levels includes different prompt content and / or different output parameters.

8. The method according to claim 7, characterized in that, The determination of the target cue level based on the obstacle information includes one or more of the following: Based on the distance between the obstacle and the vehicle and / or the speed of the obstacle as indicated by the obstacle information, the target warning level is determined; Predict the movement trajectory of the obstacle based on the obstacle information; determine the target indication level based on the movement trajectory and the vehicle's driving trajectory.

9. The method according to claim 1, characterized in that, The vehicle's driving modes include manual driving mode and automatic driving mode. If the vehicle's current driving mode is manual driving mode, the method further includes: Switch the vehicle's driving mode to automatic driving mode.

10. A vehicle, characterized in that, The vehicles include: A first sensor and a second sensor; the sensing range of the second sensor covers the sensing blind zone of the first sensor; processor; Memory used to store processor-executable instructions; Wherein, when the processor invokes the executable instructions, it implements the operation of any one of the methods described in claims 1-9.

11. An obstacle warning system, mounted on a vehicle, characterized in that, The system includes: A first sensor and a second sensor, wherein the sensing range of the second sensor covers the sensing blind zone of the first sensor; A perception controller, connected to the first sensor and the second sensor, is configured to, in response to a triggering condition, acquire sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle; and to determine obstacle information within the sensing blind zone based on the sensing information; the triggering condition includes at least: the predicted position of a moving object within a preset time period is within the sensing blind zone; when the triggering condition is that the predicted position of the moving object within the preset time period is within the sensing blind zone, acquiring the sensing information obtained by the second sensor from sensing the surrounding environment of the vehicle includes: acquiring the sensing information obtained by the second sensor from sensing the moving object that has moved into the sensing blind zone; An output device, connected to the perception controller, is used to output prompt information based on the obstacle information.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer instructions, which, when executed, perform the method described in any one of claims 1-9.

Images