A risk alert method and device

Abstract

This application discloses a risk warning method, which includes: acquiring a driver's driving information, including a target object that poses a driving risk to the driver and the corresponding risk level of the target object; and presenting a risk warning for the target object based on the driving information, wherein the presentation method of the risk warning is related to the risk level. In this embodiment, the presentation information of the risk warning for the target object can be determined based on the risk level in the driving information, which can reduce the visual interference of the risk warning for the driver while ensuring the effectiveness of the driving risk warning.

Description

Technical Field

[0001] This application relates to the field of vehicle driving, and more particularly to a risk warning method and device. Background Technology

[0002] With the rapid development of automotive cockpit electronics, the types and content of information that can be displayed on screens (instrument panels / center consoles) are becoming increasingly rich. However, this requires drivers to frequently look down to view the information, leading to an increase in the time and frequency with which drivers' eyes are off the road, resulting in greater potential driving safety hazards. Against this backdrop, head-up displays (HUDs) have emerged.

[0003] Compared to traditional screen displays, HUDs can overlay visual content directly onto the road surface in front of the driver, offering superior readability and intuitiveness. Drivers can quickly access information and take appropriate actions while looking at the road, giving them more reaction and decision-making time, thus improving driving safety. Therefore, HUDs can gradually integrate more and more driver assistance and safety warning information display functions.

[0004] However, the existing HUD display logic is to display content as soon as it is detected. This results in a lot of useless information being displayed (such as when the driver can already see a person, there is no need to display additional information), which increases the driver's visual cognitive load and affects the driver's observation of the external environment. Summary of the Invention

[0005] This application discloses a risk warning method and related device, which can determine the risk warning presentation information of the target object based on the risk level in the driving information, and can reduce the visual interference of risk warning to the driver while ensuring the warning effect of driving risk.

[0006] Firstly, this application provides a risk warning method, the method comprising: obtaining a driver's driving information, the driving information including a target object that poses a driving risk to the driver, and the degree of risk corresponding to the target object; wherein, the target object may be an object that may have a driving risk impact on the vehicle driven by the driver.

[0007] In one possible implementation, the target object can be an obstacle to the vehicle being driven by the driver. An obstacle is a general term for terrain features, landforms, and engineering structures that can impede or delay vehicle movement. In this application, obstacles can be people, vehicles, road infrastructure, road signs, etc.

[0008] In one possible implementation, the target object could be something like lane lines that could influence the driver's driving strategy.

[0009] In one possible implementation, the level of risk is determined based on at least one of the following information:

[0010] The degree to which the driver is aware of the driving risk;

[0011] The degree of responsiveness of the driver after perceiving the driving risk;

[0012] The target object is the target person, and the degree to which the target person is aware of the driving risk;

[0013] The target object is the target person, and the target person's responsiveness after perceiving the driving risk.

[0014] The degree of risk can be quantified by specific numerical values ​​or divided into multiple risk levels.

[0015] For example, the level of risk can be divided into three levels: 1) possibly not noticeable; 2) noticeable, but difficult to judge / understand (low level of notice / understanding / judging the target); 3) normal level of notice / understanding / judging the target.

[0016] The degree to which a driver can perceive the driving risk can be understood as the degree to which a driver can notice the target object.

[0017] Based on the driving information, a risk warning is presented for the target object, wherein the presentation method of the risk warning is related to the degree of risk.

[0018] In this embodiment, the presentation information of risk warnings for the target object can be determined based on the risk level in the driving information, wherein the presentation method of the risk warning is related to the risk level. Specifically, when the driving risk level is high, the presentation intensity of the driving warning is increased (presentation intensity can represent the level of reminder to the user; the higher the intensity, the higher the probability that the user is aware of the driving risk); when the driving risk level is high, the presentation intensity of the driving warning is reduced, or even when there is no driving risk, no driving risk warning is given to the target object. This reduces the visual interference of risk warnings on the driver while ensuring the effectiveness of the driving risk warning.

[0019] In one possible implementation, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including brightness values ​​and / or the brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and / or the brightness difference between the target object and the environment.

[0020] The brightness of the target object itself affects the driver's ability to perceive it. For example, when the brightness of the target object is very low, the driver is not likely to observe it, meaning the driver's perception of the driving risk is poor. On the other hand, when the brightness of the target object is high, the driver can observe it relatively easily, meaning the driver's perception of the driving risk is good.

[0021] The difference in brightness between the target object and the environment also affects the driver's ability to perceive the target object. For example, when both the target object and the surrounding environment are dim, the driver is unlikely to observe the target object, meaning the driver's perception of the driving risk is poor. Conversely, when the target object is bright and the surrounding environment is dim, the driver can observe the target object relatively easily, meaning the driver's perception of the driving risk is good.

[0022] In one possible implementation, sensors mounted on the vehicle (such as a specific external camera) can be used to identify information such as the brightness of the target object and the area (environment) where the target object is located, and then compare and analyze this information. For example, based on the relationship between the brightness of the target and the brightness of the surrounding environment, four scenarios can be identified: 1) Dark environment, dark target; 2) Dark environment, bright target; 3) Bright environment, dark target; 4) Bright environment, bright target.

[0023] In one possible implementation, the driver's perception of the driving risk is related to color information of the target object in its environment, including the color difference between the target object and the environment, and the driver's perception of the driving risk is positively correlated with the color difference between the target object and the environment.

[0024] The color difference between the target object and its environment also affects the driver's perception of the target object (the greater the difference, the easier it is for the driver to perceive the target object). This color difference can also be described as the degree of color overlap. For example, when the color overlap between the target object and its environment is high, the driver is less likely to observe the target object, meaning the driver's perception of the driving risk is poor. Conversely, when the color overlap between the target object and its environment is low, the driver can observe the target object relatively easily, meaning the driver's perception of the driving risk is good.

[0025] In one possible implementation, sensors mounted on the vehicle (such as a specific external camera) can be used to identify information such as the color of the target object and the area (environment) where the target object is located, and then compare and analyze this information. For example, based on the relationship between the target color and the background environment, there are three cases: 1) the target color and the background color are basically the same; 2) the target color and the background color are partially the same; 3) the target color and the background color are not the same.

[0026] In one possible implementation, presenting the risk warning for the target object includes: projecting the risk warning for the target object onto a head-up display (HUD); or displaying the risk warning for the target object on a display in a driving scenario.

[0027] For example, when dividing risk into different risk levels, the target threshold can be the risk level threshold for dividing different risk levels. For example, the risk level can be divided into level 1, level 2, and level 3. The target threshold can be the threshold for dividing level 1 and level 2. When the risk level is greater than the target threshold, the risk level is level 1, and when the risk level is less than the target threshold, the risk level is level 2. Alternatively, the target threshold can be the threshold for dividing level 2 and level 3. When the risk level is greater than the target threshold, the risk level is level 2, and when the risk level is less than the target threshold, the risk level is level 3.

[0028] In one possible implementation, the presentation of the risk warning is related to the degree of risk. Specifically, the intensity of the risk warning regarding driving risks is positively correlated with the degree of risk. That is, as the driving risk increases, the intensity of the warning can change continuously.

[0029] In this embodiment of the application, when the risk level is greater than the target threshold, the risk warning is presented in a first manner; when the risk level is less than the target threshold, the risk warning is presented in a second manner; wherein, the first manner provides a higher level of warning for driving risks than the second manner provides a higher level of warning for driving risks.

[0030] In other words, the intensity of risk warnings is higher when the risk level is high, and lower when the risk level is low.

[0031] In one possible implementation, the alert intensity includes the presentation time of the risk alert; the earlier the presentation time, the higher the alert intensity. The presentation time is related to the predicted time of collision (TTC). Normally, risk alerts can be based on TTC. In this embodiment, when the risk level is high, the alert time can be brought forward further so that the driver can perceive the driving risk earlier, thereby reducing the occurrence of driving accidents.

[0032] In one possible implementation, the alert intensity includes the area of ​​the risk alert's presentation region; the larger the presentation region, the higher the alert intensity. Specifically, in scenarios where the driving alert is displayed on a HUD, the presentation region can be understood as the projection area; in scenarios where the driving alert is displayed on a dashboard or other display, the presentation region can be understood as the display area. In situations with high driving risk, because the risk alert's presentation region is larger, users are more likely to perceive the target object and the corresponding driving risk, thus making it easier for the driver to detect the risk and reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, because the risk alert's presentation region is smaller (or there is no alert at all), the visual interference of the risk alert on the driver can be reduced while ensuring the effectiveness of the risk warning.

[0033] In one possible implementation, the alert intensity includes the presentation content of the risk alert; the more prominent the presentation content, the higher the alert intensity. The presentation content can be icons, text, etc., used for alerting. In situations with high driving risk, because the risk alert content is more prominent, users are more likely to notice the target object and the corresponding driving risk. Consequently, drivers can more easily perceive the driving risk, thereby reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, because the risk alert content is relatively inconspicuous (or there is no alert at all), the visual interference of the risk alert on the driver can be reduced while ensuring the effectiveness of the risk warning.

[0034] In one possible implementation, the alert intensity includes the brightness of the risk alert; the higher the brightness, the higher the alert intensity. In situations with high driving risk, the higher brightness of the risk alert makes it easier for users to perceive the target object and the corresponding driving risk, thus making it easier for drivers to detect the driving risk and reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, the lower brightness of the risk alert reduces visual interference for the driver while ensuring the effectiveness of the risk warning.

[0035] In one possible implementation, the alert intensity includes the color of the risk alert; the more vibrant the color, the higher the alert intensity. In situations with high driving risk, because the risk alert's color is more vibrant, users are more likely to notice the target object and the corresponding driving risk. Consequently, drivers can more easily perceive the driving risk, thereby reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, because the risk alert's color is relatively less vibrant, the visual interference to the driver can be reduced while still ensuring the effectiveness of the risk warning.

[0036] In one possible implementation, the target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object;

[0037] The step of presenting a risk warning for the target object based on the driving information includes:

[0038] If the driving information indicates that the driver's perception of the driving risk is below a threshold, then the head-up display (HUD) projects the indicator corresponding to the target object based on the location information.

[0039] In one possible implementation, the driver's perception of the driving risk may be related to whether the target object is within the driver's field of vision. When the target object is not within the driver's field of vision, the driver's perception of the target object is very poor.

[0040] In one possible implementation, when the driving information indicates that the driver's perception of the driving risk is below a threshold, it can be considered that from the driver's perspective, it is difficult to observe the target object. In order to enable the driver to perceive the target object, the HUD can project an indicator that can indicate the target object onto the area in front of the driver to assist the driver in observing the target object.

[0041] Among them, obstacles can be people, vehicles, traffic lights, etc.

[0042] Among them, lane lines can be single solid lines, double solid lines, single dashed lines, guide lines, no-parking grid lines, etc.

[0043] Traffic signs can be warning signs, prohibitory signs, instruction signs, directional signs, tourist area signs, or road construction safety signs, etc.

[0044] The indicator of the target object can satisfy the following:

[0045] The indicator of the target object can indicate the type of the target object.

[0046] In order for drivers to know what the target object is, they need to be able to identify the type of the target object, such as the type of obstacle (e.g., people, vehicles, traffic lights, etc.) and the type of lane lines (e.g., single solid line, double solid line, single dashed line, guide lines, no-stopping grid lines, etc.). The type of obstacle can be related to the projected shape of the target object, or it can be related to the projected color, such as white solid line, yellow solid line, etc.

[0047] In one possible implementation, the indicator of the target object can be the target object itself (e.g., directly projected lane lines) or an indicator that can indicate the target object (e.g., text, patterns, etc.).

[0048] The indicator of the target object can indicate the location of the target object.

[0049] The location of the target object can be the relative position between the target object and the driving vehicle. From the driver's perspective, the driver can know the true location of the target object through the projection of the target object's sign.

[0050] In one possible implementation, the driver's perceptibility to the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein the driver's perceptibility to the driving risk is positively correlated with the mental state; and the driver's perceptibility to the driving risk is negatively correlated with the cognitive load.

[0051] The driver's driving state also affects the driver's ability to perceive the target object. The driving state may include mental state, and optionally, the mental state is used to indicate at least one of the following information: fatigue level, drowsiness level, concentration level, alertness level, road rage level, and low mood level.

[0052] Among them, driving status can include cognitive load, which is optional. Cognitive load is mainly the workload of driving tasks. It is mainly caused by the brain being overloaded due to complex road conditions and performing multiple tasks at the same time. When driving with good road conditions and focusing on driving, the load is low. Conversely, when driving with poor road conditions and needing to focus on driving tasks and other tasks (such as making phone calls) at the same time, the load is high.

[0053] For example, when the target is in a poor mental state, the driver is less likely to observe the target, meaning the driver's perception of the driving risk is poor. Similarly, when the target's cognitive load is high, the driver is less likely to observe the target, meaning the driver's perception of the driving risk is poor.

[0054] In one possible implementation, the driving state includes the difference between the driver's current mental state and the driver's normal driving state.

[0055] Normal driving conditions can be related to the driver's historical driving conditions, such as the average value of historical driving conditions.

[0056] In one possible implementation, the driver's driving state (such as fatigue level and attention level) can be collected by sensors on the vehicle (such as in-vehicle cameras) and compared with historical averages (or a preset reference) to determine the driver's current driving state.

[0057] In one possible implementation, the driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the hazard, mental state, and / or cognitive load; wherein the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

[0058] In this context, "response" can be understood as the degree to which a user, after becoming aware of a target object, realizes the need to avoid the driving risks caused by that target object.

[0059] The driver's driving state affects the driver's responsiveness after perceiving the driving risk. The driving state may include mental state. Optionally, the mental state is used to indicate at least one of the following information: fatigue level, drowsiness level, concentration level, alertness level, road rage level, and low mood level.

[0060] Among them, driving status can include cognitive load, which is optional. Cognitive load is mainly the workload of driving tasks. It is mainly caused by the brain being overloaded due to complex road conditions and performing multiple tasks at the same time. When driving with good road conditions and focusing on driving, the load is low. Conversely, when driving with poor road conditions and needing to focus on driving tasks and other tasks (such as making phone calls) at the same time, the load is high.

[0061] The driver's reaction time to danger affects the driver's responsiveness after perceiving the driving risk. The driver's reaction time to danger can be determined from the user's historical (e.g., recent) driving information, which may include, but is not limited to, the user's reaction speed when braking after facing danger.

[0062] For example, when the target's mental state is poor, the driver's ability to respond after perceiving the driving risk is poor. Similarly, when the target's cognitive load is high, the driver's ability to respond after perceiving the driving risk is poor. In other words, the driver's ability to perceive the driving risk is poor. For example, when the driver's reaction time to danger is long, the driver's ability to respond after perceiving the driving risk is poor.

[0063] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0064] In one possible implementation, the intrinsic property includes at least one of the following:

[0065] The target person's age, the target person's field of vision, and the target person's movement status.

[0066] In one possible implementation, the target object can be a person who is also capable of being aware of the vehicle being driven by the driver and of responding to potential risks.

[0067] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0068] In one possible implementation, the intrinsic attributes include at least one of the following: the target person's age, the target person's field of vision, and the target person's movement status.

[0069] In one possible implementation, the target's position, size, and movement can be detected using the vehicle's sensors (such as specific cameras outside the vehicle). The target's state is then categorized into: external feature discriminability (such as size and height), the target's orientation (such as whether it is facing the vehicle), and its movement state (such as being stationary or swaying).

[0070] Optionally, the age of a target person can be inferred by detecting the discriminative power of their external features (such as size and height).

[0071] Optionally, the field of vision of the target person can be inferred by detecting the target person's orientation (such as whether they are facing the vehicle).

[0072] Optionally, when the target is young, the target's ability to perceive the driving risk and their ability to respond after perceiving the driving risk are both poor. When the vehicle driven by the driver is not within the target's field of vision, the target's ability to perceive the driving risk is also poor. When the target's movement indicates that they are moving rapidly, the target's ability to perceive the driving risk and their ability to respond after perceiving the driving risk are both poor.

[0073] This application also provides a risk warning method, the method comprising:

[0074] Acquire the driver's driving information, which includes target objects that pose a driving risk to the driver, and the degree of risk corresponding to the target objects;

[0075] Based on the driving information, risk warning information is obtained. The risk warning information is used by the presentation device to present a risk warning for the target object, wherein the presentation method of the risk warning is related to the degree of risk.

[0076] In one possible implementation, presenting the risk warning for the target object includes:

[0077] The risk warning for the target object is projected onto the head-up display (HUD); or,

[0078] The risk warning for the target object is displayed on the monitor in the driving scenario.

[0079] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0080] When the risk level exceeds the target threshold, the risk alert is presented in the first manner.

[0081] When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein,

[0082] The first method provides a higher level of warning about driving risks than the second method.

[0083] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0084] The intensity of the risk warning regarding driving risks is positively correlated with the degree of risk.

[0085] In one possible implementation, the alert intensity includes at least one of the following:

[0086] The earlier the risk alert is presented, the stronger the alert is.

[0087] The larger the area where the risk warning is displayed, the stronger the warning.

[0088] The more prominent the content of the risk warning, the stronger the warning.

[0089] The brightness of the risk warning is adjusted; the higher the brightness, the stronger the warning.

[0090] The more vibrant the color used to display the risk warning, the stronger the warning.

[0091] In one possible implementation, the target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object; if the driving information indicates that the driver's perception of the driving risk is below a threshold, the head-up display (HUD) can project the corresponding indicator of the target object based on the location information.

[0092] In one possible implementation, the level of risk is determined based on at least one of the following information:

[0093] The degree to which the driver is aware of the driving risk;

[0094] The degree of responsiveness of the driver after perceiving the driving risk;

[0095] The target object is the target person, and the degree to which the target person is aware of the driving risk;

[0096] The target object is the target person, and the target person's responsiveness after perceiving the driving risk.

[0097] In one possible implementation, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including brightness values ​​and / or the brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and / or the brightness difference between the target object and the environment.

[0098] In one possible implementation, the driver's perception of the driving risk is related to color information of the target object in its environment, including the color difference between the target object and the environment, and the driver's perception of the driving risk is positively correlated with the color difference between the target object and the environment.

[0099] In one possible implementation, the driver's perceptibility to the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein the driver's perceptibility to the driving risk is positively correlated with the mental state; and the driver's perceptibility to the driving risk is negatively correlated with the cognitive load.

[0100] In one possible implementation, the driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the hazard, mental state, and / or cognitive load; wherein the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

[0101] In one possible implementation, the driving state includes the difference between the driver's current mental state and the driver's normal driving state.

[0102] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0103] In one possible implementation, the intrinsic property includes at least one of the following:

[0104] The target person's age, the target person's field of vision, and the target person's movement status.

[0105] Secondly, this application provides a risk warning device, the device comprising:

[0106] The acquisition module is used to acquire the driver's driving information, which includes target objects that pose a driving risk to the driver, and the degree of risk corresponding to the target objects. The degree of risk is determined based on at least one of the following information:

[0107] The degree to which the driver is aware of the driving risk, the degree to which the driver is able to respond after being aware of the driving risk, the degree to which the target person is aware of the driving risk, or the degree to which the target person is able to respond after being aware of the driving risk;

[0108] An information presentation module is used to present a risk warning for the target object based on the driving information, wherein the presentation method of the risk warning is related to the degree of risk.

[0109] In one possible implementation, the information presentation module is specifically used for:

[0110] The risk warning for the target object is projected onto the head-up display (HUD); or,

[0111] The risk warning for the target object is displayed on the monitor in the driving scenario.

[0112] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0113] When the risk level exceeds the target threshold, the risk alert is presented in the first manner.

[0114] When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein,

[0115] The first method provides a higher level of warning about driving risks than the second method.

[0116] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0117] The intensity of the risk warning regarding driving risks is positively correlated with the degree of risk.

[0118] In one possible implementation, the alert intensity includes at least one of the following:

[0119] The earlier the risk alert is presented, the stronger the alert is.

[0120] The larger the area where the risk warning is displayed, the stronger the warning.

[0121] The more prominent the content of the risk warning, the stronger the warning.

[0122] The brightness of the risk warning is adjusted; the higher the brightness, the stronger the warning.

[0123] The more vibrant the color used to display the risk warning, the stronger the warning.

[0124] In one possible implementation, the target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object;

[0125] The information presentation module is specifically used for:

[0126] If the driving information indicates that the driver's perception of the driving risk is below a threshold, then the head-up display (HUD) projects the indicator corresponding to the target object based on the location information.

[0127] In one possible implementation, the level of risk is determined based on at least one of the following information:

[0128] The degree to which the driver is aware of the driving risk;

[0129] The degree of responsiveness of the driver after perceiving the driving risk;

[0130] The target object is the target person, and the degree to which the target person is aware of the driving risk;

[0131] The target object is the target person, and the target person's responsiveness after perceiving the driving risk.

[0132] In one possible implementation, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including brightness values ​​and / or the brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and / or the brightness difference between the target object and the environment.

[0133] In one possible implementation, the driver's perception of the driving risk is related to color information of the target object in its environment, including the color difference between the target object and the environment, and the driver's perception of the driving risk is positively correlated with the color difference between the target object and the environment.

[0134] In one possible implementation, the driver's perceptibility to the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein the driver's perceptibility to the driving risk is positively correlated with the mental state; and the driver's perceptibility to the driving risk is negatively correlated with the cognitive load.

[0135] In one possible implementation, the driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the hazard, mental state, and / or cognitive load; wherein the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

[0136] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0137] In one possible implementation, the intrinsic property includes at least one of the following:

[0138] The target person's age, the target person's field of vision, and the target person's movement status.

[0139] This application also provides a risk warning device, the device comprising:

[0140] The acquisition module is used to acquire the driver's driving information, which includes target objects that pose a driving risk to the driver and the degree of risk corresponding to the target objects;

[0141] An information generation module is used to obtain risk warning information based on the driving information. The risk warning information is used by a presentation device to present a risk warning for the target object, wherein the presentation method of the risk warning is related to the degree of risk.

[0142] In one possible implementation, presenting the risk warning for the target object includes:

[0143] The risk warning for the target object is projected onto the head-up display (HUD); or,

[0144] The risk warning for the target object is displayed on the monitor in the driving scenario.

[0145] In one possible implementation, the target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object;

[0146] The risk warning information is used to provide the presentation device with an indicator corresponding to the target object based on the location information if the driving information indicates that the driver's perception of the driving risk is below a threshold.

[0147] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0148] When the risk level exceeds the target threshold, the risk alert is presented in the first manner.

[0149] When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein,

[0150] The first method provides a higher level of warning about driving risks than the second method.

[0151] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0152] The intensity of the risk warning regarding driving risks is positively correlated with the degree of risk.

[0153] In one possible implementation, the alert intensity includes at least one of the following:

[0154] The earlier the risk alert is presented, the stronger the alert is.

[0155] The larger the area where the risk warning is displayed, the stronger the warning.

[0156] The more prominent the content of the risk warning, the stronger the warning.

[0157] The brightness of the risk warning is adjusted; the higher the brightness, the stronger the warning.

[0158] The more vibrant the color used to display the risk warning, the stronger the warning.

[0159] In one possible implementation, the level of risk is determined based on at least one of the following information:

[0160] The degree to which the driver is aware of the driving risk;

[0161] The degree of responsiveness of the driver after perceiving the driving risk;

[0162] The target object is the target person, and the degree to which the target person is aware of the driving risk;

[0163] The target object is the target person, and the target person's responsiveness after perceiving the driving risk.

[0164] In one possible implementation, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including brightness values ​​and / or the brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and / or the brightness difference between the target object and the environment.

[0165] In one possible implementation, the driver's perception of the driving risk is related to color information of the target object in its environment, including the color difference between the target object and the environment, and the driver's perception of the driving risk is positively correlated with the color difference between the target object and the environment.

[0166] In one possible implementation, the driver's perceptibility to the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein the driver's perceptibility to the driving risk is positively correlated with the mental state; and the driver's perceptibility to the driving risk is negatively correlated with the cognitive load.

[0167] In one possible implementation, the driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the hazard, mental state, and / or cognitive load; wherein the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

[0168] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0169] In one possible implementation, the intrinsic property includes at least one of the following:

[0170] The target person's age, the target person's field of vision, and the target person's movement status.

[0171] Thirdly, embodiments of this application provide a risk warning device, which may include a memory, a processor, and a bus system, wherein the memory is used to store a program, and the processor is used to execute the program in the memory to perform any of the optional methods described in the first aspect above.

[0172] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform any of the optional methods described in the first aspect.

[0173] Fifthly, embodiments of this application provide a computer program product, including code, which, when executed, is used to implement any of the optional methods described in the first aspect above.

[0174] Sixthly, this application provides a chip system including a processor for supporting a device in implementing the functions involved in the foregoing aspects, such as transmitting or processing data involved in the foregoing methods; or, information. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for executing the device or risk warning device. This chip system may be composed of chips or may include chips and other discrete devices.

[0175] This application provides a risk warning method, the method comprising: acquiring a driver's driving information, the driving information including a target object posing a driving risk to the driver, and a risk level corresponding to the target object, the risk level being determined based on at least one of the following: the driver's perceptibility to the driving risk, the driver's responsiveness after perceiving the driving risk, the target object being a target person and the target person's perceptibility to the driving risk, or the target object being a target person and the target person's responsiveness after perceiving the driving risk; presenting a risk warning to the target object based on the driving information, wherein the presentation method of the risk warning is related to the risk level. In this application embodiment, the presentation information of the risk warning for the target object can be determined based on the risk level in the driving information, wherein the presentation method of the risk warning is related to the risk level. Specifically, when the level of driving risk is high, the intensity of the driving warning should be increased (intensity indicates the strength of the warning to the user; the higher the intensity, the higher the probability that the user is aware of the driving risk). Conversely, when the level of driving risk is still high, the intensity of the driving warning should be decreased, or even, in the absence of driving risk, no warning should be given to the target driver. This approach reduces visual interference with the driver's perception of driving risks while ensuring the effectiveness of the warning. Attached Figure Description

[0176] Figure 1 A functional block diagram of an autonomous driving device with autonomous driving function provided in the embodiments of this application;

[0177] Figure 2 This is a schematic diagram of the structure of a driving system provided in an embodiment of this application;

[0178] Figure 3a and Figure 3b An internal structure of a vehicle provided in an embodiment of this application;

[0179] Figure 4 This is a schematic diagram of a HUD scenario;

[0180] Figure 5 A flowchart illustrating a risk warning method provided in this application embodiment;

[0181] Figure 6 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0182] Figure 7 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0183] Figure 8This application provides an example of a risk warning method.

[0184] Figure 9 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0185] Figure 10 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0186] Figure 11 This application provides an example of a risk warning method.

[0187] Figure 12 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0188] Figure 13 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0189] Figure 14 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0190] Figure 15 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0191] Figure 16 This application provides an example of a risk warning method.

[0192] Figure 17 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0193] Figure 18 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0194] Figure 19 This application provides a schematic diagram of a risk warning for an embodiment of the present application.

[0195] Figure 20 This application provides an example of a risk warning method.

[0196] Figure 21 A schematic diagram of a risk warning device provided in an embodiment of this application;

[0197] Figure 22 A schematic diagram of a risk warning device provided in an embodiment of this application;

[0198] Figure 23 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0199] The specific implementations of this application are described below with reference to the accompanying drawings in the embodiments. However, the implementations of this application may also include combining these embodiments without departing from the spirit or scope of this application, such as using other embodiments and making structural changes. Therefore, the detailed description of the following embodiments should not be understood in a limiting sense. The terminology used in the embodiment section of this application is only used to explain the specific embodiments of this application and is not intended to limit this application.

[0200] The functions, modules, features, units, etc., mentioned in the specific embodiments of this application can be understood as being implemented in any way by any physical or tangible component (e.g., by software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any other combination thereof). In some embodiments, the division of various devices into different modules or units shown in the figures may reflect the use of corresponding different physical and tangible components in actual implementation. Optionally, a single module in the figures of the embodiments of this application may also be implemented by multiple actual physical components. Similarly, any two or more modules depicted in the figures may also reflect different functions performed by a single actual physical component.

[0201] Regarding the method flowcharts of embodiments of this application, certain operations are described as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0202] The following description may identify one or more features as “optional.” This type of statement should not be interpreted as an exhaustive indication of features that can be considered optional; that is, other features may be considered optional even if not explicitly identified in the text. Furthermore, any description of a single entity is not intended to exclude the use of multiple such entities; similarly, a description of multiple entities is not intended to exclude the use of a single entity. Finally, the term “exemplary” refers to one implementation among many potential implementations.

[0203] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0204] This application can be used in driving scenarios.

[0205] The following examples illustrate the application scenarios of this application.

[0206] The vehicles described in this specification (such as the vehicles driven by the driver described in the embodiments of this application) may be internal combustion engine vehicles that use an engine as a power source, hybrid vehicles that use an engine and an electric motor as power sources, electric vehicles that use an electric motor as a power source, and so on.

[0207] In this embodiment of the application, the vehicle may include a driving device 100 with driving functions.

[0208] Reference Figure 1 , Figure 1 This is a functional block diagram of a driving device 100 with autonomous driving functionality provided in an embodiment of this application. In one embodiment, the driving device 100 is configured in a fully or partially autonomous driving mode. For example, the driving device 100 can control itself while in autonomous driving mode, and can determine the current state of the autonomous driving device and its surrounding environment through human operation, determine the possible behavior of at least one other autonomous driving device in the surrounding environment, and determine the confidence level corresponding to the probability that the other autonomous driving device will perform the possible behavior, and control the driving device 100 based on the determined information. When the driving device 100 is in autonomous driving mode, the driving device 100 can be set to operate without human interaction.

[0209] The driving device 100 may include various subsystems, such as a driving system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, and a user interface 116. Optionally, the driving device 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the driving device 100 may be interconnected via wired or wireless means.

[0210] The propulsion system 102 may include components that provide powered motion to the driving device 100. In one embodiment, the propulsion system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air-compressed engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 118 converts the energy source 119 into mechanical energy.

[0211] Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 may also provide power to other systems of driving device 100.

[0212] The transmission 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 121.

[0213] Sensor system 104 may include several sensors for sensing information about the environment surrounding the driving device 100. For example, sensor system 104 may include a positioning system 122 (which may be a Global Positioning System (GPS) system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU) 124, radar 126, a laser rangefinder 128, and a camera 130. Sensor system 104 may also include sensors for the internal systems of the driving device 100 being monitored (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous driving device 100.

[0214] The positioning system 122 can be used to estimate the geographic location of the driving device 100. The IMU 124 is used to sense changes in the position and orientation of the driving device 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope.

[0215] Radar 126 can use radio signals to sense objects in the surrounding environment of the driving device 100. In some embodiments, in addition to sensing objects, radar 126 can also be used to sense the speed and / or direction of travel of objects.

[0216] Radar 126 may include an electromagnetic wave transmitting unit and a receiving unit. In terms of electromagnetic wave transmission principle, radar 126 can be implemented as either pulse radar or continuous wave radar. In the continuous wave radar mode, radar 126 can be implemented as either frequency-modulated continuous wave (FMCW) or frequency shift keying (FSK) depending on the signal waveform.

[0217] Radar 126 uses electromagnetic waves as a medium to detect objects based on time-of-flight (TOF) or phase-shift methods, and detects the position of the detected object, the distance to the detected object, and the relative velocity. To detect objects located in front of, behind, or to the side of the vehicle, radar 126 can be configured at an appropriate location on the exterior of the vehicle. LiDAR 126 uses laser light as a medium to detect objects based on TOF or phase-shift methods, and detects the position of the detected object, the distance to the detected object, and the relative velocity.

[0218] Alternatively, in order to detect objects located in front of, behind or to the side of the vehicle, the lidar 126 may be configured at an appropriate location on the exterior of the vehicle.

[0219] The laser rangefinder 128 can use lasers to sense objects in the environment in which the driving device 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0220] Camera 130 can be used to capture multiple images of the surrounding environment of the driving device 100. Camera 130 can be a still camera or a video camera.

[0221] Optionally, to acquire images of the vehicle's exterior, the camera 130 can be located at an appropriate position outside the vehicle. For example, to acquire images of the front of the vehicle, the camera 130 can be positioned inside the vehicle's interior, close to the windshield. Alternatively, the camera 130 can be positioned around the front bumper or radiator grille. For example, to acquire images of the rear of the vehicle, the camera 130 can be positioned inside the vehicle's interior, close to the rear window. Alternatively, the camera 130 can be positioned around the rear bumper, trunk, or tailgate. For example, to acquire images of the sides of the vehicle, the camera 130 can be positioned inside the vehicle's interior, close to at least one of the side windows. Alternatively, the camera 130 can be positioned around a side mirror, fender, or door.

[0222] The control system 106 controls the operation of the driving device 100 and its components. The control system 106 may include various elements, including a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

[0223] The steering system 132 is operable to adjust the forward direction of the driving device 100. For example, in one embodiment, it may be a steering wheel system.

[0224] The throttle 134 is used to control the operating speed of the engine 118 and thus the speed of the driving device 100.

[0225] Braking unit 136 is used to control the deceleration of driving device 100. Braking unit 136 can use friction to slow down wheel 121. In other embodiments, braking unit 136 can convert the kinetic energy of wheel 121 into electric current. Braking unit 136 may also take other forms to slow down the rotational speed of wheel 121 to control the speed of driving device 100.

[0226] The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the driving device 100. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate object velocities, and so on.

[0227] The route control system 142 is used to determine the driving route of the driving device 100. In some embodiments, the route control system 142 may combine data from the sensor 138, the positioning system 122, and one or more predetermined maps to determine the driving route for the driving device 100.

[0228] The obstacle avoidance system 144 is used to identify, assess and avoid or otherwise traverse potential obstacles in the environment of the driving device 100.

[0229] Of course, in one instance, the control system 106 may include additional or alternative components besides those shown and described. Alternatively, some of the components shown above may be reduced.

[0230] The driving device 100 interacts with external sensors, other autonomous driving devices, other computer systems, or the user via peripheral devices 108. Peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152.

[0231] In some embodiments, peripheral device 108 provides a means for the user of driving device 100 to interact with user interface 116. For example, on-board computer 148 may provide information to the user of driving device 100. User interface 116 may also operate on-board computer 148 to receive user input. On-board computer 148 may be operated via touchscreen. In other cases, peripheral device 108 may provide a means for driving device 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (e.g., voice commands or other audio input) from the user of driving device 100. Similarly, speaker 152 may output audio to the user of driving device 100.

[0232] The wireless communication system 146 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. The wireless communication system 146 can communicate using WiFi and a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 can communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various autonomous driving device communication systems, may also be used. For example, the wireless communication system 146 may include one or more dedicated short-range communications (DSRC) devices, which may include public and / or private data communications between autonomous driving devices and / or roadside stations.

[0233] Power source 110 can provide power to various components of driving device 100. In one embodiment, power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of driving device 100. In some embodiments, power source 110 and energy source 119 can be implemented together, as is the case in some fully electric vehicles.

[0234] Some or all of the functions of the driving device 100 are controlled by a computer system 112. The computer system 112 may include at least one processor 113, which executes instructions 115 stored in a non-transitory computer-readable medium such as memory 114. The computer system 112 may also be multiple computing devices that control individual components or subsystems of the driving device 100 in a distributed manner.

[0235] Processor 113 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a special-purpose device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1 The processor, memory, and other components of computer 110 within the same block are functionally illustrated; however, those skilled in the art will understand that the processor, computer, or memory may or may not be stored in the same physical housing. For example, memory may be a hard disk drive or other storage media located in a housing different from computer 110. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0236] In the various aspects described herein, the processor may be located remotely from the driving device and may communicate wirelessly with the driving device. In other aspects, some of the processes described herein are executed on a processor located within the driving device, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0237] In some embodiments, memory 114 may contain instructions 115 (e.g., program logic) that can be executed by processor 113 to perform various functions of driving device 100, including those described above. Memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 102, sensor system 104, control system 106, and peripheral devices 108.

[0238] In addition to instruction 115, memory 114 may also store data such as road maps, route information, the position, direction, speed, and other data of the autonomous driving device, as well as other information. This information can be used by driving device 100 and computer system 112 during operation of driving device 100 in autonomous, semi-autonomous, and / or manual modes.

[0239] User interface 116 is used to provide information to or receive information from the user of driving device 100. Optionally, user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as wireless communication system 146, vehicle computer 148, microphone 150, and speaker 152.

[0240] Computer system 112 can control the functions of driving device 100 based on input received from various subsystems (e.g., driving system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can utilize input from control system 106 to control steering unit 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 is operable to provide control over many aspects of driving device 100 and its subsystems.

[0241] Optionally, one or more of these components may be installed separately from or associated with the driving device 100. For example, the memory 114 may exist partially or completely separate from the driving device 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0242] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0243] Autonomous vehicles traveling on roads, such as the driving device 100 described above, can identify objects in their surrounding environment to determine adjustments to their current speed. These objects can be other autonomous driving devices, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the autonomous driving device, the speed adjustment to be made by the autonomous vehicle can be determined.

[0244] Optionally, the autonomous vehicle driving device 100 or the computing device associated with the driving device 100 (such as...) Figure 1The computer system 112, computer vision system 140, and memory 114 can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can also be considered together to predict the behavior of a single identified object. The driving device 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what steady state the autonomous driving device will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of the driving device 100, such as the lateral position of the driving device 100 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0245] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the driving device 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0246] The aforementioned driving device 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park autopilot device, construction equipment, tram, golf cart, train, and handcart, etc., and the embodiments of this application do not impose any special limitations.

[0247] Figure 1 The functional block diagram of the driving device 100 has been introduced. The driving system 101 in the driving device 100 is described below. Figure 2 This is a schematic diagram of the structure of a driving system provided in an embodiment of this application. Figure 1 and Figure 2 The driving device 100 is described from different perspectives, for example... Figure 2 Computer system 101 in Figure 1 Computer system 112 in the middle.

[0248] like Figure 2As shown, the computer system 101 includes a processor 103, which is coupled to a system bus 105. The processor 103 can be one or more processors, each of which can include one or more processor cores. The system bus 105 is coupled to an input / output (I / O) bus 113 via a bus bridge 111. An I / O interface 115 is coupled to the I / O bus. The I / O interface 115 communicates with various I / O devices, such as input devices 117 (e.g., keyboard, mouse, touchscreen), a media tray 121 (e.g., CD-ROM), a multimedia interface, etc., a transceiver 123 (capable of sending and / or receiving radio communication signals), a camera 155 (capable of capturing still and moving digital video images), and an external USB port 125. Optionally, the interface connected to the I / O interface 115 can be a USB interface.

[0249] The processor 103 can be any conventional processor, including a Reduced Instruction Set Computing (“RISC”) processor, a Complex Instruction Set Computing (“CISC”) processor, or a combination thereof. Optionally, the processor can be a special-purpose device such as an Application-Specific Integrated Circuit (“ASIC”). Optionally, the processor 103 can be a neural network processing unit (NPU) or a combination of a neural network processor and the aforementioned conventional processors. Optionally, the processor 103 may be equipped with a neural network processor.

[0250] Computer system 101 can communicate with server 149 via network interface 129. Network interface 129 is a hardware network interface, such as a network interface card (NIC). Network 127 can be an external network, such as the Internet, or an internal network, such as Ethernet or a Virtual Private Network (VPN). Optionally, network 127 can also be a wireless network, such as a WiFi network or a cellular network.

[0251] Server 149 can be a high-precision map server connection, allowing vehicles to obtain high-precision map information through communication with the high-precision map.

[0252] Server 149 can be a vehicle management server, which can process data uploaded by vehicles and also distribute data to vehicles via the network.

[0253] In addition, the computer system 101 can communicate wirelessly with other vehicles 160 (vehicle to vehicle, V2V) or pedestrians (vehicle to pedestrian, V2P) via the network interface 129.

[0254] The hard disk drive interface is coupled to the system bus 105. The hardware drive interface is connected to the hard disk drive. The system memory 135 is coupled to the system bus 105. The data running in the system memory 135 may include the operating system 137 and application programs 143 of the computer system 101.

[0255] An operating system consists of a shell (139) and a kernel (141). The shell (139) is an interface between the user and the kernel. The shell (139) is the outermost layer of the operating system. The shell (139) manages the interaction between the user and the operating system: it waits for user input, interprets the user input for the operating system, and processes various operating system outputs.

[0256] The kernel 141 consists of the parts of the operating system used to manage memory, files, peripherals, and system resources. Interacting directly with the hardware, the operating system kernel typically runs processes and provides inter-process communication, CPU time-slice management, interrupts, memory management, I / O management, and so on.

[0257] Application 141 includes autonomous driving related programs, such as programs that manage the interaction between the autonomous driving device and obstacles on the road, programs that control the driving route or speed of the autonomous driving device, and programs that control the interaction between the driving device 100 and other autonomous driving devices on the road.

[0258] Sensor 153 is associated with computer system 101. Sensor 153 is used to detect the environment surrounding computer system 101. For example, sensor 153 can detect animals, cars, obstacles, and pedestrian crossings, etc. Furthermore, the sensor can also detect the environment around these objects, such as the environment around the animal (e.g., other animals nearby), weather conditions, ambient light levels, etc. Optionally, if computer system 101 is located on an autonomous driving device, the sensor can be a camera, infrared sensor, chemical detector, microphone, etc. When activated, sensor 153 senses information at preset intervals and provides the sensed information to computer system 101 in real-time or near real-time.

[0259] Computer system 101 is used to determine the driving state of driving device 100 based on sensor data collected by sensor 153, and to determine the driving operations required to be performed by automatic driving transducer 100 based on the driving state and the current driving task, and to report to control system 106. Figure 1The system sends control commands corresponding to the driving operation. The driving state of the driving device 100 can include its own driving status, such as the direction of the vehicle, speed, position, acceleration, etc., as well as the state of the surrounding environment of the driving device 100, such as the position of obstacles, the position and speed of other vehicles, the position of pedestrian crossings, traffic light signals, etc. The computer system 101 can include a task abstraction network and a shared policy network implemented by the processor 103. Specifically, the processor 103 determines the current autonomous driving task; the processor 103 inputs at least one set of historical paths of the autonomous driving task into the task abstraction network for feature extraction to obtain a task feature vector representing the characteristics of the autonomous driving task; the processor 103 determines a state vector representing the current driving state of the autonomous driving device based on sensor data collected by the sensor 153; the processor 103 inputs the task feature vector and the state vector into the shared policy network for processing to obtain the driving operation that the autonomous driving device needs to perform; the processor 103 executes the driving operation through the control system; the processor 103 repeats the steps of determining and executing the driving operation until the autonomous driving task is completed.

[0260] Optionally, in the various embodiments described herein, the computer system 101 may be located remotely from the autonomous driving device and may wirelessly communicate with the autonomous driving device. The transceiver 123 may transmit autonomous driving tasks, sensor data collected by the sensor 153, and other data to the computer system 101; it may also receive control commands sent by the computer system 101. The autonomous driving device may execute the control commands received from the computer system 101 by the transceiver and perform corresponding driving operations. In other aspects, some of the processes described herein are executed on a processor located within the autonomous vehicle, while others are executed by a remote processor, including taking actions necessary to perform a single maneuver.

[0261] like Figure 2As shown, display adapter 107 can drive display 109, which is coupled to system bus 105. Display 109 can be used for visual display, voice playback of information input by the user or information provided to the user, and various menus of the in-vehicle equipment. Display 109 may include one or more of the following: liquid crystal display (LCD), thin film transistor-liquid crystal display (TFT LCD), organic light-emitting diode (OLED), flexible display, 3D display, and e-ink display. Touch panel may cover display 109. When touch panel detects a touch operation on or near it, it transmits the information to processor to determine the type of touch event. The processor then provides corresponding visual output on display 109 based on the type of touch event. Alternatively, touch panel and display 109 may be integrated to realize the input and output functions of the in-vehicle equipment.

[0262] Furthermore, the display 109 can be implemented as a head-up display (HUD). Additionally, the display 109 may be equipped with a projection module to output information via an image projected onto the windshield or vehicle window. The display 109 may include a transparent display. The transparent display can be attached to the windshield or vehicle window. The transparent display can display a specified image with a specified transparency. To achieve transparency, the transparent display may include one or more of the following: transparent thin-film electroluminescent (TFEL), transparent organic light-emitting diode (OLED), transparent liquid crystal display (LCD), transmissive transparent display, and transparent light-emitting diode (LED) display. The transparency of the transparent display is adjustable.

[0263] Additionally, the display 109 can be configured in multiple areas inside the vehicle, see reference. Figure 3a and Figure 3b , Figure 3a and Figure 3b The internal structure of a vehicle according to an embodiment of the present invention is shown. For example... Figure 3a and Figure 3bAs shown, the display 109 can be configured in areas 300 and 301 of the dashboard, area 302 of the seat 308, area 303 of the pillar trim, area 304 of the door, area 305 of the center console, area of ​​the head lining, area of ​​the sunvisor, or may be implemented in area 306 of the windshield or area 307 of the window. It should be noted that the above configuration positions of the display 109 are only illustrative and do not constitute a limitation of this application.

[0264] The processor 103 can execute the driving reminder method in this embodiment of the application by cooperating with the display 109.

[0265] Next, using the display 109 as an example, which is implemented by a head-up display (HUD), we will introduce the product form of HUD:

[0266] A HUD system can project, but is not limited to, information from the vehicle's instruments (speed, temperature, fuel level, etc.), driving alerts, and navigation information through the windshield into the driver's field of vision. The virtual image corresponding to the navigation information can be overlaid on the real-world environment outside the vehicle, providing the driver with an augmented reality visual experience. This can be used for applications such as AR navigation, adaptive cruise control, and lane departure warning. Because the virtual image corresponding to the navigation information needs to be combined with the real-world scene, the vehicle must have precise positioning and detection capabilities. Typically, a HUD system needs to work in conjunction with the vehicle's advanced driver assistance system (ADAS). For example... Figure 4 The virtual image shown is projected onto the vehicle's windshield at a speed of 20 kilometers per hour. Of course, the image could also be projected onto the driver's side window, the passenger side window, or other areas (such as the dashboard), etc. The specific location is not limited here.

[0267] With the rapid development of automotive cockpit electronics, the types and content of information that can be displayed on screens (instrument panels / center consoles) are becoming increasingly rich. However, this requires drivers to frequently look down to view the information, leading to an increase in the time and frequency with which drivers' eyes are off the road, resulting in greater potential driving safety hazards. Against this backdrop, head-up displays (HUDs) have emerged.

[0268] Compared to traditional screen displays, HUDs can overlay visual content directly onto the road surface in front of the driver, offering superior readability and intuitiveness. Drivers can quickly access information and take appropriate actions while looking at the road, giving them more reaction and decision-making time, thus improving driving safety. Therefore, HUDs can gradually integrate more and more driver assistance and safety warning information display functions.

[0269] By directly overlaying AR information onto the road environment in front of the driver, AR-HUD can mark pedestrians, vehicles, and environmental elements (such as lane lines) perceived on the road, and even attribute information such as distance and height can be presented to the user in an intuitive overlay format. Among these features, vehicle collision warning and lane departure warning are already integrated into AR-HUDs in mass-produced vehicles, while pedestrian alerts, crosswalk warnings, and other display functions are appearing in many AR-HUD concept solutions.

[0270] The aforementioned method of overlaying AR information in front of the driver can also be described as projecting AR information through a HUD (e.g., projecting it onto the environment in front of the driver). For example, a HUD can project AR information using light-emitting devices and optical modules.

[0271] However, current AR-HUD information display strategies are rather mechanical: 1) They reuse traditional screen-based information transmission methods, only displaying relevant content in emergencies, such as vehicle collision warnings. This only reduces the time drivers would otherwise need to look down at the screen, but it may still lead to dangerous situations where drivers cannot react in time; 2) Or they display content as soon as it is detected, which results in a lot of useless information being displayed (such as when the driver can already see a person, there is no need to display additional information), increasing the driver's visual cognitive load, affecting the driver's observation of the external environment, and preventing important information from being transmitted in time, thus creating safety hazards.

[0272] Reference Figure 5 , Figure 5 This is a flowchart illustrating a risk warning method provided in an embodiment of this application, such as... Figure 5 As shown in the embodiments of this application, the risk warning method includes:

[0273] 501. Obtain the driver's driving information, the driving information including the target object that poses a driving risk to the driver, and the degree of risk corresponding to the target object, the degree of risk being determined based on at least one of the following: the driver's perceptibility to the driving risk, the driver's responsiveness after perceiving the driving risk, the target object being a target person and the target person's perceptibility to the driving risk, or the target object being a target person and the target person's responsiveness after perceiving the driving risk.

[0274] The target object can be any object that poses a driving risk to the vehicle being driven by the driver.

[0275] In one possible implementation, the target object can be an obstacle to the vehicle being driven by the driver. An obstacle is a general term for terrain features, landforms, and engineering structures that can impede or delay vehicle movement. In this application, obstacles can be people, vehicles, road infrastructure, road signs, etc.

[0276] In one possible implementation, the target object could be something like lane lines that could influence the driver's driving strategy.

[0277] In one possible implementation, the level of risk is determined based on at least one of the following: the driver's perceptibility to the driving risk; the driver's responsiveness after perceiving the driving risk; the target being a person, and the person's perceptibility to the driving risk; and the target being a person, and the person's responsiveness after perceiving the driving risk. These will be explained separately below:

[0278] 1. The degree to which the driver can perceive the aforementioned driving risks.

[0279] The degree to which a driver can perceive the driving risk can be understood as the degree to which a driver can notice the target object.

[0280] In one possible implementation, the driver's perception of the driving risk may be related to whether the target object is within the driver's field of vision. For example, when the target object is not within the driver's field of vision, the driver is less likely to perceive the target object, that is, the driver's perception of the driving risk is poor.

[0281] In one possible implementation, the position of a target relative to the driver can be determined using the vehicle's sensors, combined with external target detection results and the vehicle's own motion. This includes determining the target's distance and whether it is within the driver's line of sight.

[0282] In one possible implementation, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including brightness values ​​and / or the brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and / or the brightness difference between the target object and the environment.

[0283] The brightness of the target object itself affects the driver's ability to perceive it. For example, when the brightness of the target object is very low, the driver is not likely to observe it, meaning the driver's perception of the driving risk is poor. On the other hand, when the brightness of the target object is high, the driver can observe it relatively easily, meaning the driver's perception of the driving risk is good.

[0284] The difference in brightness between the target object and the environment also affects the driver's ability to perceive the target object. For example, when both the target object and the surrounding environment are dim, the driver is unlikely to observe the target object, meaning the driver's perception of the driving risk is poor. Conversely, when the target object is bright and the surrounding environment is dim, the driver can observe the target object relatively easily, meaning the driver's perception of the driving risk is good.

[0285] In one possible implementation, sensors mounted on the vehicle (such as a specific external camera) can be used to identify information such as the brightness of the target object and the area (environment) where the target object is located, and then compare and analyze this information. For example, based on the relationship between the brightness of the target and the brightness of the surrounding environment, four scenarios can be identified: 1) Dark environment, dark target; 2) Dark environment, bright target; 3) Bright environment, dark target; 4) Bright environment, bright target.

[0286] For example, in a dark environment, when the target object is a pedestrian standing next to you or an obstacle on the road, the brightness of the target object is low, and the brightness of the environment in which the target object is located is also low.

[0287] For example, in a dark environment, when the target object is a pedestrian standing under a street lamp or a person driving on the road with their headlights on, the brightness of the target object is high, while the brightness of the environment in which the target object is located is low.

[0288] For example, in a daytime environment, when the target object is an obstacle located in the center of the driving road at a distance, the brightness of the target object is low, while the brightness of the environment in which the target object is located is high.

[0289] For example, in a daytime environment, when the target object is a pedestrian crossing the road, the brightness of the target object is high, and the brightness of the environment in which the target object is located is also high.

[0290] In one possible implementation, the driver's perception of the driving risk is related to color information of the target object in its environment, including the color difference between the target object and the environment, and the driver's perception of the driving risk is positively correlated with the color difference between the target object and the environment.

[0291] The color difference between the target object and its environment also affects the driver's perception of the target object (the greater the difference, the easier it is for the driver to perceive the target object). This color difference can also be described as the degree of color overlap. For example, when the color overlap between the target object and its environment is high, the driver is less likely to observe the target object, meaning the driver's perception of the driving risk is poor. Conversely, when the color overlap between the target object and its environment is low, the driver can observe the target object relatively easily, meaning the driver's perception of the driving risk is good.

[0292] In one possible implementation, sensors mounted on the vehicle (such as a specific external camera) can be used to identify information such as the color of the target object and the area (environment) where the target object is located, and then compare and analyze this information. For example, based on the relationship between the target color and the background environment, there are three cases: 1) the target color and the background color are basically the same; 2) the target color and the background color are partially the same; 3) the target color and the background color are not the same.

[0293] For example, in a rainy nighttime environment, when the target object is a lane line, the lane line and the road surface reflect light simultaneously, causing the target object and the background color to basically overlap.

[0294] For example, when the target object is an obstacle in the distance of the driving road that is similar in color to the road surface, the target object and the background color can partially overlap.

[0295] For example, when the target object is a driving line on a daytime driving road, the target object and the background color do not overlap.

[0296] In one possible implementation, the driver's perceptibility to the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein the driver's perceptibility to the driving risk is positively correlated with the mental state; and the driver's perceptibility to the driving risk is negatively correlated with the cognitive load.

[0297] The driver's driving state also affects the driver's ability to perceive the target object. The driving state may include mental state, and optionally, the mental state is used to indicate at least one of the following information: fatigue level, drowsiness level, concentration level, alertness level, road rage level, and low mood level.

[0298] Among them, driving status can include cognitive load, which is optional. Cognitive load is mainly the workload of driving tasks. It is mainly caused by the brain being overloaded due to complex road conditions and performing multiple tasks at the same time. When driving with good road conditions and focusing on driving, the load is low. Conversely, when driving with poor road conditions and needing to focus on driving tasks and other tasks (such as making phone calls) at the same time, the load is high.

[0299] For example, when the target is in a poor mental state, the driver is less likely to observe the target, meaning the driver's perception of the driving risk is poor. Similarly, when the target's cognitive load is high, the driver is less likely to observe the target, meaning the driver's perception of the driving risk is poor.

[0300] In one possible implementation, the driving state includes the difference between the driver's current mental state and the driver's normal driving state.

[0301] Normal driving conditions can be related to the driver's historical driving conditions, such as the average value of historical driving conditions.

[0302] In one possible implementation, the driver's driving state (such as fatigue level and attention level) can be collected by sensors on the vehicle (such as in-vehicle cameras) and compared with historical averages (or a preset reference) to determine the driver's current driving state.

[0303] 2. The degree to which the driver is aware of the driving risk:

[0304] In one possible implementation, the driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the hazard, mental state, and / or cognitive load; wherein the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, positively correlated with the mental state, and negatively correlated with the cognitive load.

[0305] In this context, "response" can be understood as the degree to which a user, after becoming aware of a target object, realizes the need to avoid the driving risks caused by that target object.

[0306] The driver's driving state affects the driver's responsiveness after perceiving the driving risk. The driving state may include mental state. Optionally, the mental state is used to indicate at least one of the following information: fatigue level, drowsiness level, concentration level, alertness level, road rage level, and low mood level.

[0307] Among them, driving status can include cognitive load, which is optional. Cognitive load is mainly the workload of driving tasks. It is mainly caused by the brain being overloaded due to complex road conditions and performing multiple tasks at the same time. When driving with good road conditions and focusing on driving, the load is low. Conversely, when driving with poor road conditions and needing to focus on driving tasks and other tasks (such as making phone calls) at the same time, the load is high.

[0308] The driver's reaction time to danger affects the driver's responsiveness after perceiving the driving risk. The driver's reaction time to danger can be determined from the user's historical (e.g., recent) driving information, which may include, but is not limited to, the user's reaction speed when braking after facing danger.

[0309] For example, when the target's mental state is poor, the driver's ability to respond after perceiving the driving risk is poor. Similarly, when the target's cognitive load is high, the driver's ability to respond after perceiving the driving risk is poor. In other words, the driver's ability to perceive the driving risk is poor. For example, when the driver's reaction time to danger is long, the driver's ability to respond after perceiving the driving risk is poor.

[0310] 3. When the target is a person, the degree to which the person is aware of the driving risk and the degree to which the person can respond after becoming aware of the driving risk:

[0311] In one possible implementation, the target object can be a person who is also capable of being aware of the vehicle being driven by the driver and of responding to potential risks.

[0312] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0313] In one possible implementation, the intrinsic attributes include at least one of the following: the target person's age, the target person's field of vision, and the target person's movement status.

[0314] In one possible implementation, the target's position, size, and movement can be detected using the vehicle's sensors (such as specific cameras outside the vehicle). The target's state is then categorized into: external feature discriminability (such as size and height), the target's orientation (such as whether it is facing the vehicle), and its movement state (such as being stationary or swaying).

[0315] Optionally, the age of a target person can be inferred by detecting the discriminative power of their external features (such as size and height).

[0316] Optionally, the field of vision of the target person can be inferred by detecting the target person's orientation (such as whether they are facing the vehicle).

[0317] Optionally, when the target is young, the target's ability to perceive the driving risk and their ability to respond after perceiving the driving risk are both poor. When the vehicle driven by the driver is not within the target's field of vision, the target's ability to perceive the driving risk is also poor. When the target's movement indicates that they are moving rapidly, the target's ability to perceive the driving risk and their ability to respond after perceiving the driving risk are both poor.

[0318] In one possible implementation, the level of risk is determined based on at least one of the following: the driver's perceptibility to the driving risk; the driver's responsiveness after perceiving the driving risk; the target being a person, and the person's perceptibility to the driving risk; and the target being a person, and the person's responsiveness after perceiving the driving risk.

[0319] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk.

[0320] In one possible implementation, the level of risk is determined based on the driver's responsiveness after perceiving the driving risk.

[0321] In one possible implementation, the level of risk is determined based on the degree to which the target person is aware of the driving risk.

[0322] In one possible implementation, the level of risk is determined based on the target person's responsiveness after perceiving the driving risk.

[0323] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk and the driver's ability to respond after perceiving the driving risk.

[0324] In one possible implementation, the level of risk is determined based on the driver's and the target's perceptions of the driving risk.

[0325] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk and the target's responsiveness after perceiving the driving risk.

[0326] In one possible implementation, the level of risk is determined based on the driver's responsiveness after perceiving the driving risk; the target is a person whose perceptibility of the driving risk is determined by that person.

[0327] In one possible implementation, the level of risk is determined based on the driver's responsiveness after perceiving the driving risk and the target's responsiveness after perceiving the driving risk.

[0328] In one possible implementation, the level of risk is determined based on the target person's perception of the driving risk and their ability to respond after perceiving the driving risk.

[0329] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk; the driver's responsiveness after perceiving the driving risk; and the target's perception of the driving risk.

[0330] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk; the driver's responsiveness after perceiving the driving risk; and the target's responsiveness after perceiving the driving risk.

[0331] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk, the target's perception of the driving risk, and the target's responsiveness after perceiving the driving risk.

[0332] In one possible implementation, the driver's responsiveness after perceiving the driving risk, the target's perceptibility of the driving risk, and the target's responsiveness after perceiving the driving risk are determined.

[0333] In one possible implementation, the level of risk is determined based on the driver's perception of the driving risk; the driver's responsiveness after perceiving the driving risk; the target object being a target person, the target person's perception of the driving risk; and the target object being a target person, the target person's responsiveness after perceiving the driving risk.

[0334] For example, the level of risk can be divided into three levels: 1) possibly not noticeable; 2) noticeable, but difficult to judge / understand (low level of notice / understanding / judging the target); 3) normal level of notice / understanding / judging the target.

[0335] For example, by combining the relationship between the target's brightness and the surrounding environment, the relationship between the target's color and the background environment, the driver's characteristic changes, and the target's state, and further considering factors such as weather, environment, and vehicle, the vehicle processor can determine from the driver's perspective whether the target is dangerous to the driver, whether the target is visible to the driver relative to the environment, and whether the driver's information processing ability is consistent with their usual level, thereby deriving a risk level (or describing it as the level at which the driver can notice / understand / judge the target). The level can be exemplarily divided into three levels:

[0336] Level 1: May not be noticed.

[0337] For example, in the relationship between the target color and the background environment, the target color and the background color basically overlap.

[0338] For example, in the relationship between the brightness of the target and the surrounding environment, the environment is dark and the target is dark.

[0339] Level 2: Can be noticed, but difficult to judge / understand (low level of noticing / understanding / judging the target).

[0340] For example, based on the target state, the pedestrian is a child, the target is facing away from the vehicle, or the target is in motion rather than stationary.

[0341] For example, based on the changes in driver characteristics, there may be situations where the current driving ability (driving performance) is weaker than usual (e.g., slow braking response), attention is weaker than usual (e.g., answering the phone, daydreaming), and fatigue is higher than usual (e.g., constant yawning, prolonged eyelid drooping).

[0342] For example, in the relationship between the target color and the background environment, the target color and the background color partially overlap.

[0343] For example, in the relationship between the brightness of the target and the surrounding environment, the target is bright when the environment is dark, and the target is dark when the environment is bright.

[0344] Level 3: The level of attention / understanding / judgment of the target is normal.

[0345] For example, based on the target's state, the pedestrian is an adult, the target is facing the vehicle, and the target is in a stationary and safe place.

[0346] For example, based on the driver's characteristic changes, the driving ability, attention, and fatigue level at this time are the same as usual.

[0347] For example, in a relationship between the target color and the background environment, the target color and the background color do not overlap.

[0348] For example, in the relationship between the brightness of the target and the surrounding environment, if the environment is bright, the target will also be bright.

[0349] 502. Based on the driving information, present a risk warning for the target object, wherein the presentation method of the risk warning is related to the degree of risk.

[0350] In one possible implementation, the presentation method of the risk warning for the target object can be determined based on the driving information, and presentation information can be generated. The presentation information can be transmitted to a display device (e.g., but not limited to HUD or driver's dashboard), and the display device can present the risk warning for the target object based on the presentation information.

[0351] In this embodiment, the presentation information of risk warnings for the target object can be determined based on the risk level in the driving information, wherein the presentation method of the risk warning is related to the risk level. Specifically, when the driving risk level is high, the presentation intensity of the driving warning is increased (presentation intensity can represent the level of reminder to the user; the higher the intensity, the higher the probability that the user is aware of the driving risk); when the driving risk level is high, the presentation intensity of the driving warning is reduced, or even when there is no driving risk, no driving risk warning is given to the target object. This reduces the visual interference of risk warnings on the driver while ensuring the effectiveness of the driving risk warning.

[0352] In one possible implementation, the way the risk warning is presented is related to the level of risk.

[0353] In one possible implementation, the presentation of the risk warning is related to the degree of risk, specifically:

[0354] When the risk level exceeds the target threshold, the risk alert is presented in the first manner.

[0355] When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein,

[0356] The first method provides a higher level of warning about driving risks than the second method.

[0357] For example, when dividing risk into different risk levels, the target threshold can be the risk level threshold for dividing different risk levels. For example, the risk level can be divided into level 1, level 2, and level 3. The target threshold can be the threshold for dividing level 1 and level 2. When the risk level is greater than the target threshold, the risk level is level 1, and when the risk level is less than the target threshold, the risk level is level 2. Alternatively, the target threshold can be the threshold for dividing level 2 and level 3. When the risk level is greater than the target threshold, the risk level is level 2, and when the risk level is less than the target threshold, the risk level is level 3.

[0358] In one possible implementation, the presentation of the risk warning is related to the degree of risk. Specifically, the intensity of the risk warning regarding driving risks is positively correlated with the degree of risk. That is, as the driving risk increases, the intensity of the warning can change continuously.

[0359] In this embodiment of the application, when the risk level is greater than the target threshold, the risk warning is presented in a first manner; when the risk level is less than the target threshold, the risk warning is presented in a second manner; wherein, the first manner provides a higher level of warning for driving risks than the second manner provides a higher level of warning for driving risks.

[0360] In other words, the intensity of risk warnings is higher when the risk level is high, and lower when the risk level is low.

[0361] In one possible implementation, the alert intensity includes the presentation time of the risk alert; the earlier the presentation time, the higher the alert intensity. The presentation time is related to the predicted time of collision (TTC). Normally, risk alerts can be based on TTC. In this embodiment, when the risk level is high, the alert time can be brought forward further so that the driver can perceive the driving risk earlier, thereby reducing the occurrence of driving accidents.

[0362] In one possible implementation, the alert intensity includes the area of ​​the risk alert's presentation region; the larger the presentation region, the higher the alert intensity. Specifically, in scenarios where the driving alert is displayed on a HUD, the presentation region can be understood as the projection area; in scenarios where the driving alert is displayed on a dashboard or other display, the presentation region can be understood as the display area. In situations with high driving risk, because the risk alert's presentation region is larger, users are more likely to perceive the target object and the corresponding driving risk, thus making it easier for the driver to detect the risk and reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, because the risk alert's presentation region is smaller (or there is no alert at all), the visual interference of the risk alert on the driver can be reduced while ensuring the effectiveness of the risk warning.

[0363] In one possible implementation, the alert intensity includes the presentation content of the risk alert; the more prominent the presentation content, the higher the alert intensity. The presentation content can be icons, text, etc., used for alerting. In situations with high driving risk, because the risk alert content is more prominent, users are more likely to notice the target object and the corresponding driving risk. Consequently, drivers can more easily perceive the driving risk, thereby reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, because the risk alert content is relatively inconspicuous (or there is no alert at all), the visual interference of the risk alert on the driver can be reduced while ensuring the effectiveness of the risk warning.

[0364] In one possible implementation, the alert intensity includes the brightness of the risk alert; the higher the brightness, the higher the alert intensity. In situations with high driving risk, the higher brightness of the risk alert makes it easier for users to perceive the target object and the corresponding driving risk, thus making it easier for drivers to detect the driving risk and reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, the lower brightness of the risk alert reduces visual interference for the driver while ensuring the effectiveness of the risk warning.

[0365] In one possible implementation, the alert intensity includes the color of the risk alert; the more vibrant the color, the higher the alert intensity. In situations with high driving risk, because the risk alert's color is more vibrant, users are more likely to notice the target object and the corresponding driving risk. Consequently, drivers can more easily perceive the driving risk, thereby reducing the occurrence of driving accidents. Conversely, in situations with low driving risk, because the risk alert's color is relatively less vibrant, the visual interference to the driver can be reduced while still ensuring the effectiveness of the risk warning.

[0366] For example, the level of risk can be divided into three levels: 1) possibly not noticeable; 2) noticeable, but difficult to judge / understand (low level of notice / understanding / judging the target); 3) normal level of notice / understanding / judging the target. Corresponding information presentation strategies can be: 1) weak reminder when there is no danger; 2) strong reminder in advance when there is danger; 3) strong reminder when there is danger; 4) no reminder when there is no danger.

[0367] In one possible implementation, the target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object; if the driving information indicates that the driver's perception of the driving risk is below a threshold, the head-up display (HUD) can project the corresponding indicator of the target object based on the location information.

[0368] In one possible implementation, when the driving information indicates that the driver's perception of the driving risk is below a threshold, it can be considered that from the driver's perspective, it is difficult to observe the target object. In order to enable the driver to perceive the target object, the HUD can project an indicator that can indicate the target object onto the area in front of the driver to assist the driver in observing the target object.

[0369] The indicator of the target object can satisfy the following:

[0370] The indicator of the target object can indicate the type of the target object.

[0371] In order for drivers to know what the target object is, they need to be able to identify the type of the target object, such as the type of obstacle (e.g., people, vehicles, traffic lights, etc.) and the type of lane lines (e.g., single solid line, double solid line, single dashed line, guide lines, no-stopping grid lines, etc.). The type of obstacle can be related to the projected shape of the target object, or it can be related to the projected color, such as white solid line, yellow solid line, etc.

[0372] In one possible implementation, the indicator of the target object can be the target object itself (e.g., directly projected lane lines) or an indicator that can indicate the target object (e.g., text, patterns, etc.).

[0373] The indicator of the target object can indicate the location of the target object.

[0374] The location of the target object can be the relative position between the target object and the driving vehicle. From the driver's perspective, the driver can know the true location of the target object through the projection of the target object's sign.

[0375] The risk warning method provided in the embodiments of this application will be described below with reference to specific illustrations.

[0376] Application Example 1: Adjusting the display of risk warnings based on the driver's driving status:

[0377] Reference Figure 6 When the driver's perception of a driving risk and their ability to respond to such risks are deemed low based on their driving status, the risk warning display strategy can be adjusted to make the driver's driving style more "conservative." Specifically, the vehicle processor determines whether a target is dangerous to the driver; if so, it issues a strong early warning. For example, adjusting the lane change assist function might change the lane that was originally a lane change lane (e.g., when there are vehicles in front of or behind the target lane) to a different lane if there are vehicles in front of or behind the target lane. Figure 6 The message will be displayed as a weak warning of a vehicle or a strong reminder that "changing lanes is not recommended" (e.g., ...). Figure 7 If the collision warning is adjusted, a weak warning about the distance and a strong collision alert will be issued some time in advance (e.g., TTC changes from 3 to 5). No alert if there is no danger. For example, if the driver is on the phone but the current road conditions are good and do not affect their driving performance. When the vehicle processor determines that the driver's ability to notice / understand / judge the target is normal, it further determines whether the target is dangerous to the driver. If there is danger, a strong alert is issued; if there is no danger, no alert is issued.

[0378] In summary, based on the driver's characteristic changes, the information display strategy can be adjusted in three ways: 1) If there is danger, issue a strong early warning; 2) If there is no danger, do not issue a warning; 3) If there is danger, issue a strong, normal warning. The complete process can be illustrated as follows: Figure 8 As shown.

[0379] Application Example 2: Adjusting the display method of risk warnings based on the brightness information of the target object in its environment:

[0380] When the vehicle processor determines that the driver may not notice the information, or may notice it but find it difficult to judge / understand, it adjusts the information display strategy to make driving more "relaxed." At this point, the vehicle processor assesses whether the target poses a danger to the driver. If there is no danger, a weak warning is given. For example, in low-light conditions at night, the warning is adjusted from no warning to a weak warning for pedestrians, alerting the driver that there are pedestrians walking on the sidewalk (e.g., ...). Figure 9 When danger is imminent, provide strong advance warning. For example, in low-light conditions at night, display strong pedestrian warning signs a certain time in advance to inform drivers to be aware of pedestrians preparing to cross the road (e.g., Figure 9 When the vehicle processor determines that the driver's level of awareness / understanding / judgment of a target is normal, it further determines whether the target poses a danger to the driver. If there is no danger, no warning is given. For example, in well-lit daytime conditions, if there are pedestrians walking on the sidewalk (e.g.... Figure 10 There is a danger; a strong warning is issued. For example, during daytime when there are pedestrians preparing to cross the road.

[0381] In summary, based on the relationship between the target and the brightness of its surrounding environment, the information display strategy can be adjusted in four ways: 1) No danger, weak warning; 2) Danger, strong warning in advance; 3) Danger, strong warning as usual; 4) No danger, no warning. A complete flowchart can be seen as follows... Figure 11 As shown.

[0382] Application Example 3: Adjusting the display method of risk warnings based on the color information of the target object in its environment:

[0383] When the vehicle processor determines that the driver may not notice the information, or may notice it but find it difficult to judge / understand, it adjusts the information display strategy to make the road condition information clearer for the driver. At this point, the vehicle processor assesses whether the target poses a danger to the driver. If there is no danger, a subtle warning is given using a color that contrasts sharply with the background. For example, at night or in rain, lane markings and road surfaces reflect light simultaneously, making lane markings invisible to the driver. If there is no danger of the vehicle deviating from its lane, no warning color is used to display the ideal lane markings (e.g., lane lines). Figure 12 (There is danger; a strong early warning is given using a color that contrasts sharply with the background color.) If the vehicle is about to deviate from its lane, it helps the driver correct the vehicle's direction in advance (e.g.,...). Figure 13 When the vehicle processor determines that the driver's level of awareness / understanding / judgment of a target is normal, it further determines whether the target poses a danger to the driver. If there is no danger, no warning is issued. For example, in rainy weather during the day, the driver can clearly see the lane markings. If there is no danger of the vehicle deviating from its lane, no warning is issued (e.g., ...). Figure 14 There is a danger; a strong warning will be issued. If a vehicle is at risk of deviating from its lane (e.g., TTC≤3), a strong warning color will be displayed to indicate the lane it is deviating from (e.g., ...). Figure 15 ).

[0384] In summary, based on the relationship between the target and the brightness of its surrounding environment, the information display strategy can be adjusted in four ways: 1) No danger, a weak warning is given using a color that contrasts sharply with the background; 2) Danger exists, a strong warning is given in advance using a color that contrasts sharply with the background; 3) Danger exists, a normal strong warning is given; 4) No danger exists, no warning is given. A complete illustration of this process can be seen as follows: Figure 16 As shown.

[0385] Application Example 4: Adjust the display method of risk warnings based on the target person's perception of the driving risk and their ability to respond after perceiving the driving risk:

[0386] When the vehicle processor determines that the driver's ability to notice, understand, and judge a target is low, it adjusts the information display strategy to make the driver's driving style more "alert." The vehicle processor determines whether the target poses a danger to the driver; if not, it provides a weak warning. For example, it might switch from a state of no warning (e.g., ...). Figure 17 The pedestrian warning display will change to alert drivers to children moving away from the roadside or adults chatting on the roadside (e.g., children walking away from the roadside). Figure 18 There is danger; provide strong advance warning. For example, warn the driver at a certain time in advance of situations such as children on the road, adults with their backs to the vehicle, or obstacles ahead (e.g.). Figure 19 Conversely, when the vehicle processor determines that the driver's ability to notice / understand / judge a target is normal, it further determines whether the target poses a danger to the driver. If there is no danger, no warning is given. For example, if pedestrians continue their current movement, the HUD will not display pedestrian warning information. If there is danger, a strong pedestrian warning is given. For example, if someone chatting on the roadside suddenly wants to cross the road, a strong pedestrian warning is displayed.

[0387] In summary, based on the target state, the information display strategy is dynamically adjusted in four ways: 1) No danger, weak warning; 2) Danger, strong warning in advance; 3) Danger, strong warning as usual; 4) No danger, no warning. A complete flowchart can be seen as follows... Figure 20 As shown.

[0388] This application provides a risk warning method, the method comprising: acquiring a driver's driving information, the driving information including a target object posing a driving risk to the driver, and a risk level corresponding to the target object, the risk level being determined based on at least one of the following: the driver's perceptibility to the driving risk, the driver's responsiveness after perceiving the driving risk, the target object being a target person and the target person's perceptibility to the driving risk, or the target object being a target person and the target person's responsiveness after perceiving the driving risk; presenting a risk warning to the target object based on the driving information, wherein the presentation method of the risk warning is related to the risk level. In this application embodiment, the presentation information of the risk warning for the target object can be determined based on the risk level in the driving information, wherein the presentation method of the risk warning is related to the risk level. Specifically, when the level of driving risk is high, the intensity of the driving warning should be increased (intensity indicates the strength of the warning to the user; the higher the intensity, the higher the probability that the user is aware of the driving risk). Conversely, when the level of driving risk is still high, the intensity of the driving warning should be decreased, or even, in the absence of driving risk, no warning should be given to the target driver. This approach reduces visual interference with the driver's perception of driving risks while ensuring the effectiveness of the warning.

[0389] Reference Figure 21 , Figure 21 This is a schematic diagram of the structure of a risk warning device provided in an embodiment of this application, such as... Figure 21 As shown, the device 2100 may include:

[0390] The acquisition module 2101 is used to acquire the driver's driving information, which includes the target object that poses a driving risk to the driver and the risk level corresponding to the target object. The risk level is determined based on at least one of the following: the driver's perceptibility to the driving risk, the driver's responsiveness after perceiving the driving risk, the target object being a target person and the target person's perceptibility to the driving risk, or the target object being a target person and the target person's responsiveness after perceiving the driving risk.

[0391] For a detailed description of the acquisition module 2101, please refer to the description of step 501 in the above embodiments, which will not be repeated here.

[0392] The information presentation module 2102 is used to present a risk warning for the target object based on the driving information, wherein the presentation method of the risk warning is related to the degree of risk.

[0393] For a detailed description of the information presentation module 2102, please refer to the description of step 502 in the above embodiments, which will not be repeated here.

[0394] In one possible implementation, the information presentation module 2102 is specifically used for:

[0395] The risk warning for the target object is projected onto the head-up display (HUD); or,

[0396] The risk warning for the target object is displayed on the monitor in the driving scenario.

[0397] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0398] When the risk level exceeds the target threshold, the risk alert is presented in the first manner.

[0399] When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein,

[0400] The first method provides a higher level of warning about driving risks than the second method.

[0401] In one possible implementation, the presentation of the risk alert is related to the level of risk, including:

[0402] The intensity of the risk warning regarding driving risks is positively correlated with the degree of risk.

[0403] In one possible implementation, the alert intensity includes at least one of the following:

[0404] The earlier the risk alert is presented, the stronger the alert is.

[0405] The larger the area where the risk warning is displayed, the stronger the warning.

[0406] The more prominent the content of the risk warning, the stronger the warning.

[0407] The brightness of the risk warning is adjusted; the higher the brightness, the stronger the warning.

[0408] The more vibrant the color used to display the risk warning, the stronger the warning.

[0409] In one possible implementation, the target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object;

[0410] The information presentation module is specifically used for:

[0411] If the driving information indicates that the driver's perception of the driving risk is below a threshold, then the head-up display (HUD) projects the indicator corresponding to the target object based on the location information.

[0412] In one possible implementation, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including brightness values ​​and / or the brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and / or the brightness difference between the target object and the environment.

[0413] In one possible implementation, the driver's perception of the driving risk is related to color information of the target object in its environment, including the color difference between the target object and the environment, and the driver's perception of the driving risk is positively correlated with the color difference between the target object and the environment.

[0414] In one possible implementation, the driver's perceptibility to the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein the driver's perceptibility to the driving risk is positively correlated with the mental state; and the driver's perceptibility to the driving risk is negatively correlated with the cognitive load.

[0415] In one possible implementation, the driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the hazard, mental state, and / or cognitive load; wherein the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

[0416] In one possible implementation, the degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

[0417] In one possible implementation, the intrinsic property includes at least one of the following:

[0418] The target person's age, the target person's field of vision, and the target person's movement status.

[0419] This application also provides a risk warning device; please refer to [link / reference]. Figure 22 , Figure 22 This is a schematic diagram of a risk warning device provided in an embodiment of this application. Specifically, the risk warning device 2200 can be implemented by one or more terminal devices (e.g., a driving device). The risk warning device 2200 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 2222 (e.g., one or more processors) and a memory 2232, and one or more storage media 2230 (e.g., one or more mass storage devices) for storing application programs 2242 or data 2244. The memory 2232 and storage media 2230 can be temporary or persistent storage. The program stored in the storage media 2230 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the risk warning device. Furthermore, the central processing unit 2222 may be configured to communicate with the storage media 2230 and execute the series of instruction operations in the storage media 2230 on the risk warning device 2200.

[0420] The risk alert device 2200 may also include one or more power supplies 2226, one or more wired or wireless network interfaces 2250, one or more input / output interfaces 2258; or, one or more operating systems 2241, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0421] Specifically, the risk alert device can perform... Figure 5 The risk warning method in the corresponding embodiment.

[0422] This application also provides a computer program product that, when run on a computer, causes the computer to perform the steps performed by the aforementioned risk warning method.

[0423] This application also provides a computer-readable storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform the steps performed by the aforementioned risk warning method.

[0424] The risk warning device provided in this application embodiment can specifically be a chip, which includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip within the execution device to execute the data processing method described in the above embodiment, or to cause the chip within the risk warning device to execute the data processing method described in the above embodiment. Optionally, the storage unit can be a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0425] For details, please refer to Figure 23 , Figure 23 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU2300). The NPU2300 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core of the NPU is the arithmetic circuit 2303, which is controlled by the controller 2304 to extract matrix data from the memory and perform multiplication operations.

[0426] The NPU 2300 achieves this through the cooperation of its various internal components. Figure 5 The methods provided in the described embodiments.

[0427] More specifically, in some implementations, the arithmetic circuitry 2303 within the NPU 2300 includes multiple processing engines (PEs). In some implementations, the arithmetic circuitry 2303 is a two-dimensional pulsating array. The arithmetic circuitry 2303 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuitry 2303 is a general-purpose matrix processor.

[0428] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 2302 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 2301 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 2308.

[0429] Unified memory 2306 is used to store input and output data. Weight data is directly transferred to weight memory 2302 via Direct Memory Access Controller (DMAC) 2305. Input data is also transferred to unified memory 2306 via DMAC.

[0430] BIU stands for Bus Interface Unit 2310, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 2309.

[0431] The Bus Interface Unit (BIU) 2310 is used by the instruction fetch memory 2309 to fetch instructions from external memory, and also by the memory access controller 2305 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0432] The DMAC is mainly used to move input data from external memory DDR to unified memory 2306, or to weight data to weight memory 2302, or to input data to input memory 2301.

[0433] The vector computation unit 2307 includes multiple arithmetic processing units that, when needed, further process the output of the computation circuit 2303, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0434] In some implementations, the vector computation unit 2307 can store the processed output vector in the unified memory 2306. For example, the vector computation unit 2307 can apply a linear function, or a nonlinear function, to the output of the computation circuit 2303, such as performing linear interpolation on the feature planes extracted by the convolutional layer, or, for example, accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 2307 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as an activation input to the computation circuit 2303, for example, for use in subsequent layers of the neural network.

[0435] The instruction fetch buffer 2309 connected to the controller 2304 is used to store the instructions used by the controller 2304;

[0436] The unified memory 2306, input memory 2301, weighted memory 2302, and instruction fetch memory 2309 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0437] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the above program.

[0438] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0439] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a risk warning device, or a network device, etc.) to execute the methods described in the various embodiments of this application.

[0440] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0441] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, risk alert device, or data center to another website, computer, risk alert device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a risk alert device or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

Claims

1. A risk warning method, characterized in that, The method includes: The system acquires the driver's driving information, which includes a target object posing a driving risk to the driver and the degree of risk corresponding to the target object. The degree of risk is determined based on the following information: the driver's perceptibility to the driving risk; wherein the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including a brightness value and a brightness difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the brightness of the target object and the brightness difference between the target object and the environment; and the driver's perceptibility to the driving risk is related to the color information of the target object in its environment, the color information including the color difference between the target object and the environment, and the driver's perceptibility to the driving risk is positively correlated with the color difference between the target object and the environment. Based on the driving information, a risk warning is presented for the target object, wherein the presentation method of the risk warning is related to the degree of risk.

2. The method according to claim 1, characterized in that, The presentation of risk alerts targeting the object includes: The risk warning for the target object is projected via a head-up display (HUD); or... The risk warning for the target object is displayed on the monitor in the driving scenario.

3. The method according to claim 1, characterized in that, The way the risk warning is presented is related to the level of risk, including: When the risk level exceeds the target threshold, the risk alert is presented in the first manner. When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein, The first method provides a higher level of warning about driving risks than the second method.

4. The method according to claim 1, characterized in that, The way the risk warning is presented is related to the level of risk, including: The intensity of the risk warning regarding driving risks is positively correlated with the degree of risk.

5. The method according to claim 4, characterized in that, The alert intensity includes at least one of the following: The earlier the risk alert is presented, the stronger the alert is. The larger the area where the risk warning is displayed, the stronger the warning. The more prominent the content of the risk warning, the stronger the warning. The higher the brightness of the risk alert or the greater the difference between the displayed brightness and the ambient brightness of the risk alert, the stronger the alert. The more vibrant the color of the risk alert or the greater the difference between the color and the ambient color, the stronger the alert.

6. The method according to claim 1, characterized in that, The target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object; The step of presenting a risk warning for the target object based on the driving information includes: If the driving information indicates that the driver's perception of the driving risk is below a threshold, then the head-up display (HUD) projects the indicator corresponding to the target object based on the location information.

7. The method according to claim 1, characterized in that, The driver's perception of the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein, the driver's perception of the driving risk is positively correlated with the mental state; and the driver's perception of the driving risk is negatively correlated with the cognitive load.

8. The method according to claim 7, characterized in that, The driving state includes the difference between the driver's current mental state and the driver's normal driving state.

9. The method according to any one of claims 1 to 8, characterized in that, The degree of risk is also determined based on at least one of the following: the driver's responsiveness after perceiving the driving risk; the target being a person and the person's perceptibility of the driving risk; or the target being a person and the person's responsiveness after perceiving the driving risk.

10. The method according to claim 9, characterized in that, The driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the danger, mental state, and / or cognitive load; wherein, the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

11. The method according to claim 9, characterized in that, The degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

12. The method according to claim 11, characterized in that, The intrinsic properties include at least one of the following: The target person's age, the target person's field of vision, and the target person's movement status.

13. A risk warning device, characterized in that, The device includes: An acquisition module is used to acquire driving information of a driver, the driving information including a target object that poses a driving risk to the driver, and the degree of risk corresponding to the target object, the degree of risk being determined based on at least one of the following: the driver's perceptibility to the driving risk; wherein, the driver's perceptibility to the driving risk is related to the brightness information of the target object in its environment, the brightness information including a brightness value and a brightness difference between the target object and the environment, the driver's perceptibility to the driving risk being positively correlated with the brightness of the target object and the brightness difference between the target object and the environment; and, the driver's perceptibility to the driving risk is related to the color information of the target object in its environment, the color information including the color difference between the target object and the environment, the driver's perceptibility to the driving risk being positively correlated with the color difference between the target object and the environment; An information presentation module is used to present a risk warning for the target object based on the driving information, wherein the presentation method of the risk warning is related to the degree of risk.

14. The apparatus according to claim 13, characterized in that, The information presentation module is specifically used for: The risk warning for the target object is projected onto the head-up display (HUD); or, The risk warning for the target object is displayed on the monitor in the driving scenario.

15. The apparatus according to claim 13, characterized in that, The way the risk warning is presented is related to the level of risk, including: When the risk level exceeds the target threshold, the risk alert is presented in the first manner. When the risk level is less than the target threshold, the risk alert is presented in the second manner; wherein, The first method provides a higher level of warning about driving risks than the second method.

16. The apparatus according to claim 13, characterized in that, The way the risk warning is presented is related to the level of risk, including: The intensity of the risk warning regarding driving risks is positively correlated with the degree of risk.

17. The apparatus according to claim 16, characterized in that, The alert intensity includes at least one of the following: The earlier the risk alert is presented, the stronger the alert is. The larger the area where the risk warning is displayed, the stronger the warning. The more prominent the content of the risk warning, the stronger the warning. The brightness of the risk warning is adjusted; the higher the brightness, the stronger the warning. The more vibrant the color used to display the risk warning, the stronger the warning.

18. The apparatus according to claim 13, characterized in that, The target object is an obstacle, lane line, or traffic sign, and the driving information also includes the location information of the target object; The information presentation module is specifically used for: If the driving information indicates that the driver's perception of the driving risk is below a threshold, then the head-up display (HUD) projects the indicator corresponding to the target object based on the location information.

19. The apparatus according to claim 13, characterized in that, The driver's perception of the driving risk is related to the driver's driving state, which includes mental state and / or cognitive load; wherein, the driver's perception of the driving risk is positively correlated with the mental state; and the driver's perception of the driving risk is negatively correlated with the cognitive load.

20. The apparatus according to claim 19, characterized in that, The driving state includes the difference between the driver's current mental state and the driver's normal driving state.

21. The apparatus according to any one of claims 13 to 20, characterized in that, The degree of risk is also determined based on at least one of the following: the driver's responsiveness after perceiving the driving risk; the target being a person and the person's perceptibility of the driving risk; or the target being a person and the person's responsiveness after perceiving the driving risk.

22. The apparatus according to claim 21, characterized in that, The driver's responsiveness after perceiving the driving risk is related to the driver's reaction time to the danger, mental state, and / or cognitive load; wherein, the driver's responsiveness after perceiving the driving risk is negatively correlated with the reaction time, the driver's responsiveness after perceiving the driving risk is positively correlated with the mental state, and the driver's responsiveness after perceiving the driving risk is negatively correlated with the cognitive load.

23. The apparatus according to claim 21, characterized in that, The degree to which the target person is aware of the driving risk and the degree to which the target person can respond after being aware of the driving risk are related to the target person's own attributes, which are determined by information collected from the target person through sensors.

24. The apparatus according to claim 23, characterized in that, The intrinsic properties include at least one of the following: The target person's age, the target person's field of vision, and the target person's movement status.

25. A risk warning device, characterized in that, The apparatus includes a memory, a processor, and a presentation device; the memory stores code, and the processor is configured to retrieve the code and, in conjunction with the presentation device, perform the method as described in any one of claims 1 to 12.

26. A computer storage medium, characterized in that, The computer storage medium stores one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method of any one of claims 1 to 12.

27. A computer program product, comprising code, characterized in that, When the code is executed, it is used to implement the method as described in any one of claims 1 to 12.

Images