Vehicle information prompting method, device, electronic device and medium

Abstract

The present disclosure provides a vehicle information prompt method, device, electronic device and medium, which relates to the field of computer technology, and specifically to the technical fields of intelligent driving, intelligent cockpit, intelligent transportation and cloud computing. The specific implementation scheme is: when the current light state of the target traffic light is a red light state, obtain the in-vehicle captured image of the current vehicle; determine the current eye state of the driver in the current vehicle according to the in-vehicle captured image; generate corresponding target prompt information according to the current eye state; output the target prompt information to the driver; wherein the target prompt information is used to prompt the driver to start driving. The present disclosure can generate corresponding target prompt information to prompt the driver to start driving according to the current eye state of the driver, thereby ensuring the effectiveness of the prompt information transmitted to the driver, so that the driver can start the vehicle in time, and improve the traffic efficiency at the traffic light intersection.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to technical fields such as intelligent driving, intelligent cockpit, intelligent transportation and cloud computing, and in particular to a vehicle information prompting method, device, electronic device and medium. Background Art

[0002] Traffic lights are a very common road scene. When the traffic light is in red state, the driver of the vehicle needs to stop and wait, and when the traffic light is in green state, the driver of the vehicle needs to drive the vehicle to start passing through the intersection.

[0003] However, due to subjective factors or other objective factors of the driver, when the traffic light is in the green state, the driver sometimes fails to start the vehicle in time, thereby affecting the traffic efficiency at the traffic light intersection. Summary of the invention

[0004] The present disclosure provides a vehicle information prompting method, device, electronic device and medium for prompting a driver to start the vehicle in time.

[0005] According to one aspect of the present disclosure, a vehicle information prompting method is provided, comprising:

[0006] When the current light state of the target traffic light is a red light state, obtaining an in-vehicle captured image of the current vehicle;

[0007] Determining a current eye state of a driver in the current vehicle according to the in-vehicle collected image;

[0008] Generating corresponding target prompt information according to the current eye state;

[0009] The target prompt information is output to the driver, wherein the target prompt information is used to prompt the driver to start driving.

[0010] According to another aspect of the present disclosure, a vehicle information prompting device is provided, comprising:

[0011] The image acquisition module is used to acquire the interior image of the current vehicle when the current light state of the target traffic light is the red light state;

[0012] An eye state determination module, used to determine the current eye state of the driver in the current vehicle according to the in-vehicle collected image;

[0013] A prompt information generating module, used for generating corresponding target prompt information according to the current eye state;

[0014] The prompt information output module is used to output the target prompt information to the driver; wherein the target prompt information is used to prompt the driver to start driving.

[0015] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor to enable the at least one processor to perform any method in the present disclosure.

[0019] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute any one of the methods in the present disclosure.

[0020] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and the computer program executes any one of the methods of the present disclosure when a processor is used.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1A It is a schematic diagram of an interface for indicating the status of a traffic light in some prior arts disclosed in an embodiment of the present disclosure;

[0024] Figure 1B is a flow chart of some vehicle information prompting methods disclosed in embodiments of the present disclosure;

[0025] Figure 2A is a flow chart of other vehicle information prompting methods disclosed in embodiments of the present disclosure;

[0026] Figure 2B is a schematic diagram of some vehicle front areas disclosed according to embodiments of the present disclosure;

[0027] Figure 2C is a schematic diagram of some gaze areas disclosed according to an embodiment of the present disclosure;

[0028] Figure 3It is a schematic diagram of the structure of some vehicle information prompting devices disclosed in the embodiments of the present disclosure;

[0029] Figure 4 It is a block diagram of an electronic device used to implement the vehicle information prompting method disclosed in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] During daily driving, due to subjective factors of the driver, such as the driver lowering his head to play with the mobile phone, looking at the passengers in the car to talk, etc., or other objective factors, such as the driver's line of sight being blocked by the vehicle in front and unable to see the traffic light, when the traffic light is in the green state, the driver sometimes cannot start the vehicle in time, thereby affecting the traffic efficiency at the traffic light intersection.

[0032] To address this problem, the prior art usually prompts the driver of the traffic light status through the vehicle. However, the prompting method of the prior art is very simple and fixed, and most of them are through the display device inside the vehicle, such as the central control screen or the instrument panel screen, to provide the driver with the traffic light status prompt. Figure 1A is a schematic diagram of an interface for indicating the status of a traffic light in some prior arts disclosed in an embodiment of the present disclosure, such as Figure 1A As shown, the remaining time of the red light is displayed on the display device of the vehicle, such as Figure 1A The number "47" in the figure, as well as the total duration of the green light, for example Figure 1A The number "44" in the

[0033] It can be seen that since the prompt method of the existing technology is single and fixed, when the driver performs non-driving related behaviors, such as lowering his head to play with the mobile phone, looking out the window, or looking at the passengers in the car to talk, the driver will not be able to perceive the prompt information in time, making the prompt information transmitted to the driver less effective, weakening the role of the prompt information, and ultimately causing the driver to be unable to start the vehicle in time, affecting the traffic efficiency at the traffic light intersection.

[0034] Figure 1BThis is a flowchart of some vehicle information prompting methods disclosed in the embodiments of the present disclosure. This embodiment can be applied to the situation of prompting the driver to start driving the vehicle at a traffic light intersection. The method of this embodiment can be executed by the vehicle information prompting device disclosed in the embodiments of the present disclosure. The device can be implemented in software and / or hardware and can be integrated in any electronic device with computing capabilities, for example, it can be integrated in a vehicle control unit inside the vehicle, and the vehicle control unit is communicated with other electronic devices in the vehicle, such as the vehicle-mounted unit, image acquisition device, audio playback device, display device and seat controller in the vehicle.

[0035] like Figure 1B As shown, the vehicle information prompting method disclosed in this embodiment may include:

[0036] S101. When the current light state of the target traffic light is a red light state, obtain an interior captured image of the current vehicle.

[0037] The target traffic light refers to the traffic light installed at the traffic light intersection where the current vehicle is located. The target traffic light can be determined based on the vehicle position of the current vehicle and the traffic light positions of each traffic light. The current vehicle refers to any vehicle waiting in line at the traffic light intersection. The light state refers to the light color currently displayed by the target traffic light, including the red light state, the green light state and the yellow light state. An image acquisition device is installed in the interior of the current vehicle, including but not limited to a high-precision camera or an infrared camera, etc., which can be used to acquire images of the interior of the current vehicle to obtain an in-vehicle captured image.

[0038] It can be understood that the in-vehicle captured image at least includes a facial image portion of the driver in the current vehicle and an in-vehicle scene portion of the current vehicle.

[0039] In one embodiment, the vehicle control unit obtains the vehicle position of the current vehicle through the positioning device carried by the current vehicle, obtains the vehicle speed of the current vehicle through the driving status device carried by the current vehicle, and also obtains the traffic light position of each traffic light through the road network data. If the distance between the vehicle position of the current vehicle and the traffic light position of any traffic light is less than the distance threshold, and the vehicle speed of the current vehicle is zero, then the traffic light is used as the target traffic light. The distance threshold can be selected as 100 meters.

[0040] Get the current light status of the target traffic light, including but not limited to the following two methods:

[0041] 1) The vehicle control unit controls the forward-looking visual camera installed on the current vehicle to collect images of the target traffic light, obtains the image acquisition results collected by the forward-looking visual camera, and then identifies the light status of the target traffic light based on the image acquisition results. Since the forward-looking visual camera is usually a wide-angle camera, the image acquisition area is wide, which can ensure the success rate of identifying the status of the traffic light.

[0042] 2) The vehicle control unit controls the on-board unit (OBU) installed in the current vehicle, and receives the communication data containing the light status sent by the roadside unit (RSU) installed at the target traffic light based on the vehicle-road cooperative technology. The vehicle control unit then determines the current light status of the target traffic light based on the analysis results of the communication data.

[0043] If the vehicle control unit determines that the current light state is a red light state, it continues to determine the waiting time required for the light state to change from the current red light state to the green light state, and when the waiting time meets the time threshold value, controls the image acquisition device installed in the vehicle to trigger the image acquisition operation inside the vehicle, thereby acquiring the interior image of the current vehicle. By acquiring the interior image of the current vehicle when the current light state of the target traffic light is a red light state, the effect of acquiring the interior image based on the light state is achieved, meaningless image acquisition operations are avoided, and the steps of method execution are simplified.

[0044] S102: Determine the current eye state of the driver in the current vehicle according to the collected image inside the vehicle.

[0045] The current eye state reflects whether the driver's eyes are paying attention to a certain area at the current moment. The current eye state includes a gaze state and a non-gaze state.

[0046] The attention state means that the driver is paying attention to a certain area at the current moment, that is, the driver is in a state of full concentration. At this time, the driver has a certain possibility to pay attention to the changes in the light state in time through the visual sense. The non-attention state means that the driver is not paying attention to a certain area at the current moment, that is, the driver is in a distracted state or fatigue state. The non-attention state includes but is not limited to the closed eye state, half-open and half-closed state, frequent blinking state, etc. At this time, the driver cannot pay attention to the changes in the light state in time through the visual sense.

[0047] In one embodiment, the vehicle control unit acquires an in-vehicle captured image, and identifies the driver's eye image area in the in-vehicle captured image. The left vertex of the eye, the right vertex of the eye, the upper vertex of the eye, and the lower vertex of the eye are determined according to the eye image area. The lateral opening and closing distance of the eye is determined according to the left vertex of the eye and the right vertex of the eye, and the longitudinal opening and closing distance of the eye is determined according to the upper vertex of the eye and the lower vertex of the eye. The ratio between the longitudinal opening and closing distance and the lateral opening and closing distance is calculated as the eye aspect ratio. If the eye aspect ratio is greater than or equal to the aspect ratio threshold, it is determined that the driver's current eye state is a gaze state; if the eye aspect ratio is less than the aspect ratio threshold and greater than zero, it is determined that the eyes are in a half-open and half-closed state, and further determines that the driver's current eye state is a non-gaze state; if the eye aspect ratio is equal to zero, it is determined that the eyes are in a closed state, and further determines that the driver's current eye state is a non-gaze state.

[0048] In another embodiment, the vehicle control unit acquires at least two consecutive in-vehicle images, and calculates the eye aspect ratio in the at least two in-vehicle images respectively. The eye aspect ratio is calculated in the same manner as above and will not be described in detail herein. The aspect ratio difference of the eye aspect ratio between any two adjacent in-vehicle images is calculated, and if the aspect ratio difference between two adjacent in-vehicle images exceeding a preset number is greater than a difference threshold, it is determined that the eyes are in a frequent blinking state, and further determined that the current eye state of the driver is a non-attention state.

[0049] In another embodiment, the vehicle control unit inputs the in-vehicle captured image into a pre-trained eye state prediction model, and determines the current eye state based on the predicted probability values ​​of the non-gaze state and the gaze state output by the eye state prediction model. For example, if the predicted probability value of the output non-gaze state is greater than the predicted probability value of the gaze state, the current eye state is determined to be the non-gaze state; correspondingly, if the predicted probability value of the output gaze state is greater than the predicted probability value of the non-gaze state, the current eye state is determined to be the gaze state. Among them, the types of eye state prediction models include machine learning models, such as convolutional neural network models, etc. The eye state prediction model is trained based on sample images labeled with eye states.

[0050] Determine the current eye state of the driver in the current vehicle based on the image collected inside the vehicle

[0051] S103: Generate corresponding target prompt information according to the current eye state.

[0052] Among them, the target prompt information is used to prompt the driver to start driving.

[0053] In one embodiment, the vehicle control unit generates target prompt information corresponding to the current eye state according to the current eye state and a preset association relationship between candidate eye states and candidate prompt information.

[0054] Optionally, if the current eye state is a non-gaze state, it means that the driver is in a distracted or fatigued state and cannot pay attention to changes in the light state in time through visual senses. Therefore, the vehicle control unit generates sound prompt information and / or tactile prompt information as target prompt information.

[0055] Optionally, if the current eye state is a gaze state, it means that the driver is in a state of full concentration, and the vehicle control unit generates a further judgment of the driver's current gaze area, and determines the target prompt information associated with the gaze area from the candidate prompt information according to the gaze area. Among them, the association relationship between each gaze area and the candidate prompt information is pre-established, that is, the vehicle control unit generates the candidate prompt information associated with any gaze area and can directly determine it as the target prompt information. For example, assuming that the gaze area is the area outside the current vehicle, the sound prompt information can be used as the target prompt information.

[0056] By generating corresponding target prompt information according to the current eye state, the corresponding target prompt information is determined according to the different current eye states of the driver, thereby ensuring the validity of the target prompt information.

[0057] S104: Output target prompt information to the driver.

[0058] In one embodiment, the vehicle control unit sends the target prompt information to an electronic device associated with the prompt information type in the current vehicle according to the prompt information type of the target prompt information, so as to control the electronic device to output the target prompt information to the driver. Optionally, in the case where the prompt information type of the target prompt information is a sound prompt information, the vehicle control unit sends the sound prompt information to an audio playback device built into the current vehicle, such as a car audio system, a car speaker, etc., so as to control the audio playback device to play the sound prompt information to the driver. Among them, the sound prompt information includes a broadcast voice or a broadcast prompt tone, for example, a countdown time for broadcasting a light state change, such as 5 seconds, 4 seconds, 3 seconds... etc.; for another example, a "beep" style prompt tone is broadcast, and the broadcast frequency of the prompt tone can be set according to demand, such as when the countdown time for the light state change is shorter, the broadcast frequency of the prompt tone is increased to improve the driver's perceived effectiveness of the prompt tone.

[0059] Optionally, when the prompt information type of the target prompt information is visual prompt information, the vehicle control unit sends the visual prompt information to a display device built into the current vehicle, such as a central control screen or an instrument panel screen, etc., thereby controlling the display device to display visual prompt information including image type and / or text type to the driver. For example, on the central control screen or the instrument panel screen, the countdown time of the light status change is displayed to the driver by popping up a card.

[0060] Optionally, when the prompt information type of the target prompt information is tactile prompt information, the vehicle control unit sends the tactile prompt information to the controller of the driver's seat, so that the seat controller controls the driver's seat and transmits tactile prompts to the driver, such as controlling the driver's seat to vibrate, and / or controlling the seat belt of the driver's seat to tighten, etc.

[0061] The present invention obtains an in-vehicle captured image of a current vehicle when the current light state of a target traffic light is a red light state, determines the current eye state of a driver in the current vehicle according to the in-vehicle captured image, generates corresponding target prompt information according to the current eye state, and then outputs the target prompt information to the driver; wherein the target prompt information is used to prompt the driver to start driving, thereby achieving an effect of outputting corresponding target prompt information to prompt the driver to start driving according to the current eye state of the driver, ensuring the effectiveness of the target prompt information being transmitted to the driver, so that the driver can start the vehicle in time, thereby improving the traffic efficiency at the traffic light intersection, and avoiding the problem that a single and fixed prompt method is used as in the prior art, and the driver cannot perceive the prompt information in time.

[0062] Figure 2A It is a flowchart of other vehicle information prompting methods disclosed in the embodiments of the present disclosure, which is further optimized and expanded based on the above technical solution and can be combined with the above optional implementation methods.

[0063] like Figure 2A As shown, the vehicle information prompting method disclosed in this embodiment may include:

[0064] S201. When the current light state of the target traffic light is red, determine the target time when the light state of the target traffic light changes to green, and determine the waiting time required for the light state to change to green according to the current time and the target time.

[0065] The target time indicates the time when the target traffic light changes from a red light state to a green light state during this round of light state changes. For example, if the light state changes from a red light state to a green light state at 9:00 during this round of light state changes, "9:00" will be used as the target time.

[0066] In one embodiment, when the current light state of the target traffic light is red, the vehicle control unit controls the on-board unit installed in the current vehicle to send a data acquisition request for the target time to the target traffic light based on the vehicle-road cooperative technology. In response to the data acquisition request, the target traffic light sends a data packet containing the target time to the on-board unit through the roadside unit installed in the target traffic light based on the vehicle-road cooperative technology. The on-board unit parses the data packet to obtain the target time, and transmits the target time to the vehicle control unit.

[0067] The vehicle control unit obtains the current time from the timing device of the current vehicle, and calculates the time difference between the current time and the target time as the waiting time required for the light state to change to the green light state.

[0068] For example, assuming that the current time is 8:59:45 and the target time is 9:00, the waiting time is 15 seconds.

[0069] S202, compare the waiting time with the time threshold value, and when the waiting time is less than or equal to the time threshold value, obtain the interior captured image of the current vehicle. In one embodiment, the vehicle control unit obtains the time threshold value, and compares the waiting time with the time threshold value. If it is determined that the waiting time is greater than the time threshold value, the operation of obtaining the interior captured image of the vehicle is not triggered. If it is determined that the waiting time is less than or equal to the time threshold value, the image acquisition device installed in the vehicle is controlled to trigger the image acquisition operation inside the vehicle, thereby obtaining the interior captured image of the current vehicle.

[0070] Optionally, the time threshold value may be set as a fixed value based on experience, such as setting the time threshold value to a fixed value of 5 seconds; the time threshold value may also be dynamically determined based on the current waiting order of vehicles at the traffic light intersection.

[0071] Optionally, the vehicle control unit controls the on-board unit installed in the current vehicle, and sends an intersection image acquisition request to the target traffic light based on the vehicle-road cooperative technology. In response to the intersection image acquisition request, the target traffic light collects the intersection image of the traffic light intersection through the installed image acquisition device, and sends a data packet containing the intersection image to the on-board unit through the installed roadside unit based on the vehicle-road cooperative technology. The on-board unit parses the data packet to obtain the intersection image, and transmits the intersection image to the vehicle control unit.

[0072] The vehicle control unit inputs the intersection image into the target detection model, and thereby determines the current vehicle and other vehicles in the intersection image based on the output results of the target detection model, and further determines the current waiting order of the current vehicle at the traffic light intersection, and finally, based on the correspondence between the candidate waiting order and the candidate time threshold value, uses the candidate time threshold value corresponding to the current waiting order as the time threshold value.

[0073] For example, assuming that the current waiting order is determined to be the 5th, and the candidate time threshold value corresponding to the candidate waiting order "5th" is "10 seconds", then "10 seconds" is used as the time threshold value.

[0074] By determining the target moment for the light state of the target traffic light to change to the green light state when the light state is in the red light state, and determining the waiting time required for the light state to change to the green light state according to the current moment and the target moment, the waiting time is compared with the time threshold value, and when the waiting time is less than or equal to the time threshold value, the in-vehicle captured image of the current vehicle is obtained, thereby triggering the prompt process before the light state has not changed to the green light state, ensuring that the driver has enough driving start reaction time, avoiding the problem of the prompt process being triggered only when the light state is in the green light state, which may cause the driver to react untimely to driving start; and, since the in-vehicle captured image is obtained according to the comparison result of the waiting time and the time threshold value, the prompt process is avoided to be triggered too early, which may cause excessive interference to the driver.

[0075] S203: Determine the current eye state of the driver in the current vehicle according to the collected image inside the vehicle.

[0076] S204. When the current eye state is a non-gaze state, execute S2051; when the current eye state is a gaze state, execute S2052.

[0077] S2051: Generate tactile prompt information for the driver as target prompt information.

[0078] In one embodiment, if the driver's current eye state is a non-attention state, then a vibration prompt information is generated for the driver's seat, a seat belt tightening prompt information, and a vibration prompt information for the current vehicle steering wheel. Optionally, a sound prompt information can also be generated as the target prompt information, that is, if the driver's current eye state is a non-attention state, then a sound prompt information and a tactile prompt information are generated together as the target prompt information.

[0079] Since the driver cannot perceive the visual prompt information inside the vehicle in time when the current eye state is non-gaze, the tactile prompt information for the driver is used as the target prompt information at this time, which can effectively convey the prompt information to the driver and improve the effectiveness of the driver's perception of the prompt information.

[0080] S2052: Determine the driver's current gaze area, and generate corresponding target prompt information according to the gaze area.

[0081] In one embodiment, the vehicle control unit uses the eye tracking technology to track the collected images in the vehicle, thereby determining the current eye direction of the driver, and determining the current gaze area of ​​the driver according to the current eye direction. The current gaze area of ​​the driver is matched with the association relationship between the gaze area and the candidate prompt information, and the candidate prompt information associated with the current gaze area of ​​the driver is determined according to the matching result as the target prompt information.

[0082] By determining the driver's current gaze area when the current eye state is a gaze state, and generating corresponding target prompt information based on the gaze area, the effect of generating corresponding target prompt information according to different gaze areas is achieved, thereby ensuring the effectiveness of the prompt information transmitted to the driver.

[0083] Optionally, S2052 includes:

[0084] When the gaze area is the area outside the front of the current vehicle, a vehicle captured image of the vehicle in front of the current vehicle is obtained; the vehicle height of the vehicle in front is determined according to the vehicle captured image, and target prompt information is generated according to the vehicle height.

[0085] The area in front of the vehicle refers to the area outside the vehicle that the driver can see through the front windshield of the current vehicle, indicating that the driver is looking forward at this time. Figure 2B is a schematic diagram of some vehicle front areas disclosed according to the embodiments of the present disclosure, such as Figure 2B As shown, 20 represents the current vehicle, 21 represents the direction of the area outside the vehicle that can be viewed through the front windshield at the left boundary of the driver's line of sight, 22 represents the direction of the area outside the vehicle that can be viewed through the front windshield at the right boundary of the driver's line of sight, and the area outside the vehicle formed by the left boundary 21 and the right boundary 22 represents the area in front of the outside of the vehicle.

[0086] The front vehicle refers to a vehicle that belongs to the same lane as the current vehicle and is directly in front of the current vehicle.

[0087] In one embodiment, when it is determined that the driver's current gaze area is the area outside the current vehicle in front of the vehicle, the vehicle control unit controls the forward-looking visual camera installed in the current vehicle to capture images of the vehicle in front and obtains the vehicle captured image of the vehicle in front. The vehicle control unit performs target detection on the vehicle captured image, determines the image size of the vehicle in front in the vehicle captured image, determines the real size of the vehicle in front in the real world based on the image size, and determines the vehicle height of the vehicle in front based on the real size. The vehicle control unit further determines whether the vehicle in front will constitute a visual blind spot for the driver of the current vehicle based on the vehicle height of the vehicle in front, and if so, generates sound prompt information acting on the audio playback device as target prompt information, otherwise, no prompt information is generated.

[0088] By determining the vehicle height of the vehicle in front based on the vehicle captured image when the gaze area is the area outside the front of the current vehicle, and generating target prompt information based on the vehicle height, the effect of judging whether to give a prompt is achieved based on whether the vehicle in front constitutes a visual blind spot for the driver, thereby avoiding meaningless prompts to the driver when the vehicle in front does not constitute a visual blind spot, causing the driver to be distracted while driving, and ensuring the driver's driving safety.

[0089] Optionally, target prompt information is generated based on the vehicle height, including:

[0090] The blind spot height threshold is determined according to the vehicle type of the current vehicle, and the blind spot height threshold is compared with the vehicle height; when the vehicle height is greater than or equal to the blind spot height threshold, a sound prompt information is generated as the target prompt information.

[0091] Among them, vehicles of different types have different heights, so the driver's line of sight height is also different. For example, under normal circumstances, the driver's line of sight height of an SUV (Sport Utility Vehicle) is higher than that of a driver in a sedan, and the corresponding heights of the visual blind spots are also different. Therefore, corresponding blind spot height thresholds are set for different vehicle types.

[0092] In one embodiment, the vehicle control unit determines the blind spot height threshold corresponding to the vehicle type of the current vehicle based on the vehicle type of the current vehicle and the association between the vehicle type and the blind spot height threshold. Optionally, assuming that the vehicle type of the current vehicle is SUV, the blind spot height threshold is determined to be 2.4 meters.

[0093] The vehicle control unit compares the blind spot height threshold with the vehicle height of the vehicle in front. If the vehicle height of the vehicle in front is greater than or equal to the blind spot height threshold, it means that the vehicle in front will constitute a visual blind spot for the driver of the current vehicle. At this time, a sound prompt message is generated as the target prompt message; if the vehicle height of the vehicle in front is less than the blind spot height threshold, it means that the vehicle in front will not constitute a visual blind spot for the driver of the current vehicle, that is, the driver can observe the changes in the lighting status, and no prompt message is generated at this time.

[0094] By determining the blind spot height threshold according to the vehicle type of the current vehicle and comparing the blind spot height threshold with the vehicle height, when the vehicle height is greater than or equal to the blind spot height threshold, a sound prompt message is generated as the target prompt message. Since the blind spot height threshold is determined according to the vehicle type, the accuracy of the visual blind spot judgment is guaranteed, avoiding the problem of low accuracy of visual blind spot judgment based on only a single blind spot height threshold.

[0095] Optionally, S2052 includes:

[0096] When the gaze area is the in-vehicle display area of ​​the current vehicle, visual prompt information is generated as target prompt information.

[0097] The in-vehicle display area refers to the area in the current vehicle where information can be visualized, including but not limited to the central control screen or the instrument panel screen, etc. Visual prompt information includes but is not limited to image prompt information and text prompt information, etc.

[0098] In one embodiment, if it is determined that the driver's gaze area is the in-vehicle display area of ​​the current vehicle, visual prompt information is generated for the in-vehicle display area, including but not limited to image prompt information and / or text prompt information on the central control screen, and image prompt information and / or text prompt information on the instrument panel screen.

[0099] By generating visual prompt information as target prompt information when the gaze area is the in-vehicle display area of ​​the current vehicle, the type of the generated target prompt information is made more reasonable, ensuring that the driver can perceive the target prompt information in time.

[0100] Optionally, S2052 includes:

[0101] When the attention area is other interior areas of the current vehicle or other exterior areas of the vehicle, sound prompt information is generated as target prompt information.

[0102] Among them, other in-car areas are in-car areas other than the in-car display area, such as when the driver lowers his head to play with his mobile phone or looks at the passengers in the car to talk, etc. Other outside-car areas are outside-car areas other than the front area outside the car, such as the left side area outside the car that the driver can see through the left windshield of the current vehicle, or the right side area outside the car that the driver can see through the right windshield of the current vehicle, etc.

[0103] In one implementation, if it is determined that the driver's gaze area is other interior areas of the current vehicle, or other exterior areas of the vehicle, then sound prompt information acting on the audio playback device is generated as target prompt information.

[0104] By generating sound prompt information as target prompt information when the gaze area is other areas inside the current vehicle or other areas outside the vehicle, the applicable scenarios for triggering prompts are expanded to ensure that corresponding target prompt information can be generated regardless of the driver's gaze area.

[0105] S206: Output target prompt information to the driver. Figure 2C is a schematic diagram of some gaze areas disclosed according to the embodiments of the present disclosure, such as Figure 2C As shown, 23 represents the front area outside the current vehicle, 24 represents the instrument area in the interior display area of ​​the current vehicle, and 25 represents the central control area in the interior display area of ​​the current vehicle.

[0106] Figure 3 The present invention is a schematic diagram of the structure of some vehicle information prompting devices disclosed in the embodiments of the present invention, which can be applied to the situation where the driver is prompted to start driving the vehicle at a traffic light intersection. The device of this embodiment can be implemented by software and / or hardware, and can be integrated on any electronic device with computing capabilities.

[0107] like Figure 3 As shown, the vehicle information prompting device 30 disclosed in this embodiment may include an image acquisition module 31, an eye state determination module 32, a prompt information generation module 33 and a prompt information output module 34, wherein:

[0108] The image acquisition module 31 is used to acquire the interior image of the current vehicle when the current light state of the target traffic light is the red light state;

[0109] The eye state determination module 32 is used to determine the current eye state of the driver in the current vehicle according to the collected image in the vehicle;

[0110] A prompt information generating module 33, used to generate corresponding target prompt information according to the current eye state;

[0111] The prompt information output module 34 is used to output target prompt information to the driver; wherein the target prompt information is used to prompt the driver to start driving. Optionally, the image acquisition module 31 is specifically used to:

[0112] Determine a target time when the light state of the target traffic light changes to a green light state, and determine a waiting time required for the light state to change to a green light state according to the current time and the target time;

[0113] The waiting time is compared with the time threshold value, and when the waiting time is less than or equal to the time threshold value, the interior captured image of the current vehicle is acquired.

[0114] Optionally, the prompt information generating module 33 is specifically used for:

[0115] When the current eye state is the non-attention state, tactile prompt information for the driver is generated as the target prompt information.

[0116] Optionally, the prompt information generating module 33 is specifically used for:

[0117] When the current eye state is a gaze state, the driver's current gaze area is determined, and corresponding target prompt information is determined according to the gaze area.

[0118] Optionally, the prompt information generating module 33 is further used for:

[0119] When the attention area is the area outside and in front of the current vehicle, a vehicle captured image of the vehicle in front of the current vehicle is obtained;

[0120] The vehicle height of the vehicle ahead is determined based on the vehicle collected image, and target prompt information is generated based on the vehicle height.

[0121] Optionally, the prompt information generating module 33 is further used for:

[0122] Determine a blind spot height threshold according to the vehicle type of the current vehicle, and compare the blind spot height threshold with the vehicle height;

[0123] When the vehicle height is greater than or equal to the blind spot height threshold, sound prompt information is generated as target prompt information.

[0124] Optionally, the prompt information generating module 33 is further used for:

[0125] When the gaze area is the in-vehicle display area of ​​the current vehicle, visual prompt information is generated as target prompt information.

[0126] Optionally, the prompt information generating module 33 is further used for:

[0127] When the gaze area is other interior areas of the current vehicle or other exterior areas of the vehicle, generating sound prompt information as target prompt information;

[0128] Among them, other in-vehicle areas are the in-vehicle areas except the in-vehicle display area, and other outside-vehicle areas are the outside-vehicle areas except the front area outside the vehicle.

[0129] The vehicle information prompting device 30 disclosed in the embodiment of the present disclosure can execute the vehicle information prompting method disclosed in the embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. The contents not described in detail in this embodiment can refer to the description in the embodiment of the method of the present disclosure.

[0130] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0131] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0132] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0133] like Figure 4 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0134] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0135] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as the vehicle information prompt method. For example, in some embodiments, the vehicle information prompt method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the vehicle information prompt method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the vehicle information prompt method in any other appropriate manner (e.g., by means of firmware).

[0136] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0138] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0141] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A vehicle information prompting method, comprising: When the current light state of the target traffic light is a red light state, obtaining an in-vehicle captured image of the current vehicle; Determining a current eye state of a driver in the current vehicle according to the in-vehicle collected image; Generating corresponding target prompt information according to the current eye state; Outputting the target prompt information to the driver; wherein the target prompt information is used to prompt the driver to start driving; The step of generating corresponding target prompt information according to the current eye state includes: When the current eye state is a gaze state, determining the driver's current gaze area, and generating corresponding target prompt information according to the gaze area; The step of generating corresponding target prompt information according to the gaze area includes: When the gaze area is the area outside and in front of the current vehicle, acquiring a vehicle captured image of a vehicle in front of the current vehicle; The vehicle height of the front vehicle is determined according to the vehicle captured image, and the target prompt information is generated according to the vehicle height.

2. The method according to claim 1, wherein: The step of acquiring the interior captured image of the current vehicle includes: Determine a target time at which the light state of the target traffic light changes to a green light state, and determine a waiting time required for the light state to change to a green light state according to the current time and the target time; The waiting time is compared with a time threshold value, and when the waiting time is less than or equal to the time threshold value, an in-vehicle captured image of the current vehicle is acquired.

3. The method according to claim 1, wherein: The generating corresponding target prompt information according to the current eye state also includes: When the current eye state is a non-attention state, tactile prompt information for the driver is generated as the target prompt information.

4. The method according to claim 1, wherein: The generating the target prompt information according to the vehicle height includes: Determining a blind spot height threshold according to the vehicle type of the current vehicle, and comparing the blind spot height threshold with the vehicle height; When the vehicle height is greater than or equal to the blind spot height threshold, sound prompt information is generated as the target prompt information.

5. The method according to claim 1, wherein: The generating corresponding target prompt information according to the gaze area includes: When the gaze area is an in-vehicle display area of ​​the current vehicle, visual prompt information is generated as the target prompt information.

6. The method according to claim 1, wherein: The generating corresponding target prompt information according to the gaze area includes: When the gaze area is another area inside the current vehicle or another area outside the vehicle, generating sound prompt information as the target prompt information; The other in-vehicle areas are in-vehicle areas except the in-vehicle display area, and the other out-vehicle areas are out-vehicle areas except the front area outside the vehicle.

7. A vehicle information prompting device, comprising: The image acquisition module is used to acquire the interior image of the current vehicle when the current light state of the target traffic light is the red light state; An eye state determination module, used to determine the current eye state of the driver in the current vehicle according to the in-vehicle collected image; A prompt information generating module, used for generating corresponding target prompt information according to the current eye state; A prompt information output module, used to output the target prompt information to the driver; wherein the target prompt information is used to prompt the driver to start driving; Wherein, the prompt information generating module is specifically used for: When the current eye state is a gaze state, determining the driver's current gaze area, and generating corresponding target prompt information according to the gaze area; Wherein, the prompt information generating module is further used for: When the gaze area is the area outside and in front of the current vehicle, acquiring a vehicle captured image of a vehicle in front of the current vehicle; The vehicle height of the front vehicle is determined according to the vehicle captured image, and the target prompt information is generated according to the vehicle height.

8. The device according to claim 7, wherein: The image acquisition module is specifically used for: Determine a target time at which the light state of the target traffic light changes to a green light state, and determine a waiting time required for the light state to change to a green light state according to the current time and the target time; The waiting time is compared with a time threshold value, and when the waiting time is less than or equal to the time threshold value, an in-vehicle captured image of the current vehicle is acquired.

9. The device according to claim 7, wherein: The prompt information generating module is further specifically used for: When the current eye state is a non-attention state, tactile prompt information for the driver is generated as the target prompt information.

10. The device according to claim 7, wherein: The prompt information generating module is further specifically used for: Determining a blind spot height threshold according to the vehicle type of the current vehicle, and comparing the blind spot height threshold with the vehicle height; When the vehicle height is greater than or equal to the blind spot height threshold, sound prompt information is generated as the target prompt information.

11. The device according to claim 7, wherein: The prompt information generating module is further specifically used for: When the gaze area is an in-vehicle display area of ​​the current vehicle, visual prompt information is generated as the target prompt information.

12. The device according to claim 7, wherein: The prompt information generating module is further specifically used for: When the gaze area is another area inside the current vehicle or another area outside the vehicle, generating sound prompt information as the target prompt information; The other in-vehicle areas are in-vehicle areas except the in-vehicle display area, and the other out-vehicle areas are out-vehicle areas except the front area outside the vehicle.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions to be executed by the at least one processor, where the instructions are executed by the at least one processor so as to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images