Vehicle take-over reminding method and device

Abstract

The invention discloses a vehicle takeover reminding method and device, and belongs to the technical field of automatic driving, the method is characterized in that a driver does not need to wear additional equipment through image information acquisition and posture type judgment, the scene of vehicle automatic driving to be switched to manual driving is adapted, the problems of inconvenient monitoring and poor scene adaptability in the prior art are solved, and the user experience is improved. Accurate recognition of the takeover ability of the driver in the non-driving task posture is achieved. And the takeover capability evaluation condition is determined based on the posture type, and the takeover prompt is sent in a targeted manner, so that the driver can make takeover preparation according to the takeover capability, the takeover performance is effectively improved, and the driving safety and driving comfort of the vehicle in the scene of switching from automatic driving to manual driving are improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a vehicle takeover warning method and device. Background Technology

[0002] In Level 3 conditional automated driving, the driver does not need to continuously hold the steering wheel and monitor the vehicle's driving status. However, when the vehicle needs to switch from automated driving to manual driving, the driver must respond to the takeover request and take over the vehicle in a timely manner. Non-driving tasks performed by the driver during the automated driving phase can affect the takeover capability to varying degrees, leading to a deterioration in the quality of the takeover and significantly threatening driving safety. In existing technologies, most driver takeover capability monitoring based on physiological signal characteristics requires the driver to wear specialized physiological monitoring equipment, which is inconvenient for real-world application and suffers from data processing delays, failing to meet real-time monitoring needs. Furthermore, driver state monitoring technologies based on computer vision are mostly designed for manual driving scenarios. In scenarios where the vehicle is transitioning from automated driving to manual driving, there are few applications of takeover capability monitoring and corresponding takeover alert methods based on the driver's non-driving task postures, making it difficult to guarantee driving safety in this scenario. Summary of the Invention

[0003] This application provides a vehicle takeover reminder method and device, which aims to effectively solve the problem that when the vehicle operation needs to switch from automatic driving to manual driving, the driver needs to respond to the takeover request and take over the vehicle in a timely manner. However, the non-driving tasks performed by the driver during the automatic driving stage will affect the takeover ability to varying degrees, resulting in poor takeover quality and greatly threatening driving safety.

[0004] In a first aspect, this application provides a vehicle takeover alert method, the method comprising: If the vehicle is in a state of transitioning from autonomous driving to manual driving, the image information of the vehicle's driver is acquired. Based on the image information, determine the driver's posture type; If the driver's posture type is a non-driving task posture type, the driver's takeover ability evaluation is determined based on the driver's posture type. Based on the driver's takeover ability assessment, the driver is given a driving takeover reminder.

[0005] Secondly, this application provides a vehicle takeover warning device, the device comprising: The first unit is used to acquire the image information of the driver of the vehicle when the vehicle is in the process of switching from an autonomous driving state to a manual driving state. The second unit is used to determine the driver's posture type based on the image information; The third unit is used to determine the driver's takeover capability evaluation based on the driver's posture type if the driver's posture type belongs to the non-driving task posture type. The fourth unit is used to remind the driver to take over driving based on the evaluation of the driver's takeover ability.

[0006] Thirdly, this application provides a readable medium including executable instructions, which, when executed by a processor of an electronic device, cause the electronic device to perform any of the methods described in the first aspect.

[0007] Fourthly, this application provides an electronic device including a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method as described in any of the first aspects.

[0008] As can be seen from the above technical solution, the vehicle takeover alert method provided in this application, when the vehicle is in a state of transitioning from autonomous driving to manual driving, first acquires the driver's image information, then determines the driver's posture type based on the image information. When the posture type is a non-driving task posture type, the driver's takeover ability evaluation is determined based on this posture type, and finally, a takeover alert is issued to the driver based on the takeover ability evaluation. It is evident that this method, through image information acquisition and posture type determination, eliminates the need for the driver to wear additional equipment, adapts to the scenario of switching from autonomous driving to manual driving, and solves the problems of inconvenient monitoring and poor scenario adaptability in existing technologies. It achieves accurate identification of the driver's takeover ability under non-driving task postures; by determining the takeover ability evaluation based on posture type and issuing targeted takeover alerts, the driver can prepare for takeover based on their own ability, effectively improving takeover performance, and thus enhancing driving safety and comfort when the vehicle switches from autonomous driving to manual driving.

[0009] The further effects of the aforementioned non-conventional preferred method will be explained below in conjunction with specific embodiments. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a vehicle takeover alert method provided in this application; Figure 2 A flowchart illustrating the process of constructing a driver posture feature recognition model provided in this application; Figure 3 A flowchart illustrating a multi-information fusion driver posture recognition process provided in this application; Figure 4 A flowchart illustrating the process of constructing a driver takeover capability evaluation model provided in this application; Figure 5 A flowchart illustrating a vehicle takeover alert method provided in this application; Figure 6 This application provides a structural schematic diagram of a vehicle takeover warning device; Figure 7 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The various non-limiting embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0014] See Figure 1 , Figure 5 This paper illustrates a vehicle takeover alert method according to an embodiment of the present application. The method includes the following steps: S101: If the vehicle is in a state of transitioning from automatic driving to manual driving, obtain the image information of the vehicle's driver.

[0015] In this embodiment, if the vehicle is in a state of transitioning from autonomous driving to manual driving, the image information of the vehicle's driver can be obtained.

[0016] Specifically, if the vehicle is in autonomous driving mode, and it is detected that the vehicle's driving conditions do not meet the conditions for autonomous driving, such as a traffic accident ahead, road construction requiring a detour, or a vehicle suddenly cutting in front, or if the surrounding environment or vehicle-related factors make the vehicle unsuitable for autonomous driving, then the vehicle's operating state can be adjusted from autonomous driving mode to a state awaiting switch to manual driving mode. That is, the vehicle is currently in autonomous driving mode, but it is in a stage of preparing and waiting to switch to manual driving mode.

[0017] When the vehicle is in the process of switching from autonomous driving to manual driving, the driver's image information can be obtained using the image acquisition device inside the vehicle (such as a camera installed inside the vehicle).

[0018] As an example, firstly, the vehicle monitors its own operating status in real time. When the vehicle is currently in autonomous driving mode, and the onboard sensors (such as millimeter-wave radar, cameras, lidar, etc.) detect that the driving conditions do not meet the autonomous driving conditions—for example, detecting a traffic accident ahead, road construction requiring a detour, or a vehicle suddenly cutting in front—the vehicle's autonomous driving system cannot continue to safely perform autonomous driving tasks. The vehicle will automatically adjust its operating status from autonomous driving mode to "waiting to switch from autonomous driving mode to manual driving mode."

[0019] When the vehicle is in the aforementioned waiting-to-switch operating state, the preset image acquisition device inside the vehicle is activated (this device is usually an in-vehicle camera, installed in a position that can cover the driver's seating area to ensure that the driver's head, upper body and hand movements can be clearly captured). The image acquisition device acquires the driver's image information in real time, and this image information will serve as the basis for subsequent determination of the driver's posture type.

[0020] S102: Determine the driver's posture type based on the image information.

[0021] After acquiring the image information, the driver's posture type can be determined based on the image information.

[0022] As an example, such as Figure 3 As shown, the driver's head posture features, object type features, and limb posture features can be determined first based on the image information. As an example, such as... Figure 3The image information can be input into a trained head pose and object recognition model to obtain the driver's head pose features and object type features. The image information can also be input into a trained limb pose recognition model to obtain the driver's limb pose features. The object type features are used to characterize the features of the object held by the driver in the image information; for example, they can indicate whether the driver is holding a mobile phone, a water cup, or a specific item.

[0023] In one implementation, the head pose and object recognition model can refer to a model trained on an improved YOLOv8n object detection model, used to extract the driver's head pose features and the features of the object being held. The limb pose recognition model refers to a model trained on an XGBoost (Extreme Gradient Boosting) machine learning model, used to extract the driver's limb pose features.

[0024] Then, the driver's posture type can be determined based on the driver's head posture characteristics, object type characteristics, and limb posture characteristics.

[0025] For example, such as Figure 3 As shown, the driver's image information acquired in S101 is input into the trained head pose and object recognition model and limb pose recognition model, respectively. The head pose and object recognition model processes the image information and outputs the driver's corresponding head pose features (e.g., looking up, looking down, etc.) and object type features (e.g., holding a mobile phone, no object, etc.). The limb pose recognition model processes the image information and outputs the driver's corresponding limb pose features (e.g., normal, operating, etc.). Combined with... Figure 3 The multi-information fusion logic combines the head posture features, object type features, and limb posture features obtained above to make a combined judgment. These three features can be combined to generate 48 driver posture features in a total of 3×4×4. Based on the non-driving task type corresponding to each combination feature, the driver's posture type is determined to be one of the following: monitoring task posture, visual task posture, simple operation task posture, or complex operation task posture (if the judgment result does not belong to the above four categories, it is a driving-related posture type). At the same time, in order to improve the stability of the judgment, the posture type that occupies the most frames in the image information within a preset time window (e.g., 1 second) is counted and used as the driver's final posture type to avoid frequent changes in the posture type judgment result due to false detection or missed detection in a few individual frames.

[0026] It is important to emphasize that the head pose and object recognition model, as well as the limb pose recognition model, have been trained before performing this step. Specific training methods will be discussed later. Figure 2 Explanation: Head pose and object recognition model training: such as Figure 2As shown, video data of drivers performing monitoring, vision, simple operation, and complex operation tasks during autonomous driving is collected using a driving simulator. Frames are extracted from the video at a preset frame rate (e.g., 5 frames per second) to obtain an image dataset. Images in the dataset are labeled with head posture types (head up, head down, left turn, right turn) and handheld object types (e.g., no object, mobile phone, water cup), constructing a head posture and object recognition training dataset. This training dataset is used to train an improved YOLOv8n object detection model for 100 rounds. The model with the best performance is selected as the final head posture and object recognition model. This model achieves a recognition accuracy of 97.6% and a recall rate of 97.8%, accurately extracting the driver's head posture features and object type features (object type features represent the characteristics of the object held by the driver) from the images.

[0027] Training a body pose recognition model: such as Figure 2 As shown, based on the video data collected by the aforementioned driving simulator, the video data was analyzed using a pose estimation model (YOLOv8-pose) to extract the coordinate information of 13 key points on the driver's upper body (these key points include the nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, and left and right hips, with each key point corresponding to a set of (x,y) coordinates), thus constructing a driver's limb pose coordinate dataset. This dataset was then standardized, with each set of data containing 26 dimensions (the horizontal and vertical coordinates of the 13 key points), and the limb poses were categorized into four types based on limb features: normal, driving, operating, and raising hands, completing the dataset labeling. The labeled dataset was then used to train the XGBoost machine learning model to obtain a limb pose recognition model. This model achieved a 98.3% recognition rate for limb movements and could accurately extract the driver's limb pose features.

[0028] S103: If the driver's posture type is a non-driving task posture type, determine the driver's takeover ability evaluation based on the driver's posture type.

[0029] In this embodiment, the non-driving task posture type refers to the posture type exhibited by the driver when performing non-driving related tasks during the autonomous driving phase. Specifically, it can include four categories: monitoring task posture, visual task posture, simple operation task posture, and complex operation task posture. The takeover capability evaluation refers to the level of the driver's ability to take over the vehicle determined based on the driver's posture type, specifically divided into four levels: good, relatively good, average, and poor.

[0030] In this embodiment, if the driver's posture type is determined to be a non-driving task posture type, the driver's takeover capability can be evaluated based on the driver's posture type. Specifically, if the driver's posture type is one of the following: monitoring task posture, visual task posture, simple operation task posture, or complex operation task posture, then the driver's posture type can be determined to be a non-driving task posture type.

[0031] Then, the driver's takeover capability assessment can be determined based on the driver's posture type. For example, if the driver's posture type is a monitoring task posture, the driver's takeover capability assessment is determined to be good; if the driver's posture type is a visual task posture, the driver's takeover capability assessment is determined to be relatively good; if the driver's posture type is a simple operation task posture, the driver's takeover capability assessment is determined to be average; and if the driver's posture type is a complex operation task posture, the driver's takeover capability assessment is determined to be poor.

[0032] In other words, after determining the driver's posture type, the system first determines whether the posture type belongs to the non-driving task posture type. If the driver's posture type is any one of the monitoring task posture, visual task posture, simple operation task posture, or complex operation task posture, then it is determined to belong to the non-driving task posture type. If it does not belong to the above four categories, then there is no need to execute the subsequent takeover capability determination process based on the non-driving task posture, and the system directly provides a reminder based on the takeover capability corresponding to the regular driving posture.

[0033] It should be noted that the monitoring posture refers to the posture of the driver during the autonomous driving phase, maintaining continuous attention to the vehicle's driving status and the road environment ahead. This can be understood as follows: the head is stable and upright, with the visual focus always on the road ahead or the vehicle's central control display area (such as the speed and road condition prompts); the hands are not holding non-driving objects such as mobile phones or water cups, and are naturally placed on either side of the steering wheel or the driver's seat armrest, or other areas where the steering wheel can be quickly operated; the overall body posture has no unnecessary movements, the upper body remains upright, and there is no significant deviation or twisting, so that the driver can quickly switch to driving operation mode when the autonomous driving system issues a takeover request.

[0034] Visual task posture refers to the posture of a driver during the autonomous driving phase when briefly shifting visual attention from driving-related scenarios to non-driving scenarios without involving hand operations. This can be understood as the head making obvious turning or tilting movements, such as turning the head to the left / right to check the blind spot in the rearview mirror, or quickly scanning the storage compartment in the car, with the visual focus shifting away from the road ahead or driving-related display areas; the hands are not holding any objects or only holding items that do not require operation (such as unused tissues), and there are no pressing, sliding, or other operational actions; the upper body may twist slightly, but the overall posture does not completely deviate from the driving posture and can be restored to a takeover state without significant adjustments.

[0035] Simple operation task posture refers to the typical posture of a driver performing a single, low-complexity non-driving operation during the autonomous driving phase, requiring only one hand. This can be understood as performing an operation with one hand, such as adjusting the air conditioning temperature by swiping the infotainment screen, answering a phone call (using a handheld device or Bluetooth headset), or retrieving a cup from the cup holder. The other hand remains within easy reach of the steering wheel from the driver's seat (e.g., armrest, steering wheel edge). Head posture may be slightly adjusted during the operation (e.g., looking down at the infotainment screen), but vision remains within the driver's field of vision. Overall body posture adjustments are small, and the driver can quickly return to a takeover state after stopping the operation.

[0036] Complex operational task postures refer to typical postures of drivers performing multi-step, highly complex non-driving operations during the autonomous driving phase, requiring the use of both hands or accompanied by significant limb adjustments. This can be understood as both hands being involved in non-driving operations, i.e., both hands are occupied, such as holding a phone to watch videos, operating a laptop, or adjusting the seat angle; or holding an object with one or both hands and performing a series of actions, such as holding a water cup and looking down to drink, or holding a makeup mirror to touch up makeup; head posture significantly deviating from the driving-related field of vision (e.g., looking down at a phone for an extended period); significant twisting, forward leaning, or backward leaning of the upper body; and requiring a considerable amount of time to adjust body posture and attention after stopping the operation to return to a take-over state.

[0037] When the posture type is determined to be a non-driving task posture type, combined with Figure 4 The evaluation logic of the (driver takeover capability evaluation model construction flowchart) determines the driver's takeover capability evaluation status based on the posture type: If the driver's posture type is a monitoring task posture, it means that the driver still pays attention to the road or vehicle status during the autonomous driving stage and does not perform complex non-driving tasks. The driver's readiness to take over from autonomous driving to manual driving is the highest, so the takeover capability evaluation is determined to be good. If the driver's posture type is a visual task posture (such as turning the head to look at an area outside the vehicle that is not in the driving direction), the driver's attention is partially detached from the driving-related scene, but it does not involve hand operation. When taking over, it is only necessary to turn the attention back to the driving scene. The level of preparation for taking over is the second best. Therefore, the assessment of the takeover ability is determined to be good. If the driver's posture is a simple operation task posture (such as operating the vehicle screen with one hand or making a phone call with one hand), the driver needs to stop the hand operation and turn his attention back to the driving scene in order to take over. The level of takeover preparation is lower than that of visual task posture, so the takeover ability is determined to be average. If the driver's posture is a complex task posture (such as using a mobile phone with both hands or drinking water), the driver needs to stop using both hands, adjust body posture, and concentrate before taking over the vehicle. The level of readiness for taking over is the lowest, so the assessment of the ability to take over is determined to be poor.

[0038] The criteria for judging the above-mentioned takeover capability are derived from takeover experiments based on a simulated driving platform: by inviting drivers of different genders, heights, weights, and driving habits to perform the above four types of non-driving tasks in a simulated autonomous driving to manual driving scenario, data such as driver takeover reaction time (reflecting takeover timeliness), maximum combined acceleration of the vehicle (reflecting operational stability), maximum lateral deviation (reflecting operational accuracy), and minimum collision time (TTC, reflecting safety) are collected. Combining the subjective level of AHP (analytic hierarchy process) (based on an intelligent driving expert database to determine indicator weights) and the objective level of entropy method (based on experimental data to determine indicator weights), a takeover capability evaluation model combining subjective and objective aspects is constructed, and finally the correspondence between different non-driving task posture types and takeover capability evaluation results is obtained.

[0039] S104: Based on the driver's takeover ability evaluation, provide the driver with a driving takeover reminder.

[0040] In this embodiment, the driver can be given a driving take-off reminder based on the evaluation of the driver's take-off ability.

[0041] Specifically, the takeover alert time for the driver can be determined first based on the driver's takeover ability assessment. For example, if the driver's takeover ability assessment is "good," the takeover alert time is determined to be a first duration; if the driver's takeover ability assessment is "fair," the takeover alert time is determined to be a second duration; if the driver's takeover ability assessment is "average," the takeover alert time is determined to be a third duration; and if the driver's takeover ability assessment is "poor," the takeover alert time is determined to be a fourth duration. The first duration is shorter than the second duration, the second duration is shorter than the third duration, and the third duration is shorter than the fourth duration. For example, "first duration," "second duration," "third duration," and "fourth duration" correspond to the default takeover alert time and the extended takeover alert time based on the takeover capability evaluation, respectively. The first duration is shorter than the second duration, the second duration is shorter than the third duration, and the third duration is shorter than the fourth duration. For example, the first duration is 7 seconds, the second duration is 8 seconds, the third duration is 9 seconds, and the fourth duration is 10 seconds. It should be emphasized that the above duration settings can be adjusted according to the actual application scenario, but the duration must still increase sequentially after adjustment.

[0042] Then, a driver takeover reminder can be issued based on the corresponding takeover reminder time. That is, the start time and duration of the driver takeover reminder can be determined based on the corresponding takeover reminder time. The driver takeover reminder can be issued in at least one of the following ways: voice reminder, text reminder, message reminder, light reminder, vibration reminder, etc. It should be emphasized that the driver takeover reminder can be the same or different depending on the corresponding takeover reminder time; this is not limited in this embodiment.

[0043] As an example, after determining the driver's takeover capability assessment, a corresponding takeover reminder time can be determined based on this assessment, and the driver can be reminded to take over driving at that time, specifically in conjunction with... Figure 5 (Flowchart of driver takeover capability monitoring based on posture recognition) is described below.

[0044] If the takeover capability assessment is good, it means that the driver can respond quickly to the takeover request. Therefore, the corresponding takeover reminder time is determined to be the first duration (e.g., 7 seconds). When the vehicle is in the waiting-to-switch operation state, the autonomous driving system immediately issues a driving takeover reminder (e.g., an audible and visual reminder or a voice reminder) that lasts for the first duration (e.g., 7 seconds) to remind the driver to take over the vehicle in a timely manner.

[0045] If the takeover capability assessment is good, the driver needs a shorter time to adjust their attention. Therefore, the corresponding takeover reminder time is set as the second duration (e.g., 8 seconds), which is extended by a certain amount of time (e.g., 1 second) compared to the first duration (e.g., 7 seconds) to give the driver time to return their attention to the driving scenario. Then, the driver is reminded to take over through sound and light reminders or voice prompts.

[0046] If the takeover capability assessment is rated as average, the driver needs to stop operating the vehicle and adjust their attention. Therefore, the corresponding takeover reminder time is set to the third duration (e.g., 9 seconds), which is extended by a certain amount (e.g., 2 seconds) compared to the first duration. This ensures that the driver has enough time to stop operating the vehicle and switch their attention before issuing the takeover reminder.

[0047] If the takeover capability assessment is poor, the driver needs more time to stop complex operations, adjust body posture, and concentrate. Therefore, the corresponding takeover reminder time is set to the fourth duration (e.g., 10 seconds), which is extended by a certain amount of time (e.g., 3 seconds) compared to the first duration. At the same time, the autonomous driving system prepares to execute the minimum risk strategy (e.g., slowly decelerate to the side of the road and stop). If the driver does not respond to the takeover request within the preset time (e.g., 10 seconds), the system will activate the minimum risk strategy to avoid a safety accident.

[0048] Through the above steps, when the vehicle is switching from autonomous driving to manual driving, it can accurately determine the driver's posture type and takeover capability based on the driver's image information, and then provide appropriate takeover reminders. While taking into account the driving experience, it can effectively improve driving safety in the scenario of switching from autonomous driving to manual driving.

[0049] As can be seen from the above technical solution, the vehicle takeover alert method provided in this application, when the vehicle is in a state of transitioning from autonomous driving to manual driving, first acquires the driver's image information, then determines the driver's posture type based on the image information. When the posture type is a non-driving task posture type, the driver's takeover ability evaluation is determined based on this posture type, and finally, a takeover alert is issued to the driver based on the takeover ability evaluation. It is evident that this method, through image information acquisition and posture type determination, eliminates the need for the driver to wear additional equipment, adapts to the scenario of switching from autonomous driving to manual driving, and solves the problems of inconvenient monitoring and poor scenario adaptability in existing technologies. It achieves accurate identification of the driver's takeover ability under non-driving task postures; by determining the takeover ability evaluation based on posture type and issuing targeted takeover alerts, the driver can prepare for takeover based on their own ability, effectively improving takeover performance, and thus enhancing driving safety and comfort when the vehicle switches from autonomous driving to manual driving.

[0050] like Figure 6The image shows a specific embodiment of a vehicle takeover warning device provided in this application. The device described in this embodiment is a physical device used to perform the method described in the above embodiments. Its technical solution is essentially the same as that of the above embodiments, and the corresponding descriptions in the above embodiments are also applicable to this embodiment. The device in this embodiment includes: The first unit 601 is used to acquire the image information of the driver of the vehicle when the vehicle is in the process of switching from an automatic driving operation state to a manual driving operation state. The second unit 602 is used to determine the driver's posture type based on the image information; The third unit 603 is used to determine the driver's takeover ability evaluation based on the driver's posture type if the driver's posture type is a non-driving task posture type. The fourth unit 604 is used to remind the driver to take over driving based on the evaluation of the driver's takeover ability.

[0051] Optionally, if the vehicle is in a state of transitioning from autonomous driving to manual driving, acquiring the driver's image information includes: If the vehicle is in autonomous driving mode and the vehicle's driving conditions do not meet the autonomous driving conditions, then the vehicle's operating status will be changed from autonomous driving mode to manual driving mode. When the vehicle is in the process of switching from automatic driving to manual driving, the image information of the driver is acquired using the image acquisition device inside the vehicle.

[0052] Optionally, determining the driver's posture type based on the image information includes: Based on the image information, determine the driver's head posture features, object type features, and limb posture features; The driver's posture type is determined based on the driver's head posture characteristics, object type characteristics, and limb posture characteristics.

[0053] Optionally, determining the driver's head posture features, object type features, and limb posture features based on the image information includes: The image information is input into a trained head pose and object recognition model to obtain the head pose features and object type features corresponding to the driver; wherein, the object type features are used to characterize the feature information of the object held by the driver in the image information. The image information is input into a trained limb posture recognition model to obtain the driver's limb posture features.

[0054] Optionally, if the driver's posture type belongs to a non-driving task posture type, determining the driver's takeover capability evaluation based on the driver's posture type includes: If the driver's posture type is one of the following: monitoring task posture, visual task posture, simple operation task posture, or complex operation task posture, then the driver's posture type is determined to be a non-driving task posture type. The driver's takeover capability is evaluated based on the driver's posture type.

[0055] Optionally, determining the driver's takeover capability assessment based on the driver's posture type includes: If the driver's posture type is a monitoring task posture, then the driver's takeover capability evaluation is determined to be good. If the driver's posture type is a visual task posture, then the driver's takeover ability is assessed as good. If the driver's posture type is a simple operation task posture, then the driver's takeover capability evaluation is determined to be average. If the driver's posture type is a complex operation task posture, then the driver's takeover capability is determined to be poor.

[0056] Optionally, the step of providing a driving take-off reminder to the driver based on the driver's take-off ability evaluation includes: Based on the driver's takeover capability assessment, determine the corresponding takeover reminder time for the driver; Based on the corresponding takeover reminder time for the driver, a driving takeover reminder is given to the driver.

[0057] Optionally, determining the corresponding takeover reminder time for the driver based on the driver's takeover capability evaluation includes: If the driver's takeover capability assessment is good, then the takeover reminder time for the driver is determined to be the first duration. If the driver's takeover ability is rated as good, then the takeover reminder time for the driver is determined to be the second duration. If the driver's takeover capability assessment is rated as average, then the corresponding takeover reminder time for the driver is determined to be the third duration. If the driver's takeover ability assessment is poor, then the corresponding takeover reminder time for the driver is determined to be the fourth duration. Wherein, the first duration is shorter than the second duration, the second duration is shorter than the third duration, and the third duration is shorter than the fourth duration.

[0058] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.

[0059] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0060] Memory is used to store instructions for execution. Specifically, instructions for execution are computer programs that can be executed. Memory can include main memory and non-volatile memory, and it provides the processor with execution instructions and data.

[0061] In one possible implementation, the processor reads the corresponding execution instructions from non-volatile memory into memory and then executes them. Alternatively, it may obtain the corresponding execution instructions from other devices to form a vehicle takeover warning device at the logical level. The processor executes the execution instructions stored in memory to implement the vehicle takeover warning method provided in any embodiment of this application.

[0062] The above is as stated in this application. Figure 1The method executed by the vehicle takeover warning device provided in the illustrated embodiment can be applied to a processor, or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0063] The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0064] This application also proposes a readable medium that stores execution instructions. When the stored execution instructions are executed by the processor of an electronic device, the electronic device can execute the vehicle takeover reminder method provided in any embodiment of this application, and specifically be used to perform the above-mentioned evaluation method.

[0065] The electronic devices described in the foregoing embodiments may be computers.

[0066] Those skilled in the art will understand that the embodiments of this application can be provided as methods or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.

[0067] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0069] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A vehicle takeover alert method, characterized in that, The method includes: If the vehicle is in a state of transitioning from autonomous driving to manual driving, the image information of the vehicle's driver is acquired. Based on the image information, determine the driver's posture type; If the driver's posture type is a non-driving task posture type, the driver's takeover ability evaluation is determined based on the driver's posture type. Based on the driver's takeover ability assessment, the driver is given a driving takeover reminder.

2. The method according to claim 1, characterized in that, If the vehicle is in a state of transitioning from autonomous driving to manual driving, the image information of the driver is acquired, including: If the vehicle is in autonomous driving mode and the vehicle's driving conditions do not meet the autonomous driving conditions, then the vehicle's operating status will be changed from autonomous driving mode to manual driving mode. When the vehicle is in the process of switching from automatic driving to manual driving, the image information of the driver is acquired using the image acquisition device inside the vehicle.

3. The method according to claim 1, characterized in that, Determining the driver's posture type based on the image information includes: Based on the image information, determine the driver's head posture features, object type features, and limb posture features; The driver's posture type is determined based on the driver's head posture characteristics, object type characteristics, and limb posture characteristics.

4. The method according to claim 3, characterized in that, The step of determining the driver's head posture features, object type features, and limb posture features based on the image information includes: The image information is input into a trained head pose and object recognition model to obtain the head pose features and object type features corresponding to the driver; wherein, the object type features are used to characterize the feature information of the object held by the driver in the image information. The image information is input into a trained limb posture recognition model to obtain the driver's limb posture features.

5. The method according to claim 1, characterized in that, If the driver's posture type is a non-driving task posture type, the driver's takeover capability assessment is determined based on the driver's posture type, including: If the driver's posture type is one of the following: monitoring task posture, visual task posture, simple operation task posture, or complex operation task posture, then the driver's posture type is determined to be a non-driving task posture type. The driver's takeover capability is evaluated based on the driver's posture type.

6. The method according to claim 1 or 5, characterized in that, The step of determining the driver's takeover capability assessment based on the driver's posture type includes: If the driver's posture type is a monitoring task posture, then the driver's takeover capability evaluation is determined to be good. If the driver's posture type is a visual task posture, then the driver's takeover ability is assessed as good. If the driver's posture type is a simple operation task posture, then the driver's takeover capability evaluation is determined to be average. If the driver's posture type is a complex operation task posture, then the driver's takeover capability is determined to be poor.

7. The method according to claim 1, characterized in that, The step of providing a driving take-off reminder to the driver based on the driver's take-off ability evaluation includes: Based on the driver's takeover capability assessment, determine the corresponding takeover reminder time for the driver; Based on the corresponding takeover reminder time for the driver, a driving takeover reminder is given to the driver.

8. The method according to claim 7, characterized in that, The step of determining the corresponding takeover reminder time for the driver based on the driver's takeover capability evaluation includes: If the driver's takeover capability assessment is good, then the takeover reminder time for the driver is determined to be the first duration. If the driver's takeover ability is rated as good, then the takeover reminder time for the driver is determined to be the second duration. If the driver's takeover capability assessment is rated as average, then the corresponding takeover reminder time for the driver is determined to be the third duration. If the driver's takeover ability assessment is poor, then the corresponding takeover reminder time for the driver is determined to be the fourth duration. Wherein, the first duration is shorter than the second duration, the second duration is shorter than the third duration, and the third duration is shorter than the fourth duration.

9. A vehicle takeover warning device, characterized in that, The device includes: The first unit is used to acquire the image information of the driver of the vehicle when the vehicle is in the process of switching from an autonomous driving state to a manual driving state. The second unit is used to determine the driver's posture type based on the image information; The third unit is used to determine the driver's takeover capability evaluation based on the driver's posture type if the driver's posture type belongs to the non-driving task posture type. The fourth unit is used to remind the driver to take over driving based on the evaluation of the driver's takeover ability.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory storing execution instructions. When the processor executes the execution instructions stored in the memory, the processor performs the method as described in any one of claims 1-8.

Images