Vehicle control method based on human-vehicle interaction, vehicle and storage medium

Abstract

This invention discloses a vehicle control method, vehicle, and storage medium based on human-vehicle interaction, belonging to the field of vehicle control. The vehicle control method includes: identifying whether a target occupant is inside the vehicle; if a target occupant is identified, outputting a first inquiry message to the driver to inquire whether the driver agrees to the target occupant participating in the driving task; if confirmation from the driver is received, activating a target entertainment function for the target occupant to participate in the driving task, enabling the target occupant to interact with the vehicle through the target entertainment function to participate in the driving task. This invention solves the technical problem of poor occupant-assisted driving performance in vehicles.

Description

Technical Field

[0001] This invention belongs to the field of vehicle control, and particularly relates to a vehicle control method, vehicle, and storage medium based on human-vehicle interaction. Background Technology

[0002] Child passengers often unknowingly increase the psychological burden on drivers. To optimize driving scenarios with child passengers, in-vehicle solutions should be designed to effectively meet the unique needs of child passengers, thereby assisting drivers in driving with greater peace of mind.

[0003] The relevant technologies provide educational and entertainment content for children through built-in robotic companions or voice interaction systems, helping drivers maintain children's interest during journeys and assisting drivers in driving with peace of mind to some extent. However, the educational and entertainment content provided to children and other passengers in these technologies lacks human-vehicle interaction, preventing children and other passengers from deeply participating in the vehicle's driving tasks. Consequently, the vehicle cannot obtain assistance from children and other passengers, resulting in poor effectiveness of passenger-assisted driving. Summary of the Invention

[0004] This invention provides a vehicle control method, vehicle, and storage medium based on human-vehicle interaction to solve the technical problem of poor performance of occupant-assisted driving vehicles.

[0005] In a first aspect, a vehicle control method based on human-vehicle interaction is provided, comprising: identifying whether there is a target occupant in the vehicle; when a target occupant is identified in the vehicle, outputting a first inquiry message, the first inquiry message being used to inquire whether the driver in the vehicle agrees to the target occupant participating in a driving task; if a confirmation message from the driver in the vehicle to the first inquiry message is received, activating a target entertainment function for the target occupant to participate in a driving task, so that the target occupant can interact with the vehicle based on the target entertainment function to participate in the driving task.

[0006] In conjunction with the first aspect, in some embodiments, before activating the target entertainment function for the target occupant to participate in driving tasks, the method further includes: obtaining candidate entertainment functions from a preset set of M entertainment functions, where M is an integer greater than 1; outputting a second query message for the candidate entertainment function, the second query message being used to ask the driver in the vehicle or the target occupant whether they agree to activate the candidate entertainment function; if confirmation information from the driver in the vehicle or the target occupant regarding the second query message is received, the candidate entertainment function is adopted as the target entertainment function.

[0007] In conjunction with the first aspect, in some embodiments, obtaining candidate entertainment functions from a set of M preset entertainment functions includes: obtaining the current driving mode of the vehicle, wherein entertainment functions for the target occupant to participate in driving tasks are preset for different driving modes of the vehicle; and obtaining the candidate entertainment functions from the entertainment functions preset for the current driving mode of the vehicle.

[0008] In conjunction with the first aspect, in some embodiments, the target entertainment function is a first entertainment function, which is used for the target occupant to participate in planning the vehicle's driving route. After activating the target entertainment function for the target occupant to participate in the driving task, the method further includes: obtaining navigation information provided by the target occupant; generating a new navigation route based on the navigation information and outputting a third inquiry message, which is used to ask the driver in the vehicle whether they agree to the new navigation route; if the driver in the vehicle confirms the third inquiry message, updating the vehicle's driving route to the new navigation route.

[0009] In conjunction with the first aspect, in some embodiments, the target entertainment function is a second entertainment function for the target occupant to simulate driving the vehicle. After activating the target entertainment function for the target occupant to participate in the driving task, the method further includes: acquiring simulated driving information generated by the target occupant performing simulated driving behavior, and acquiring real driving information of the vehicle; comparing the simulated driving information with the real driving information, and outputting feedback information characterizing whether the simulated driving behavior of the target occupant is correct based on the comparison result.

[0010] In conjunction with the first aspect, in some embodiments, comparing the simulated driving information with the real driving information and outputting feedback information based on the comparison result includes: comparing whether the simulated driving information and the real driving information are consistent; if consistent, outputting first feedback information to the target occupant, wherein the first feedback information indicates that the target occupant's simulated driving behavior is correct; if inconsistent, determining whether the target occupant has engaged in dangerous driving behavior based on the simulated driving information, and outputting second feedback information based on the determination result, wherein the second feedback information indicates that the target occupant's simulated driving behavior is incorrect.

[0011] In conjunction with the first aspect, in some embodiments, the target entertainment function is a third entertainment function preset for the vehicle's autonomous driving mode. This third entertainment function is used to allow the target occupant to provide driving suggestions to the autonomous driving system. After activating the target entertainment function for the target occupant to participate in driving tasks, the system further includes: acquiring the driving suggestions provided by the target occupant; outputting a third inquiry message, which is used to inquire whether the driver in the vehicle agrees with the driving suggestions; if confirmation information from the driver in the vehicle regarding the third inquiry message is received, determining whether the driving suggestions meet driving safety conditions through the vehicle's autonomous driving system; and if the driving suggestions meet the driving safety conditions, controlling the autonomous driving system to execute driving actions based on the driving suggestions.

[0012] In conjunction with the first aspect, in some embodiments, the target occupant is a child occupant in the rear seat of the vehicle; identifying whether there is a target occupant in the vehicle includes: identifying whether there is a child occupant in the rear seat of the vehicle through at least one of the following dimensions: identifying whether there is a child occupant in the rear seat based on an image of the rear seat area captured by an in-vehicle camera; identifying whether there is a child occupant in the rear seat based on pressure information collected by a gravity sensor installed in the rear seat; and identifying whether there is a child occupant in the rear seat based on the occupant's vital signs information.

[0013] In a second aspect of the invention, a vehicle is provided, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the vehicle control method based on human-vehicle interaction as described in any embodiment of the first aspect.

[0014] In a third aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, the computer program including executable instructions that, when executed by a processor, implement the vehicle control method based on human-vehicle interaction as described in any embodiment of the first aspect.

[0015] The technical solutions provided by one or more embodiments of the present invention achieve at least the following technical effects or advantages:

[0016] By identifying whether there is a target occupant inside the vehicle, and upon identification of a target occupant, a first inquiry is output to ask the driver whether they agree to the target occupant participating in the driving task. If the driver confirms the first inquiry, the target entertainment function for the target occupant to participate in the driving task is activated, allowing the target occupant to interact with the vehicle through the target entertainment function to participate in the driving task. This enables the target occupant to deeply participate in the driving task through human-computer interaction, assisting in driving the vehicle, thereby increasing the dimension of vehicle control and improving the interactivity between the occupant and the vehicle, thus enhancing the occupant experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 A flowchart of a vehicle control method based on human-vehicle interaction according to some embodiments of the present invention is shown;

[0019] Figure 2 The control logic of a vehicle control method based on human-vehicle interaction according to some embodiments of the present invention is shown;

[0020] Figure 3 The human-computer interaction process of a first entertainment function according to some embodiments of the present invention is illustrated;

[0021] Figure 4 The human-computer interaction process of a second entertainment function according to some embodiments of the present invention is illustrated;

[0022] Figure 5 The human-computer interaction process of a third entertainment function according to some embodiments of the present invention is illustrated;

[0023] Figure 6 A schematic diagram of the structure of a vehicle according to some embodiments of the present invention is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] First, it should be clarified that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0026] Figure 1 A flowchart of a vehicle control method based on human-vehicle interaction according to some embodiments of the present invention is shown. For example... Figure 1 As shown, the vehicle control method based on human-vehicle interaction includes the following steps S101 to S103:

[0027] In step S101: Identify whether there is a target occupant inside the vehicle.

[0028] In some embodiments, the target occupant is any occupant other than the driver. In other embodiments, the target occupant is any passenger in the rear seat, so as not to affect the driver's driving behavior. In still other embodiments, the target occupant is a child passenger in the rear seat.

[0029] In some embodiments, the presence of a target occupant in the vehicle is identified based on the vehicle's start signal, so that the target occupant can participate in the driving task in a timely manner after the vehicle is started.

[0030] In other embodiments, after the vehicle starts, it is identified whether the vehicle is traveling on a target road segment or in a target road condition. If the vehicle is identified as traveling on a target road segment or in a target road condition, it is identified whether there is a target occupant inside the vehicle. The target road segment can be a road segment set by the driver, such as a road segment defined by a start and end point set by the driver. The target road segment can also be a relatively safe driving segment pushed to the vehicle by the server based on historical road traffic data. The target road segment can also be a relatively safe driving segment pushed to the vehicle by the server based on the road conditions ahead. The target road condition is a relatively safe driving condition, such as a non-congested road condition.

[0031] In some other embodiments, after the vehicle is started, it is identified whether the vehicle is traveling during a target time period; if the vehicle is identified as traveling during the target time period, it is identified whether there is a target occupant inside the vehicle. It is understood that the target time period can be set by the driver or set at the factory.

[0032] In some embodiments, if the target occupant is a child occupant in the rear seat, the step of identifying whether there is a target occupant in the vehicle may include: identifying whether there is a child occupant in the rear seat of the vehicle by at least one of the following dimensions:

[0033] Dimension 1: Identify whether there are child occupants in the rear seats based on images of the rear seat area captured by the in-vehicle camera.

[0034] Understandably, the system uses an in-car camera to capture images of the rear seat area, extracts facial features from these images, and determines if there are occupants in the rear seats if facial features are extracted. The extracted facial features are then used to identify whether the occupants in the rear seats are children.

[0035] Dimension 2: Identify whether there are child occupants in the rear seats based on pressure information collected by gravity sensors installed in the rear seats.

[0036] Understandably, the gravity sensor is a pressure sensor or weight sensor installed under the seat or backrest of the rear seat. It detects changes in pressure or weight on the seat or backrest of the rear seat to determine whether there are child occupants in the rear seat.

[0037] Dimension 3: Identify whether there are child occupants in the rear seats based on the occupants' physical characteristics.

[0038] Understandably, occupant vital signs information can include one or more of the following: respiratory rate, heart rate, and infrared radiation. These can be detected using millimeter-wave radar or ultrasonic radar to determine if there are child occupants in the rear seats. In low-light conditions, infrared night vision cameras can capture the infrared radiation of rear seat occupants to identify the presence of child occupants in the rear seats at night or under low light conditions.

[0039] To improve accuracy, the above three dimensions are combined to identify whether there are child occupants in the rear seats. If a child occupant is identified in the rear seats in all three dimensions, then it is determined that there is a child occupant in the rear seats.

[0040] In step S102: If a target occupant is identified in the vehicle, a first inquiry message is output. The first inquiry message is used to ask the driver in the vehicle whether he agrees to the target occupant participating in the driving task.

[0041] In step S102, the first query information can be output using any of the following output methods, or using both of the following output methods together:

[0042] Output method 1: Output a voice message representing the first inquiry through the vehicle's voice interaction system, asking the driver in the vehicle whether they agree to the target occupant participating in the driving task.

[0043] Output method 2: The first vehicle touch screen outputs graphic and textual content representing the first inquiry information, asking the driver whether he agrees to the target passenger participating in the driving task. The first vehicle touch screen is a touch screen located in front of the driver's seat.

[0044] In step S103: If the driver in the vehicle confirms the first inquiry information, the target entertainment function for the target occupant to participate in the driving task is activated, so that the target occupant can interact with the vehicle based on the target entertainment function to participate in the driving task.

[0045] In some embodiments, if a denial message from the driver inside the vehicle is received after the first inquiry message is output, the process ends and no entertainment functions for the target occupant to participate in driving tasks are activated.

[0046] It should be noted that the confirmation information for the first inquiry can be input by the driver in the vehicle through at least one of the following methods: input via the first in-vehicle touchscreen, voice input, gesture input, etc.

[0047] In some embodiments, if the voice representing the first inquiry is output through the vehicle's voice interaction system, then upon receiving a voice command from the driver indicating agreement to the target occupant's participation in the driving task (such as preset voice commands like "yes," "agree," or "can"), it is determined that confirmation information from the driver regarding the first inquiry has been received. Conversely, upon receiving a voice command from the driver indicating refusal to allow the target occupant to participate in the driving task (such as preset voice commands like "no," "disagree," or "cannot"), it is determined that denial information from the driver regarding the first inquiry has been received.

[0048] In other embodiments, the first in-vehicle touchscreen outputs graphic and textual content representing the first inquiry information. Upon receiving a confirmation operation indicating agreement to the target occupant's participation in the driving task, the first in-vehicle touchscreen determines that confirmation information from the driver has been received regarding the first inquiry information. Options for the driver to perform a confirmation or cancellation operation can be presented as a pop-up window on the first in-vehicle touchscreen. If a confirmation operation is received on the first in-vehicle touchscreen, confirmation information from the driver regarding the first inquiry information is confirmed; if a cancellation operation is received on the first in-vehicle touchscreen, denial information from the driver regarding the first inquiry information is confirmed.

[0049] In some other embodiments, if the voice representing the first inquiry is output through the vehicle's voice interaction system, and the in-vehicle camera captures a gesture indicating that the driver agrees to the target occupant's participation in the driving task, it is determined that the driver's confirmation of the first inquiry has been received.

[0050] Understandably, the system uses an in-vehicle camera to capture hand gestures in the driver's seat area and compares these gestures with a first target gesture. If they match, it confirms that the driver has received confirmation of the first inquiry. The system then compares these gestures with a second target gesture; if they match, it confirms that the driver has received denial of the first inquiry.

[0051] It is understood that preset entertainment functions may include one or more, wherein the preset entertainment functions are entertainment functions designed for the target occupants to participate in driving tasks. In some embodiments, the preset entertainment functions include one or more of the following:

[0052] Primary entertainment function: used to allow target occupants to participate in planning the vehicle's driving route;

[0053] Second entertainment function: used to allow target occupants to simulate driving a vehicle;

[0054] The third entertainment function is used to allow the target occupants to provide driving suggestions to the autonomous driving system.

[0055] In some embodiments, if there is only one entertainment function for the target occupant to participate in driving tasks, then this entertainment function is the target entertainment function. If the driver in the vehicle confirms the first inquiry information, the target entertainment function for the target occupant to participate in driving tasks is directly activated.

[0056] In other embodiments, there are multiple preset entertainment functions for the target occupant to participate in driving tasks. Before activating the target entertainment function for the target occupant to participate in driving tasks, the method may further include: obtaining candidate entertainment functions from M preset entertainment functions, where M is an integer greater than 1; outputting a second query message for the candidate entertainment function, the second query message being used to ask the driver or target occupant in the vehicle whether they agree to activate the candidate entertainment function; if a confirmation message from the driver in the vehicle is received regarding the second query message, the candidate entertainment function is adopted as the target entertainment function.

[0057] In some embodiments, after outputting a second query message for a candidate entertainment function, if a denial message from the driver in the vehicle is received regarding the second query message, the next candidate entertainment function is obtained from a preset set of M entertainment functions, and a second query message for the next candidate entertainment function is output. This process is repeated until a confirmation message from the driver in the vehicle is received regarding the second query message or no entertainment function can be selected.

[0058] It is understandable that the output method of the second inquiry information can be the same as or similar to that of the first inquiry information. Alternatively, the second inquiry information can be output via the vehicle's voice interaction system, asking the driver whether they agree to activate the candidate entertainment function. The second inquiry information can also be displayed as text or graphics on the vehicle's in-vehicle screen, asking the driver whether they agree to activate the candidate entertainment function.

[0059] It is understood that the confirmation and denial information for the second inquiry can be input by the driver through at least one of the following methods: in-vehicle touchscreen, voice input, or gesture input. For details, please refer to the description of the confirmation and denial information for the first inquiry above; for the sake of brevity, it will not be repeated here.

[0060] In some embodiments, regardless of the vehicle's driving mode, one of the preset M entertainment functions can be randomly selected as a candidate entertainment function.

[0061] In other embodiments, the vehicle has multiple driving modes, including autonomous driving mode and manual driving mode. The manual driving mode includes L0, L1, and L2 level driving modes, while the autonomous driving mode includes L3 level and above. To ensure that the activated entertainment functions are more compatible with the vehicle's driving mode, entertainment functions for target occupants to participate in driving tasks are preset for each of the vehicle's different driving modes. Obtaining candidate entertainment functions from the preset M entertainment functions can include: obtaining the vehicle's current driving mode; and, based on the current driving mode, obtaining candidate entertainment functions from the entertainment functions preset for the current driving mode, thereby enabling more accurate delivery of entertainment functions suitable for the driving mode.

[0062] In some embodiments, one or more of the first, second, and third entertainment functions described above may be preset for the autonomous driving mode. In some embodiments, one or more of the first and second entertainment functions described above may be preset for the manual driving mode.

[0063] In some embodiments, each driving mode of the vehicle corresponds to only one entertainment function, meaning there is a one-to-one correspondence between driving modes and entertainment functions. The current driving mode of the vehicle is obtained, and the target entertainment function is determined based on the current driving mode.

[0064] like Figure 2 As shown, taking a child passenger in the rear seat as an example, the following provides a logic of the vehicle control method based on human-computer interaction in an embodiment of the present invention:

[0065] Step S1: Start the vehicle;

[0066] Step S2: Identify whether there are child occupants in the rear seats. If so, proceed to step S3.

[0067] Step S3: Output the first query message to ask the driver whether they agree to allow a child passenger in the back seat to participate in the driving task;

[0068] Step S4: Has confirmation information for the first inquiry been received from the driver inside the vehicle? If yes, proceed to step S5.

[0069] Step S5: Detect the vehicle's current driving mode. If the current driving mode is manual driving mode, proceed to step S6; if the current driving mode is automatic driving mode, proceed to step S7.

[0070] S6: Activate the target entertainment function corresponding to the manual driving mode for child occupants in the rear seats to participate in driving tasks, and proceed to step S7;

[0071] S7: Activate the target entertainment function corresponding to the automatic driving mode for children in the rear seats to participate in driving tasks, and proceed to step S7;

[0072] S8: Has an instruction to exit the target entertainment function been received from the driver inside the vehicle? If yes, proceed to step S9; otherwise, return to step S7.

[0073] S9: Disable target entertainment functions.

[0074] In some embodiments, if the target entertainment function is a first entertainment function for allowing target occupants to participate in planning the vehicle's driving route, then after activating the first entertainment function, the method further includes: obtaining navigation information provided by the target occupant; generating a new navigation route based on the navigation information and outputting a third inquiry message, the third inquiry message being used to ask the driver in the vehicle whether they agree to the new navigation route; if confirmation information from the driver in the vehicle regarding the third inquiry message is received, updating the vehicle's driving route to the new navigation route. This facilitates the involvement of rear-seat target occupants in the planning and decision-making process for the vehicle's driving path.

[0075] In some embodiments, if a denial message is received from the driver inside the vehicle regarding the third inquiry, the navigation update is exited, and a message indicating that the navigation route cannot be changed is displayed.

[0076] In some embodiments, the step of providing a prompt that the navigation route cannot be changed may include: displaying a prompt that the navigation route cannot be changed on a second in-vehicle touchscreen at the location of the target occupant and / or prompting the target occupant that the navigation route cannot be changed via voice broadcast.

[0077] In other implementations, if no confirmation from the driver in the vehicle is received regarding the third inquiry within a preset waiting time, the navigation update is exited and a message indicating that the navigation route cannot be changed is displayed on the second in-vehicle touchscreen at the target passenger's location, and / or the target passenger is notified via voice broadcast that the navigation route cannot be changed.

[0078] To facilitate understanding of the embodiments of the present invention, the following references are made. Figure 3 The human-computer interaction process for the first entertainment function is given:

[0079] S31: Receive navigation information input by the target occupant into the vehicle's infotainment system;

[0080] S32: Generate a new navigation route based on navigation information;

[0081] S33: Output the third query message to ask the driver in the vehicle whether they agree to the new navigation route;

[0082] S34: Has confirmation information for the third inquiry from the driver inside the vehicle been received? If yes, proceed to step S35; otherwise, proceed to step S36.

[0083] S35: Update the vehicle's route to the new navigation route.

[0084] S36: Exit navigation update and provide a message indicating that the navigation route cannot be changed.

[0085] In some embodiments, the input navigation information includes settings for the changed navigation destination and route points.

[0086] In some embodiments, obtaining navigation information provided by a target occupant includes: receiving navigation information input by the target occupant through a second vehicle touchscreen, and synchronizing the navigation information input through the second vehicle touchscreen to the first vehicle touchscreen through the linkage between the second vehicle touchscreen and the first vehicle touchscreen, so that the driver in the vehicle can see the navigation information input by the target occupant.

[0087] In other embodiments, obtaining navigation information provided by the target occupant may include receiving navigation information input by voice from the target occupant through a voice interaction system (such as a voice assistant) on the vehicle's infotainment system (IVI).

[0088] In some embodiments, generating a new navigation route based on navigation information and outputting a third inquiry message may include: displaying the new navigation route and graphic content representing the third inquiry message through a first in-vehicle touchscreen to ask the driver whether they agree to the new navigation route.

[0089] It is understandable that the confirmation and denial information of the third inquiry can be input by the driver in the vehicle through at least one of the following methods: in-vehicle touch screen input, voice input, gesture input, etc. For details, please refer to the description of the confirmation and denial information of the first inquiry in the previous text. For the sake of brevity, it will not be repeated here.

[0090] In some embodiments, options for confirmation and cancellation operations can be presented in the form of a pop-up window on the first in-vehicle touchscreen. Upon receiving a confirmation operation performed on the first in-vehicle touchscreen, it is determined that the driver in the vehicle has received confirmation information for the third inquiry. Upon receiving a cancellation operation performed on the first in-vehicle touchscreen, it is determined that the driver in the vehicle has received denial information for the third inquiry.

[0091] In other embodiments, upon receiving a voice command from the driver indicating agreement to a new navigation route (such as preset voice commands like "yes," "agree," or "allow"), it is determined that confirmation of the driver's response to the third inquiry has been received. Conversely, upon receiving a voice command from the driver indicating rejection of a new navigation route (such as preset voice commands like "no," "disagree," or "cannot"), it is determined that denial of the driver's response to the third inquiry has been received.

[0092] In some other embodiments, in step S103, if the in-vehicle camera captures a gesture indicating agreement to the new navigation route performed by the driver, it is determined that confirmation information from the driver regarding the third inquiry has been received. This can be understood as follows: the in-vehicle camera captures gestures in the driver's seat area; the captured gestures are compared with a first target gesture; if they match, confirmation information from the driver regarding the third inquiry is determined; the captured phone gestures are compared with a second target gesture; if they match, denial information from the driver regarding the third inquiry is determined.

[0093] In some embodiments, if the target entertainment function is a second entertainment function for a target occupant to simulate driving the vehicle, after activating the second entertainment function, the method further includes: acquiring simulated driving information generated by the target occupant performing simulated driving behavior, and acquiring real driving information of the vehicle; comparing the simulated driving information with the real driving information, and outputting feedback information characterizing whether the simulated driving behavior of the target occupant is correct based on the comparison result.

[0094] It is understandable that a driving simulator is provided at the location of the target occupant for the target occupant to perform simulated driving behavior, and simulated driving information of the target occupant is obtained through the driving simulator.

[0095] In some embodiments, the driving simulator is an application installed on the vehicle's infotainment system. Various virtual buttons of the driving simulator are displayed on a second in-vehicle touchscreen located in the rear seat. These virtual buttons are used by the target occupant to perform simulated driving behaviors. The displayed virtual buttons include: up and down virtual buttons or up and down sliding arrows for the target occupant to perform simulated acceleration and deceleration operations, and a virtual steering wheel or left and right sliding arrows for the target occupant to simulate left and right turning operations.

[0096] In other embodiments, the driving simulator is a physical simulator mounted on the floor of the rear seats. The physical simulator may be mounted under the backrest of the front seats. The physical simulator includes a simulated accelerator and a simulated brake pedal mounted on the floor under the seat back, which are retracted within the floor. When the second entertainment function is activated, the simulated accelerator and simulated brake pedals are extended for the target occupant to perform simulated acceleration and simulated braking actions. A steering wheel is mounted on the backrest of the front seats. When the second entertainment function is turned off, it is retracted within the backrest of the front seats. After the first entertainment function is activated, it extends from the backrest of the front seats for the target occupant to perform simulated steering actions.

[0097] The vehicle-mounted system collects simulated driving information generated by the simulated driving behavior performed by the target occupant in the driving simulator, including acceleration, deceleration, braking and steering information performed by the target occupant. It also collects real driving information, including acceleration, deceleration, braking and steering information of the real driving behavior performed by the driver in the vehicle.

[0098] In some embodiments, comparing simulated driving information with real driving information and outputting feedback information based on the comparison result may include: comparing whether the simulated driving information and real driving information are consistent; if consistent, outputting first feedback information to the target occupant, the first feedback information indicating that the target occupant's simulated driving behavior is correct; if inconsistent, judging whether the target occupant has dangerous driving behavior based on the simulated driving information, and outputting second feedback information based on the judgment result, wherein the second feedback information indicates that the target occupant's simulated driving behavior is incorrect.

[0099] In some embodiments, in order to further accurately reflect the simulated driving behavior of the target occupant, the step of determining whether the target occupant has dangerous driving behavior based on the simulated driving information and outputting second feedback information based on the determination result may include: if the target occupant has dangerous driving behavior, outputting feedback information for educating the target occupant; if the target occupant does not have dangerous driving behavior, outputting feedback information indicating that the simulated driving information is inconsistent with the real driving information.

[0100] In some embodiments, the consistency between simulated driving information and real driving information is compared across four dimensions: acceleration, deceleration, left turn, and right turn. Simulated driving information and real driving information are acquired and compared at preset time intervals, such as comparing simulated driving information within 3 seconds or 2 seconds with real driving information. The comparison between simulated and real driving information is directional. When the direction of any dimension of the simulated and real driving information is consistent, the result is displayed as 1, indicating consistency; when the direction of any dimension of the simulated and real driving information is inconsistent, the result is displayed as 0, indicating consistency. The comparison results are shown in Table 1 below:

[0101] Table 1.

[0102]

[0103] In some embodiments, determining whether a target occupant has engaged in dangerous driving behavior based on simulated driving information includes: pre-setting multiple judgment conditions for dangerous driving behavior; if the simulated driving information determines that the target occupant's driving behavior meets any one of the dangerous driving conditions, then the target occupant is determined to have engaged in dangerous driving behavior; determining whether the driving speed of the simulated driving behavior exceeds the speed limit of the current road segment or the vehicle speed limit; if it does, the target occupant is determined to have engaged in dangerous driving behavior; determining whether the simulated driving behavior involves lane changing; if it interferes with vehicles in the target lane to which the lane changing behavior is intended, the target occupant is determined to have engaged in dangerous driving behavior; and determining whether the target occupant's acceleration or deceleration behavior interferes with vehicles in front or behind; if so, the target occupant is determined to have engaged in dangerous driving behavior.

[0104] For example, the first feedback information may be a "thumbs up" or other graphic content displayed on the second in-vehicle touchscreen, or a voice message such as "Little friend, you're great!" For example, feedback information indicating a discrepancy between simulated and real driving information may be a "crying face" or other graphic content displayed on the second in-vehicle touchscreen, or a voice message such as "Little friend, you still need to keep trying!" Feedback information used to educate the target occupant may be educational information displayed on the second in-vehicle touchscreen or delivered via voice, such as: "Little friend, you're speeding! There's a car on your right, we can't turn right!" or similar text or image messages.

[0105] If the current driving mode is automatic driving mode, the vehicle's actual driving information is obtained from the automatic driving system or the CAN (Controller Area Network) network. If the current driving mode is manual driving mode, the vehicle's actual driving information can be obtained from the CAN network.

[0106] To facilitate understanding of the embodiments of the present invention, the following references are made. Figure 4 Provide the human-computer interaction process for the second entertainment function:

[0107] Step S41: Receive the simulated driving actions performed by the target occupant in the driving simulator. The simulated driving actions are based on the target occupant's prediction of the actual driving actions of the driver or the automatic driving system in the vehicle and simulate driving the vehicle.

[0108] Step S42: Record the simulated driving information generated by the simulated driving actions of the target occupant;

[0109] Step S43: Record the real driving information generated by the actual driving actions of the driver or the autonomous driving system inside the vehicle;

[0110] Step S44: Compare whether the simulated driving information is consistent with the real driving information. If yes, proceed to step S45; if not, proceed to step S46.

[0111] S45: A prompt message indicating that the simulated driving information is consistent with the real driving information;

[0112] S46: Determine whether the target occupant has engaged in dangerous driving behavior based on the simulated driving information. If yes, proceed to step S47; otherwise, proceed to step S48.

[0113] S47: A prompt message indicating that the simulated driving information is inconsistent with the real driving information;

[0114] S48: Provide educational information to the target occupant regarding dangerous driving behavior.

[0115] In some embodiments, if the current driving mode is autonomous driving mode and the target entertainment function is a third entertainment function preset for the vehicle's autonomous driving mode, the third entertainment function is used to provide driving suggestions to the autonomous driving system by the target occupant in autonomous driving mode. After activating the third entertainment function, the method further includes: obtaining the driving suggestions provided by the target occupant; outputting a third inquiry message, which is used to ask the driver in the vehicle whether he agrees with the driving suggestions; and if the driver in the vehicle confirms the third inquiry message, processing the driving suggestions through the vehicle's autonomous driving system.

[0116] In some embodiments, processing driving suggestions through the vehicle's autonomous driving system may include: the autonomous driving system determining whether the driving suggestion meets driving safety conditions; if it does, controlling the autonomous driving system to execute the driving action suggested; if it does not meet driving safety conditions, outputting a prompt message indicating that the autonomous driving system rejects the driving suggestion. In some embodiments, if a denial message from the driver inside the vehicle is received regarding a third inquiry, a prompt message indicating rejection of the driving suggestion is output. This enables the autonomous driving system to listen to the driving suggestions of the target occupant while ensuring driving safety, thereby achieving a deeper driving experience for the target occupant.

[0117] In some embodiments, the autonomous driving system determines whether a driving suggestion meets driving safety conditions, including: if the driving suggestion is to accelerate, determining whether the distance to the vehicle ahead is less than a preset threshold; if so, it does not meet driving safety conditions; if the driving suggestion is to decelerate, determining whether the distance to the vehicle behind is less than a preset threshold; if so, it does not meet driving safety conditions; if the driving suggestion is to change lanes, determining whether there is interference between vehicles in the target lane to which the lane is to be changed; if so, it does not meet driving safety conditions.

[0118] In some embodiments, driving suggestions provided by a target occupant can be obtained through voice interaction. Obtaining driving suggestions from a target occupant may include: responding to a voice command from the target occupant to wake up the vehicle, causing the vehicle to enter a state of waiting for driving suggestions; and if driving suggestions are received from the target occupant within a preset time period, outputting a third query message. It is understood that driving suggestions include one or more of acceleration, deceleration, steering, and braking.

[0119] The operation of third-party entertainment functions allows the target occupants to further participate in the driving task. In Level 3 and above autonomous driving modes, the autonomous driving system drives the vehicle, and can drive according to the target occupants' driving suggestions while ensuring safety.

[0120] To facilitate understanding of the embodiments of the present invention, the following references are made. Figure 5 Provide the human-computer interaction process for the third entertainment function:

[0121] Step S51: Receive a voice command initiated by the target occupant to wake up the vehicle, so that the vehicle enters a state of waiting for driving advice;

[0122] Step S52: Receive driving suggestions initiated by the target occupant;

[0123] Step S53: Output the third query message to ask the driver in the vehicle whether they agree with the driving advice;

[0124] Step S54: Has confirmation information for the third inquiry been received from the driver inside the vehicle? If yes, proceed to step S55; otherwise, proceed to step S57.

[0125] Step S55: Determine whether the driving suggestion meets the driving safety conditions through the automatic driving system. If yes, proceed to step S56; otherwise, proceed to step S57.

[0126] Step S56: The autonomous driving system executes the driving action suggested by the driver;

[0127] Step S57: Output a message indicating that the driving suggestion is rejected.

[0128] Based on the same inventive concept, embodiments of the present invention also provide a vehicle. Figure 6 A schematic diagram of the structure of a vehicle according to some embodiments of the present invention is shown, such as... Figure 6 As shown, the vehicle includes one or more memories 604, one or more processors 602, and at least one computer program (computer execution instructions) stored in the memory 604 and executable on the processor 602. When the processor 602 executes the computer program, it implements the human-computer interaction-based vehicle control method described in any of the above embodiments.

[0129] Among them, Figure 6 In this document, a bus architecture (represented by bus 600) is used. Bus 600 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 602 and memory represented by memory 604. Bus 600 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 605 provides an interface between bus 600 and receiver 601 and transmitter 603. Receiver 601 and transmitter 603 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 602 is responsible for managing bus 600 and general processing, while memory 604 can be used to store data used by processor 602 during operation.

[0130] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, the computer program including executable instructions, which, when executed by a processor, implement the vehicle control method based on human-computer interaction described in any of the above embodiments.

[0131] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] This invention is described with reference to flowchart illustrations and / or block diagrams of a vehicle control method, device (system), and computer program product based on human-vehicle interaction according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the human-vehicle interaction method, device (system), and computer program product. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0133] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0135] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0136] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A vehicle control method based on human-vehicle interaction, characterized in that, include: Identify whether there are target occupants inside the vehicle; If a target occupant is identified in the vehicle, a first inquiry message is output, which is used to ask the driver in the vehicle whether he agrees to the target occupant participating in the driving task. If a confirmation message from the driver in the vehicle is received regarding the first inquiry, a candidate entertainment function is obtained from a preset set of M entertainment functions, where M is an integer greater than 1. This includes: obtaining the current driving mode of the vehicle, wherein entertainment functions for the target occupant to participate in driving tasks are preset for different driving modes of the vehicle; obtaining the candidate entertainment function from the entertainment functions preset for the current driving mode based on the current driving mode of the vehicle; outputting a second inquiry message regarding the candidate entertainment function, the second inquiry message being used to ask the driver in the vehicle or the target occupant whether they agree to activate the candidate entertainment function; if a confirmation message from the driver in the vehicle or the target occupant is received regarding the second inquiry message, the candidate entertainment function is selected as the target entertainment function. Activate the target entertainment function for the target occupant to participate in driving tasks, so that the target occupant can interact with the vehicle based on the target entertainment function to participate in driving tasks.

2. The vehicle control method based on human-vehicle interaction as described in claim 1, characterized in that, The target entertainment function is a first entertainment function, which is used to allow the target occupant to participate in planning the vehicle's driving route. After activating the target entertainment function that allows the target occupant to participate in the driving task, the function further includes: Obtain navigation information provided by the target occupant; Based on the navigation information, a new navigation route is generated and a third query message is output, which is used to ask the driver in the vehicle whether he agrees to the new navigation route. If the driver inside the vehicle confirms the third inquiry, the vehicle's route will be updated to the new navigation route.

3. The vehicle control method based on human-vehicle interaction as described in claim 1, characterized in that, The target entertainment function is a second entertainment function for allowing the target occupant to simulate driving the vehicle. After activating the target entertainment function for the target occupant to participate in the driving task, it further includes: Acquire simulated driving information generated by the target occupant performing simulated driving behavior, and acquire the vehicle's actual driving information; The simulated driving information is compared with the real driving information, and feedback information indicating whether the simulated driving behavior of the target occupant is correct is output based on the comparison result.

4. The vehicle control method based on human-vehicle interaction as described in claim 3, characterized in that, The comparison of the simulated driving information with the real driving information, and the output of feedback information based on the comparison results, includes: Compare whether the simulated driving information is consistent with the real driving information; If they match, the first feedback information is output to the target occupant, and the first feedback information indicates that the target occupant's simulated driving behavior is correct; If there is a discrepancy, the system determines whether the target occupant has engaged in dangerous driving behavior based on the simulated driving information, and outputs second feedback information based on the determination result. The second feedback information indicates that the target occupant's simulated driving behavior is incorrect.

5. The vehicle control method based on human-vehicle interaction as described in claim 1, characterized in that, The target entertainment function is a third entertainment function preset for the vehicle's autonomous driving mode. This third entertainment function is used to allow the target occupant to provide driving suggestions to the autonomous driving system. After activating the target entertainment function for the target occupant to participate in driving tasks, the system further includes: Obtain driving suggestions provided by the target occupant; Output a third query message, which is used to ask the driver in the vehicle whether they agree with the driving suggestion; If the vehicle receives confirmation from the driver regarding the third inquiry, the vehicle's autonomous driving system determines whether the driving suggestion meets driving safety requirements. If the driving suggestion meets the driving safety conditions, the autonomous driving system is controlled to perform driving actions according to the driving suggestion.

6. The vehicle control method based on human-vehicle interaction as described in claim 5, characterized in that, The target occupant is a child passenger in the rear seat of the vehicle; The method of identifying whether there is a target occupant inside the vehicle includes: The presence of a child passenger in the rear seat of the vehicle can be identified by at least one of the following dimensions: The system identifies whether there are child occupants in the rear seats based on images captured by the in-vehicle camera. The system identifies whether there is a child passenger in the rear seat based on pressure information collected by gravity sensors installed in the rear seat; and The presence of a child passenger in the rear seat was identified based on the occupant's vital signs.

7. A vehicle, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes the computer execution instructions stored in the memory to implement the vehicle control method based on human-vehicle interaction as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program includes executable instructions that, when executed by a processor, implement the vehicle control method based on human-vehicle interaction as described in any one of claims 1-6.

Images