Vehicle control method based on human-vehicle interaction, vehicle and storage medium

Abstract

The invention discloses a vehicle control method based on human-vehicle interaction, a vehicle and a storage medium, and belongs to the field of vehicle control. The vehicle control method comprises the steps that whether a target passenger exists in the vehicle or not is recognized; when it is identified that the target passenger exists in the vehicle, first inquiry information is output, and the first inquiry information is used for inquiring whether a driver in the vehicle agrees the target passenger to participate in the driving task or not; and if confirmation information of the driver in the vehicle for the first inquiry information is received, starting a target entertainment function for the target passenger to participate in the driving task, so that the target passenger performs man-machine interaction with the vehicle based on the target entertainment function to participate in the driving task. The technical problem that the effect of assisting a passenger to drive a vehicle is poor is solved.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle control, and in particular relates to a vehicle control method based on human-vehicle interaction, a vehicle and a storage medium. Background Art

[0002] Child passengers often increase the psychological burden on drivers. In order to optimize driving scenarios with child passengers, the design of in-vehicle solutions should focus on how to effectively meet the unique needs of child passengers, thereby assisting drivers to drive more safely.

[0003] The related technology provides children with educational and entertainment content through a built-in robot companion or voice interaction system. The driver helps children maintain their interest during the journey, which can assist the driver to drive safely to a certain extent. However, the educational and entertainment content provided to children and other passengers in the related technology lacks human-vehicle interaction, resulting in the inability of children and other passengers to deeply participate in the driving task of the vehicle, and the vehicle cannot obtain the assisted driving of children and other passengers, resulting in poor assisted driving effect of the vehicle. Summary of the invention

[0004] The embodiments of the present invention provide a vehicle control method based on human-vehicle interaction, a vehicle and a storage medium, which are used to solve the technical problem that the effect of occupant-assisted driving of the vehicle is poor.

[0005] In a first aspect, a vehicle control method based on human-vehicle interaction is provided, comprising: identifying whether there is a target occupant in the vehicle; when it is identified that there is a target occupant in the vehicle, outputting a first inquiry message, the first inquiry message being used to inquire whether the driver in the vehicle agrees that the target occupant can participate in a driving task; if confirmation information of the driver in the vehicle to the first inquiry information is received, starting a target entertainment function for the target occupant to participate in the driving task, so that the target occupant can perform human-computer interaction with the vehicle based on the target entertainment function to participate in the driving task.

[0006] In combination with the first aspect, in some embodiments, before the target entertainment function for the target occupant to participate in the driving task is started, it also includes: obtaining a candidate entertainment function from M preset entertainment functions, M is an integer greater than 1; outputting a second inquiry information for the candidate entertainment function, the second inquiry information is used to inquire whether the driver in the vehicle or the target occupant agrees to start the candidate entertainment function; if confirmation information from the driver in the vehicle or the target occupant for the second inquiry information is received, the candidate entertainment function is used as the target entertainment function.

[0007] In combination with the first aspect, in some embodiments, obtaining the candidate entertainment function from the preset M entertainment functions includes: obtaining the current driving mode of the vehicle, wherein entertainment functions for the target occupant to participate in driving tasks are preset for different driving modes of the vehicle; and according to the current driving mode of the vehicle, obtaining the candidate entertainment function from the entertainment functions preset for the current driving mode.

[0008] In combination with the first aspect, in some embodiments, the target entertainment function is a first entertainment function, which is used for the target occupant to participate in planning the vehicle's driving route. After the target entertainment function for the target occupant to participate in the driving task is started, it also includes: obtaining navigation information provided by the target occupant; generating a new navigation route based on the navigation information and outputting third inquiry information, wherein the third inquiry information is used to inquire whether the driver in the vehicle agrees to the new navigation route; if confirmation information of the driver in the vehicle to the third inquiry information is received, the vehicle's driving route is updated to the new navigation route.

[0009] In combination with the first aspect, in some embodiments, the target entertainment function is a second entertainment function for the target occupant to simulate driving the vehicle. After the target entertainment function for the target occupant to participate in a driving task is started, it also includes: obtaining simulated driving information generated by the target occupant performing the simulated driving behavior, and obtaining real driving information of the vehicle; comparing the simulated driving information with the real driving information, and outputting feedback information representing whether the simulated driving behavior of the target occupant is correct based on the comparison result.

[0010] In combination with the first aspect, in some embodiments, the comparing the simulated driving information with the real driving information and outputting feedback information based on the comparison result includes: comparing whether the simulated driving information is consistent with the real driving information; if they are consistent, outputting first feedback information to the target occupant, the first feedback information indicating that the simulated driving behavior of the target occupant is correct; if they are inconsistent, judging whether the target occupant has dangerous driving behavior based on the simulated driving information, and outputting second feedback information based on the judgment result, wherein the second feedback information indicates that the simulated driving behavior of the target occupant is incorrect.

[0011] In combination with the first aspect, in some embodiments, the target entertainment function is a third entertainment function preset for the automatic driving mode of the vehicle, and the third entertainment function is used for the target occupant to provide driving suggestions to the automatic driving system. After the target entertainment function for the target occupant to participate in the driving task is started, it also includes: obtaining the driving suggestions provided by the target occupant; outputting a third inquiry message, and the third inquiry message is used to inquire whether the driver in the vehicle agrees with the driving suggestion; if confirmation information of the driver in the vehicle to the third inquiry information is received, determining whether the driving suggestion meets the driving safety conditions through the automatic driving system of the vehicle; if the driving suggestion meets the driving safety conditions, controlling the automatic driving system to perform driving actions according to the driving suggestion.

[0012] In combination with the first aspect, in some embodiments, the target occupant is a child occupant in the rear seat of the vehicle; the identifying whether there is a target occupant in the vehicle includes: identifying whether there is a child occupant in the rear seat of the vehicle through at least one of the following dimensions: identifying whether there is a child occupant in the rear seat based on an image of the rear seat area captured by an in-vehicle camera; identifying whether there is a child occupant in the rear seat based on pressure information captured by a gravity sensor installed on the rear seat; and identifying whether there is a child occupant in the rear seat based on vital signs information of the occupant in the rear seat.

[0013] In a second aspect of the present invention, a vehicle is provided, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the vehicle control method based on human-vehicle interaction as described in any embodiment of the first aspect.

[0014] In a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. The computer program includes executable instructions. When the executable instructions are executed by a processor, the vehicle control method based on human-vehicle interaction as described in any embodiment of the first aspect is implemented.

[0015] According to the technical solutions provided by one or more embodiments of the present invention, at least the following technical effects or advantages are achieved:

[0016] By identifying whether there is a target occupant in the vehicle; and when it is identified that there is a target occupant in the vehicle, outputting a first inquiry message to inquire whether the driver in the vehicle agrees that the target occupant can participate in the driving task; if confirmation information of the driver in the vehicle to the first inquiry message is received, starting a target entertainment function for the target occupant to participate in the driving task, so that the target occupant can participate in the driving task through human-computer interaction with the vehicle based on the target entertainment function, thereby achieving the target occupant's deep participation in the driving task through human-computer interaction and assisting in driving the vehicle, thereby increasing the dimension of controlling the vehicle, and improving the interactivity between the occupant and the vehicle, thereby improving the occupant experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0018] Figure 1 A flow chart of a vehicle control method based on human-vehicle interaction according to some embodiments of the present invention is shown;

[0019] Figure 2 The control logic of a vehicle control method based on human-vehicle interaction according to some embodiments of the present invention is shown;

[0020] Figure 3 shows a human-computer interaction process of a first entertainment function according to some embodiments of the present invention;

[0021] Figure 4 shows a human-computer interaction process of a second entertainment function according to some embodiments of the present invention;

[0022] Figure 5 shows a human-computer interaction process of a third entertainment function according to some embodiments of the present invention;

[0023] Figure 6 A schematic structural diagram of a vehicle according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] First of all, the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0026] Figure 1 FIG. 1 is a flow chart of a vehicle control method based on human-vehicle interaction according to some embodiments of the present invention. Figure 1 As shown, the vehicle control method based on human-vehicle interaction includes the following steps S101 to S103:

[0027] In step S101: Identify whether there is a target occupant in the vehicle.

[0028] In some embodiments, the target occupant is any occupant other than the driver in the vehicle. In other embodiments, the target occupant is any passenger in the back seat, so as not to affect the driving behavior of the driver. In yet other embodiments, the target occupant is a child occupant in the back seat.

[0029] In some embodiments, the identification of whether there is a target occupant in the vehicle is triggered according to the vehicle's start signal, so that the target occupant can participate in the driving task in a timely manner after the vehicle is started.

[0030] In other embodiments, after the vehicle is started, it is identified whether the vehicle is traveling on a target section or target road condition. If it is identified that the vehicle is traveling on a target section or target road condition, it is identified whether there is a target occupant in the vehicle. The target section can be a section set by the driver in the vehicle, such as a section determined by a starting point and an end point set by the driver. The target section can also be a relatively safe section pushed to the vehicle by the service end based on the historical traffic data of the road. The target section can also be a relatively safe section pushed to the vehicle by the service end based on the road condition ahead of the vehicle. The target road condition is a relatively safe road condition, such as a non-congested road condition.

[0031] In some other embodiments, after the vehicle is started, it is identified whether the vehicle is traveling in a target time period; if it is identified that the vehicle is traveling in the target time period, it is identified whether there is a target passenger in the vehicle. It is understandable that the target time period can be set by the driver or set before leaving the factory.

[0032] In some embodiments, if the target occupant is a child occupant in a rear seat, the step of identifying whether there is a target occupant in the vehicle may include: identifying whether there is a child occupant in the rear seat of the vehicle by at least one of the following dimensions:

[0033] Dimension 1: Identify whether there are child passengers in the rear seats based on the images of the rear seat area captured by the in-car camera.

[0034] It can be understood that images of the rear seat area are captured by the in-car camera, and facial features are extracted from the images of the rear seat area. If facial features are extracted, it is determined that there are occupants in the rear seats, and whether the occupants in the rear seats are child occupants is identified based on the extracted facial features.

[0035] Dimension 2: Identify whether there are child passengers in the rear seats based on the pressure information collected by the gravity sensors installed in the rear seats.

[0036] It can be understood that the gravity sensor is a pressure sensor or a weight sensor, which is installed under the seat or on the backrest of the rear seat. The gravity sensor senses the pressure or weight changes of the seat or backrest of the rear seat to determine whether there are child passengers in the rear seat.

[0037] Dimension 3: Identify whether there are child passengers in the rear seats based on the vital signs of the rear seat passengers.

[0038] It is understandable that the occupant's vital signs information may include one or more of breathing rate, heart rate, infrared radiation, etc., and the breathing rate, heart rate, etc. of the occupants in the back seats can be detected by millimeter wave radar or ultrasonic radar to determine whether there are child occupants in the back seats. In a dark environment, an infrared night vision camera is used to capture the infrared radiation of the occupants in the back seats to identify whether there are child occupants in the back seats at night or in low light conditions.

[0039] In order to improve accuracy, the above three dimensions are combined to identify whether there are child passengers in the rear seats. When the above three dimensions all identify that there are child passengers in the rear seats, it is determined that there are child passengers in the rear seats.

[0040] In step S102 : when it is identified that there is a target occupant in the vehicle, a first inquiry message is output, where the first inquiry message is used to inquire whether the driver in the vehicle agrees that the target occupant participates in the driving task.

[0041] In step S102, the first inquiry information may be outputted in any one of the following output modes, or may be outputted in both of the following output modes:

[0042] Output method 1: Outputting a voice representing the first inquiry information through the vehicle's voice interaction system to inquire whether the driver in the vehicle agrees that the target occupant participates in the driving task.

[0043] Output method 2: outputting graphic content representing the first inquiry information through the first vehicle-mounted touch screen, asking the driver whether he agrees that the target passenger participates in the driving task, and the first vehicle-mounted touch screen is a touch screen located in front of the main driving seat.

[0044] In step S103: if confirmation information of the in-vehicle driver to the first inquiry information is received, a target entertainment function for the target occupant to participate in the driving task is started, so that the target occupant can participate in the driving task by performing human-computer interaction with the vehicle based on the target entertainment function.

[0045] In some embodiments, after the first inquiry information is output, if a denial message of the first inquiry information is received from the driver in the vehicle, the process ends without starting any entertainment function for the target occupant to participate in the driving task.

[0046] It should be noted that the confirmation information of the first inquiry information may be input by the driver in the vehicle through at least one of the input methods such as the first vehicle-mounted touch screen, voice input, gesture input, etc.

[0047] In some embodiments, if the voice representing the first inquiry information is output through the voice interaction system of the vehicle, then when a voice instruction indicating the consent of the target occupant to participate in the driving task (such as preset voice instructions such as "yes", "agree", "can") initiated by the driver in the vehicle is received, it is determined that the confirmation information of the first inquiry information of the driver in the vehicle is received. Then, when a voice instruction indicating the rejection of the target occupant to participate in the driving task (such as preset voice instructions such as "no", "disagree", "cannot") initiated by the driver in the vehicle is received, it is determined that the denial information of the first inquiry information of the driver in the vehicle is received.

[0048] In other embodiments, the graphic content representing the first inquiry information is outputted through the first vehicle touch screen, and when the first vehicle touch screen receives a confirmation operation indicating that the target occupant agrees to participate in the driving task, it is determined that the confirmation information of the driver in the vehicle for the first inquiry information has been received. The options for the driver in the vehicle to perform confirmation and cancellation operations can be presented in the form of a pop-up window on the first vehicle touch screen. If the confirmation operation performed on the first vehicle touch screen is received, it is determined that the confirmation information of the driver in the vehicle for the first inquiry information has been received. If the cancellation operation performed on the first vehicle touch screen is received, it is determined that the driver in the vehicle has received the denial information of the first inquiry information.

[0049] In some other embodiments, if a voice representing the first inquiry information is output through the vehicle's voice interaction system, and when the in-vehicle camera captures a gesture action performed by the in-vehicle driver indicating consent for the target occupant to participate in the driving task, it is determined that confirmation information for the first inquiry information from the in-vehicle driver is received.

[0050] It can be understood that the gesture movements in the main driving area are collected by the camera inside the car, and the collected gesture movements are compared with the first target gesture; if they are consistent, it is determined that the confirmation information of the driver in the car to the first inquiry information is received; the collected gesture movements are compared with the second target gesture, and if they are consistent, it is determined that the denial information of the driver in the car to the first inquiry information is received.

[0051] It is understandable that the preset entertainment functions may include one or more, wherein the preset entertainment functions are entertainment functions for the target occupant to participate in the driving task. In some embodiments, the preset entertainment functions include one or more of the following:

[0052] The first entertainment function: used for the target passengers to participate in planning the vehicle's driving route;

[0053] Second entertainment function: used for target passengers to simulate driving a vehicle;

[0054] The third entertainment function is used for the target occupants to provide driving suggestions to the automatic driving system.

[0055] In some embodiments, there is only one entertainment function for the target occupant to participate in the driving task, and this entertainment function is the target entertainment function. If confirmation information of the driver in the vehicle to the first inquiry information is received, the target entertainment function for the target occupant to participate in the driving task is directly started.

[0056] In other embodiments, there are multiple preset entertainment functions for the target occupant to participate in the driving task. Before starting the target entertainment function for the target occupant to participate in the driving task, it can also include: obtaining a candidate entertainment function from the preset M entertainment functions, M is an integer greater than 1; outputting a second inquiry information for the candidate entertainment function, the second inquiry information is used to inquire whether the driver in the vehicle or the target occupant agrees to start the candidate entertainment function; if confirmation information of the driver in the vehicle to the second inquiry information is received, the candidate entertainment function is used as the target entertainment function.

[0057] In some embodiments, after outputting the second inquiry information for the candidate entertainment function, if a denial of the second inquiry information is received from the driver in the vehicle, the next candidate entertainment function is obtained from the preset M entertainment functions and the second inquiry information for the next candidate entertainment function is output, and this cycle is repeated until a confirmation of the second inquiry information is received from the driver in the vehicle or there is no entertainment function to choose from.

[0058] It is understandable that the output method of the second inquiry information can be the same or similar to the output method of the first inquiry information, and the vehicle's voice interaction system can also output a voice representing the second inquiry information to inquire whether the driver agrees to activate the candidate entertainment function. The vehicle's onboard screen can also output a graphic representing the second inquiry information to inquire whether the driver agrees to activate the candidate entertainment function.

[0059] It is understandable that the confirmation information and the denial information of the second inquiry information can be at least one of the input methods of the driver in the vehicle through the vehicle touch screen input, voice input, gesture input, etc. For details, please refer to the description of the confirmation information and the denial information of the first inquiry information in the previous text, and for the sake of brevity of the specification, it will not be repeated here.

[0060] In some embodiments, the driving mode of the vehicle may not be distinguished, and one of the preset M entertainment functions may be randomly selected as a candidate entertainment function.

[0061] In other embodiments, the vehicle has a variety of different driving modes, for example, including an automatic driving mode and a manual driving mode, wherein the manual driving mode includes L0, L1, and L2 driving modes, and the automatic driving mode includes L3 and above driving modes. In order to make the activated entertainment function more consistent with the driving mode of the vehicle, entertainment functions for the target occupants to participate in driving tasks are preset for different driving modes of the vehicle. Obtaining candidate entertainment functions from the preset M entertainment functions may include: obtaining the current driving mode of the vehicle, and according to the current driving mode of the vehicle, obtaining candidate entertainment functions from the entertainment functions preset for the current driving mode, so that entertainment functions suitable for the driving mode can be pushed more accurately.

[0062] In some embodiments, one or more of the first entertainment function, the second entertainment function, and the third entertainment function may be preset for the automatic driving mode. In some embodiments, one or more of the first entertainment function and the second entertainment function may be preset for the manual driving mode.

[0063] In some embodiments, each driving mode of the vehicle corresponds to only one entertainment function, that is, the driving mode corresponds to the entertainment function one by one. The current driving mode of the vehicle is obtained, and the target entertainment function is determined according to the current driving mode of the vehicle.

[0064] like Figure 2 As shown, taking the target passenger as a child passenger in the rear seat as an example, a logic of the vehicle control method based on human-computer interaction in an embodiment of the present invention is given below:

[0065] Step S1: vehicle starts;

[0066] Step S2: Identify whether there is a child passenger in the rear seat, if so, proceed to step S3;

[0067] Step S3: outputting first inquiry information for inquiring the driver in the vehicle whether he agrees that the child passenger in the rear seat can participate in the driving task;

[0068] Step S4: whether confirmation information of the first inquiry information is received from the driver in the vehicle, if so, proceed to step S5;

[0069] Step S5: Detect the current driving mode of the vehicle. If the current driving mode is the manual driving mode, proceed to step S6; if the current driving mode is the automatic driving mode, proceed to step S7;

[0070] S6: Activate the target entertainment function corresponding to the manual driving mode for the child passengers in the rear seats to participate in the driving task, and proceed to step S7;

[0071] S7: Activate the target entertainment function corresponding to the automatic driving mode for the child passengers in the rear seats to participate in the driving task, and proceed to step S7;

[0072] S8: whether the instruction information issued by the driver in the vehicle to exit the target entertainment function is received, if so, proceed to step S9, if not, return to step S7;

[0073] S9: Turn off the target entertainment function.

[0074] In some embodiments, if the target entertainment function is a first entertainment function for the target occupant to participate in planning the driving route of the vehicle, after starting the first entertainment function, it also includes: obtaining navigation information provided by the target occupant; generating a new navigation route based on the navigation information and outputting a third inquiry information, the third inquiry information is used to inquire whether the driver in the vehicle agrees with the new navigation route; if the driver in the vehicle confirms the third inquiry information, the driving route of the vehicle is updated to the new navigation route. This facilitates the rear-seat target occupant to intervene in the planning and decision-making of the vehicle's driving route.

[0075] In some embodiments, if a denial message of the third inquiry message is received from the driver in the vehicle, the navigation update is exited and a prompt message indicating that the navigation route cannot be changed is fed back.

[0076] In some embodiments, the step of feeding back a prompt message that the navigation route cannot be changed may include: displaying a prompt message that the navigation route cannot be changed through a second vehicle-mounted touch screen at the target occupant's location and / or prompting the target occupant that the navigation route cannot be changed through a voice broadcast.

[0077] In other implementations, if no confirmation information is received from the driver in the vehicle in response to the third inquiry information within a preset waiting time, the navigation update is exited and a prompt message indicating that the navigation route cannot be changed is displayed through the second vehicle-mounted touch screen at the target occupant's location and / or a voice announcement is used to prompt the target occupant that the navigation route cannot be changed.

[0078] To facilitate understanding of the embodiments of the present invention, Figure 3 , the human-computer interaction process of the first entertainment function is given:

[0079] S31: receiving navigation information input by the target passenger to the vehicle computer;

[0080] S32: generating a new navigation route based on the navigation information;

[0081] S33: outputting a third inquiry message for inquiring the driver in the vehicle whether he agrees with the new navigation route;

[0082] S34: whether the confirmation information of the driver in the vehicle is received for the third inquiry information, if yes, proceed to step S35, if no, proceed to step S36;

[0083] S35: Update the vehicle's driving route to a new navigation route.

[0084] S36: Exit navigation update and feedback a prompt message that the navigation route cannot be changed.

[0085] In some embodiments, the input navigation information includes the settings of the changed navigation destination, en route locations, etc.

[0086] In some embodiments, obtaining navigation information provided by the target occupant includes: receiving navigation information input by the target occupant through a second vehicle-mounted touch screen, and synchronizing the navigation information input through the second vehicle-mounted touch screen to the first vehicle-mounted touch screen through linkage between the second vehicle-mounted touch screen and the first vehicle-mounted touch screen, so as to facilitate the driver in the vehicle to see the navigation information input by the target occupant.

[0087] In other embodiments, obtaining navigation information provided by the target occupant may include: receiving navigation information voice-input by the target occupant through a voice interaction system (such as a voice assistant) on a vehicle computer, i.e., an IVI (In-Vehicle Infotainment) system.

[0088] In some embodiments, generating a new navigation route based on the navigation information and outputting the third inquiry information may include: displaying the new navigation route and graphic content representing the third inquiry information through the first vehicle-mounted touch screen to ask the driver whether he agrees to the new navigation route.

[0089] It can be understood that the confirmation information and denial information of the third inquiry information can be input by the driver in the car through at least one of the input methods such as vehicle-mounted touch screen input, voice input, gesture input, etc. For details, please refer to the description of the confirmation information and denial information of the first inquiry information in the previous text. For the sake of brevity of the specification, it will not be repeated here.

[0090] In some embodiments, options for confirmation and cancellation operations can be presented in the form of a pop-up window on the first vehicle-mounted touch screen, a confirmation operation is received on the first vehicle-mounted touch screen, and it is determined that confirmation information from the driver in the vehicle regarding the third inquiry information is received; a cancellation operation is received on the first vehicle-mounted touch screen, and it is determined that denial information from the driver in the vehicle regarding the third inquiry information is received.

[0091] In other embodiments, a voice command (such as preset voice commands such as "yes", "agree", "okay", etc.) initiated by the driver in the vehicle indicating approval of the new navigation route is received, and it is determined that confirmation information of the driver in the vehicle for the third inquiry information is received. A voice command (such as preset voice commands such as "no", "disagree", "cannot", etc.) initiated by the driver in the vehicle indicating rejection of the new navigation route is received, and it is determined that denial information of the driver in the vehicle for the third inquiry information is received.

[0092] In some other embodiments, in step S103, when the in-car camera captures the gesture action performed by the in-car driver to indicate the consent to the new navigation route, it is determined that the confirmation information of the in-car driver to the third inquiry information is received. It can be understood that: the gesture action in the main driving area is captured by the in-car camera; the captured gesture action is compared with the first target gesture, and if they are consistent, it is determined that the confirmation information of the in-car driver to the third inquiry information is received; the captured mobile phone action is compared with the second target gesture, and if they are consistent, it is determined that the denial information of the third inquiry information is received by the in-car driver.

[0093] In some embodiments, if the target entertainment function is a second entertainment function for the target occupant to simulate driving the vehicle, after starting the second entertainment function, it also includes: obtaining simulated driving information generated by the target occupant performing the simulated driving behavior, and obtaining real driving information of the vehicle; comparing the simulated driving information with the real driving information, and outputting feedback information representing whether the simulated driving behavior of the target occupant is correct based on the comparison result.

[0094] It is understandable that a driving simulator for the target occupant to perform simulated driving behavior is provided at the location of the target occupant, and simulated driving information of the target occupant performing the simulated driving behavior is obtained through the driving simulator.

[0095] In some embodiments, the driving simulator is an application installed on the vehicle computer. Various virtual buttons of the driving simulator are displayed through a second vehicle-mounted touch screen arranged on the rear seat. The various virtual buttons are for the target occupants to perform simulated driving behaviors. The displayed virtual buttons include: up and down virtual buttons or up and down sliding arrows for the target occupants to perform simulated acceleration and deceleration operations, and a virtual steering wheel or left and right sliding arrows for the target occupants to simulate left and right steering operations.

[0096] In other embodiments, the driving simulator is a physical simulator disposed on the floor above the rear seats, and the physical simulator may be disposed under the backrest of the front seat. The physical simulator includes a simulated accelerator and a simulated brake pedal installed on the floor below the seat backrest, which are retracted into the floor. When the second entertainment function is activated, the simulated accelerator and the simulated brake pedal are extended for the target occupant to perform simulated acceleration and simulated braking behaviors. A steering wheel is installed on the backrest of the front seat, which is retracted into the backrest of the front seat when the second entertainment function is turned off. After the first entertainment function is activated, the steering wheel is extended from the backrest of the front seat for the target occupant to perform simulated steering behaviors.

[0097] The simulated driving information generated by the simulated driving behavior performed by the target occupant in the driving simulator is collected through the vehicle computer, including: acceleration information, deceleration information, braking information and steering information performed by the target occupant. The real driving information collected through the vehicle computer includes: acceleration information, deceleration information, braking information and steering information of the real driving behavior performed by the driver in the car.

[0098] In some embodiments, comparing the simulated driving information with the real driving information and outputting feedback information based on the comparison result may include: comparing whether the simulated driving information is consistent with the real driving information; if they are consistent, outputting first feedback information to the target occupant, the first feedback information indicating that the simulated driving behavior of the target occupant is correct; if they are inconsistent, judging whether the target occupant has dangerous driving behavior based on the simulated driving information, and outputting second feedback information based on the judgment result, wherein the second feedback information indicates that the simulated driving behavior of the target occupant is wrong.

[0099] In some embodiments, in order to further accurately provide feedback on the simulated driving behavior of the target occupant, the step of determining whether the target occupant has dangerous driving behavior based on the simulated driving information, and outputting second feedback information based on the determination result may include: if the target occupant has dangerous driving behavior, outputting feedback information for educating the target occupant; if the target occupant does not have dangerous driving behavior, outputting feedback information indicating that the simulated driving information is inconsistent with the actual driving information.

[0100] In some embodiments, the simulated driving information is compared with the real driving information to see if they are consistent, and the comparison is made from four dimensions: acceleration, deceleration, left turn, and right turn. The simulated driving information and the real driving information are acquired and compared at preset time intervals, such as comparing the simulated driving information within 3 seconds or within 2 seconds with the real driving information. The comparison between the simulated driving information and the real driving information is directional. When the direction of the simulated driving information and the real driving information is consistent in any dimension of data, the displayed result is 1, indicating that the simulated driving information is consistent with the real driving information; when the direction of the simulated driving information and the real driving information is inconsistent in any dimension of data, the displayed result is 0, indicating that the simulated driving information is consistent with the real driving information. The comparison results are shown in Table 1 below:

[0101] Table 1.

[0102]

[0103] In some embodiments, judging whether the target occupant has dangerous driving behavior based on simulated driving information includes: pre-setting a plurality of judgment conditions for dangerous driving behavior, and if it is determined that the driving behavior of the target occupant satisfies any of the dangerous driving conditions based on the simulated driving information, judging that the target occupant has dangerous driving behavior: judging whether the driving speed of the simulated driving behavior exceeds the speed limit of the current road section or the vehicle speed limit, and if so, judging that the target occupant has dangerous driving behavior; judging whether the simulated driving behavior involves lane changing, and if so, judging that the target occupant has dangerous driving behavior; judging whether the acceleration and deceleration behavior of the target occupant interferes with the vehicles in the target lane to which the lane change behavior is to be changed, and if so, judging that the target occupant has dangerous driving behavior.

[0104] Exemplarily, the first feedback information is a graphic content such as "Like" displayed on the second vehicle-mounted touch screen or a voice content such as "Little friend, you are great!". Exemplarily, the feedback information representing the inconsistency between the simulated driving information and the real driving information can be a graphic content such as "crying face" displayed on the second vehicle-mounted touch screen or a voice content such as "Little friend, you still need to work hard!". The feedback information used to educate the target occupants can be educational information for the target occupants displayed on the second vehicle-mounted touch screen or voice broadcast, such as: Little friend, you are speeding, there is a car on the right and we cannot turn right, etc.

[0105] If the current driving mode is the automatic driving mode, the real driving information of the vehicle is obtained from the automatic driving system or the CAN (Controller Area Network) network. If the current driving mode is the manual driving mode, the real driving information of the vehicle can be obtained from the CAN network.

[0106] To facilitate understanding of the embodiments of the present invention, Figure 4 The human-computer interaction process of the second entertainment function is given:

[0107] Step S41: receiving the simulated driving action performed by the target occupant in the driving simulator, wherein the simulated driving action is performed by the target occupant predicting the real driving action of the driver in the vehicle or the automatic driving system and simulating the driving of the vehicle.

[0108] Step S42: recording simulated driving information generated by the simulated driving action of the target occupant;

[0109] Step S43: recording real driving information generated by the real driving action of the driver or the automatic driving system in the vehicle;

[0110] Step S44: Compare the simulated driving information with the real driving information to see if they are consistent. If so, proceed to step S45; if not, proceed to step S46;

[0111] S45: Feedback prompt information that the simulated driving information is consistent with the actual driving information;

[0112] S46: judging whether the target occupant has dangerous driving behavior according to the simulated driving information, if so, proceeding to step S47, otherwise, proceeding to step S48;

[0113] S47: Feedback prompt information indicating that the simulated driving information is inconsistent with the actual driving information;

[0114] S48: Provides educational information on dangerous driving behavior of the target occupant.

[0115] In some embodiments, if the current driving mode is the automatic driving mode, and the target entertainment function is a third entertainment function preset for the vehicle's automatic driving mode, the third entertainment function is used for the target occupant to provide driving suggestions to the automatic driving system in the automatic driving mode. After starting the third entertainment function, it also includes: obtaining the driving suggestions provided by the target occupant; outputting a third inquiry message, the third inquiry message is used to ask the driver in the vehicle whether he agrees with the driving suggestions; if confirmation information from the driver in the vehicle for the third inquiry message is received, processing the driving suggestions through the vehicle's automatic driving system.

[0116] In some embodiments, processing driving suggestions through the vehicle's automatic driving system may include: the automatic driving system determines whether the driving suggestions meet the driving safety conditions, if the driving safety conditions are met, controlling the automatic driving system to execute the driving action of the driving suggestions, if the driving safety conditions are not met, outputting prompt information indicating that the automatic driving system rejects the driving suggestions. In some embodiments, if the in-vehicle driver denies the third inquiry information, a prompt information indicating the rejection of the driving suggestions is output. The automatic driving system can listen to the driving suggestions of the target occupant on the premise of ensuring driving safety, so as to achieve a deep experience of the target occupant driving the vehicle.

[0117] In some embodiments, the automatic driving system determines whether the driving suggestion meets the driving safety condition, including: if the driving suggestion is to suggest acceleration, determine whether the distance between the vehicle in front is less than a preset threshold, if so, it does not meet the driving safety condition; if the driving suggestion is to suggest deceleration, determine whether the distance between the vehicle behind is less than a preset threshold, if so, it does not meet the driving safety condition. If the driving suggestion is to change lanes, determine whether there is interference between vehicles in the target lane to be changed to, if so, it does not meet the driving safety condition.

[0118] In some embodiments, the driving advice provided by the target occupant can be obtained through voice interaction, and obtaining the driving advice provided by the target occupant can include: responding to the target occupant's voice command to wake up the vehicle so that the vehicle enters a state of waiting for driving advice, and if the driving advice provided by the target occupant is received within a preset time, outputting third inquiry information. It can be understood that the driving advice includes one or more of acceleration, deceleration, steering, and braking.

[0119] The operation of the third entertainment function can enable the target occupant to further participate in the driving task. In L3 and above autonomous driving modes, the vehicle is driven by the autonomous driving system, which can drive according to the driving suggestions of the target occupant while ensuring safety.

[0120] To facilitate understanding of the embodiments of the present invention, Figure 5 The human-computer interaction process of the third entertainment function is given:

[0121] Step S51: receiving a voice command initiated by a target occupant for waking up the vehicle, so that the vehicle enters a state of waiting for driving advice;

[0122] Step S52: receiving driving suggestions initiated by the target occupant;

[0123] Step S53: outputting a third inquiry message for inquiring the driver in the vehicle whether he agrees with the driving suggestion;

[0124] Step S54: whether the confirmation information of the driver in the vehicle is received for the third inquiry information, if yes, proceed to step S55, if no, proceed to step S57;

[0125] Step S55: The automatic driving system determines whether the driving suggestion meets the driving safety conditions. If so, the process proceeds to step S56; if not, the process proceeds to step S57;

[0126] Step S56: the automatic driving system executes the driving action recommended by the driver;

[0127] Step S57: Outputting prompt information for rejecting the driving suggestion.

[0128] Based on the same inventive concept, an embodiment of the present invention further provides a vehicle. Figure 6 A schematic diagram of the structure of a vehicle according to some embodiments of the present invention is shown. Figure 6 As shown, the vehicle includes one or more memories 604, one or more processors 602, and at least one computer program (computer execution instruction) stored in the memory 604 and executable on the processor 602. When the processor 602 executes the computer program, the vehicle control method based on human-computer interaction described in any one of the above embodiments is implemented.

[0129] Among them, Figure 6 In the embodiment of the present invention, a bus architecture (represented by bus 600) is shown, which may include any number of interconnected buses and bridges, and bus 600 links various circuits including one or more processors represented by processor 602 and memory represented by memory 604. Bus 600 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 605 provides an interface between bus 600 and receiver 601 and transmitter 603. Receiver 601 and transmitter 603 may be the same element, namely a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 602 is responsible for managing bus 600 and general processing, while memory 604 may be used to store data used by processor 602 when performing operations.

[0130] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored. The computer program includes executable instructions. When the executable instructions are executed by a processor, the vehicle control method based on human-computer interaction described in any of the above embodiments is implemented.

[0131] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to the flowchart and / or block diagram of the vehicle control method, device (system), and computer program product based on human-vehicle interaction according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0136] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A vehicle control method based on human-vehicle interaction, characterized in that: include: Identify whether there are target occupants in the vehicle; When it is identified that there is a target occupant in the vehicle, outputting first inquiry information, wherein the first inquiry information is used to inquire whether the driver in the vehicle agrees that the target occupant participates in the driving task; If confirmation information of the driver in the vehicle to the first inquiry information is received, a target entertainment function for the target occupant to participate in the driving task is started, so that the target occupant can participate in the driving task by performing human-computer interaction with the vehicle based on the target entertainment function.

2. The vehicle control method based on human-vehicle interaction according to claim 1, characterized in that: Before starting the target entertainment function for the target occupant to participate in the driving task, the method further includes: Obtaining a candidate entertainment function from M preset entertainment functions, where M is an integer greater than 1; Outputting second inquiry information for the candidate entertainment function, wherein the second inquiry information is used to inquire whether the driver or the target occupant in the vehicle agrees to activate the candidate entertainment function; If confirmation information of the in-vehicle driver or the target occupant in response to the second inquiry information is received, the candidate entertainment function is used as a target entertainment function.

3. The vehicle control method based on human-vehicle interaction as claimed in claim 2, characterized in that: The step of obtaining a candidate entertainment function from the preset M entertainment functions includes: Acquiring a current driving mode of the vehicle, wherein entertainment functions for allowing the target occupant to participate in a driving task are preset for different driving modes of the vehicle; According to a current driving mode of the vehicle, the candidate entertainment function is obtained from entertainment functions preset for the current driving mode.

4. The vehicle control method based on human-vehicle interaction according to any one of claims 1 to 3, characterized in that: The target entertainment function is a first entertainment function, and the first entertainment function is used for the target occupant to participate in planning the driving route of the vehicle. After starting the target entertainment function for the target occupant to participate in the driving task, the method further includes: Obtaining navigation information provided by the target occupant; generating a new navigation route based on the navigation information and outputting third inquiry information, wherein the third inquiry information is used to inquire whether the driver in the vehicle agrees with the new navigation route; If confirmation information of the driver in the vehicle to the third inquiry information is received, the driving route of the vehicle is updated to the new navigation route.

5. The vehicle control method based on human-vehicle interaction according to any one of claims 1 to 3, characterized in that: The target entertainment function is a second entertainment function for the target occupant to simulate driving the vehicle. After the target entertainment function for the target occupant to participate in the driving task is started, the method further includes: Acquiring simulated driving information generated by the target occupant performing simulated driving behavior, and acquiring real driving information of the vehicle; The simulated driving information is compared with the real driving information, and feedback information representing whether the simulated driving behavior of the target occupant is correct is output according to the comparison result.

6. The vehicle control method based on human-vehicle interaction as claimed in claim 5, characterized in that: The comparing the simulated driving information with the real driving information and outputting feedback information according to the comparison result includes: comparing the simulated driving information with the actual driving information to see whether they are consistent; If they are consistent, outputting first feedback information to the target occupant, wherein the first feedback information indicates that the simulated driving behavior of the target occupant is correct; If they are inconsistent, it is determined whether the target occupant has dangerous driving behavior based on the simulated driving information, and second feedback information is output based on the determination result, wherein the second feedback information indicates that the simulated driving behavior of the target occupant is wrong.

7. The vehicle control method based on human-vehicle interaction according to any one of claims 1 to 3, characterized in that: The target entertainment function is a third entertainment function preset for the automatic driving mode of the vehicle, and the third entertainment function is used for the target occupant to provide driving suggestions to the automatic driving system. After starting the target entertainment function for the target occupant to participate in the driving task, the method further includes: Obtaining driving advice provided by the target occupant; Outputting third inquiry information, wherein the third inquiry information is used to inquire whether the driver in the vehicle agrees with the driving suggestion; If confirmation information of the in-vehicle driver to the third inquiry information is received, determining whether the driving suggestion meets the driving safety conditions through the automatic driving system of the vehicle; If the driving suggestion meets the driving safety condition, control the automatic driving system to perform driving actions according to the driving suggestion.

8. The vehicle control method based on human-vehicle interaction as claimed in claim 7, characterized in that: The target occupant is a child occupant in the rear seat of the vehicle; The identifying whether there is a target occupant in the vehicle comprises: Identify whether there is a child passenger in the rear seat of the vehicle by at least one of the following dimensions: Identifying whether there is a child occupant in the rear seat based on an image of the rear seat area captured by an in-vehicle camera; identifying whether there is a child occupant in the rear seat according to pressure information collected by a gravity sensor installed in the rear seat; and It is identified that there is a child occupant in the rear seat according to the occupant's vital sign information of the rear seat.

9. A vehicle, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the vehicle control method based on human-vehicle interaction as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program includes executable instructions, and when the executable instructions are executed by a processor, the vehicle control method based on human-vehicle interaction described in any one of claims 1 to 8 is implemented.

Images