Vehicle control method and device

Abstract

The invention discloses a vehicle control method, device and equipment and a computer readable storage medium, and belongs to the technical field of vehicles. The method comprises the following steps: in response to a starting instruction for a voice assistant function configured in a vehicle, determining the running state of the vehicle and the remaining power percentage of the vehicle; under the condition that the running state of the vehicle meets the running requirement and the percentage of the remaining electric quantity of the vehicle is larger than a target threshold value, image acquisition equipment installed in the vehicle is controlled to be started; a first video collected by an image collection device in a first time period is obtained, the first video comprises a first gesture action of a driver and passengers of the vehicle, and the starting time of the first time period is the time for starting the image collection device; determining a first control instruction corresponding to the first gesture action; and controlling a control system corresponding to the first control instruction to execute a control operation corresponding to the first control instruction. According to the method, the use frequency of the voice assistant function is improved, and the vehicle control efficiency is higher.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of vehicles, and particularly to a vehicle control method and device. Background Art

[0002] With the continuous development of vehicle technology, more and more vehicles are equipped with a voice assistant function, which is used to obtain voice commands from the driver and passengers and control the vehicle based on the voice commands of the driver and passengers.

[0003] However, when the driver and passengers are deaf-mute, since they cannot hear or speak, the usage frequency of the voice assistant function is relatively low. When the driver and passengers want to control the vehicle, they need to manually control it, resulting in low vehicle control efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle control method and device, which can be used to solve the problems in the related art. The technical solutions are as follows:

[0005] On the one hand, the embodiments of the present application provide a vehicle control method, which includes:

[0006] In response to an activation instruction for the voice assistant function equipped in the vehicle, determine the running state of the vehicle and the percentage of the remaining power of the vehicle;

[0007] When the running state of the vehicle meets the running requirements and the percentage of the remaining power of the vehicle is greater than the target threshold, control the image acquisition device installed in the vehicle to be activated;

[0008] Obtain a first video collected by the image acquisition device in a first time period. The first video includes first gesture actions of the driver and passengers of the vehicle. The start time of the first time period is the time when the image acquisition device is activated;

[0009] Determine a first control instruction corresponding to the first gesture action;

[0010] Control the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction.

[0011] In a possible implementation manner, the determining the first control instruction corresponding to the first gesture action includes:

[0012] Determine the similarity between the gesture actions corresponding to each control instruction and the first gesture action;

[0013] Use the control instruction corresponding to the gesture action whose similarity to the first gesture action meets the similarity requirement as the first control instruction.

[0014] In a possible implementation, determining the similarity between the gesture actions corresponding to each control instruction and the first gesture action includes:

[0015] For the gesture action corresponding to any one of the control instructions, determine a first feature vector for characterizing the gesture action corresponding to the any one of the control instructions;

[0016] Determine a second feature vector for characterizing the first gesture action;

[0017] According to the first feature vector and the second feature vector, determine the similarity between the gesture action corresponding to the any one of the control instructions and the first gesture action.

[0018] In a possible implementation, the first control instruction is to turn on the air conditioner, and the control system corresponding to the first control instruction is an air conditioner control system; the method further includes:

[0019] Obtain a first image collected by the image acquisition device at a second time, where the first image includes an image of the vehicle's driver and passengers, and the second time is the time when the first control instruction is determined;

[0020] Identify the first image to obtain the physical state of the vehicle's driver and passengers;

[0021] When the physical state of the vehicle's driver and passengers includes a pregnancy state, determine a first temperature according to the pregnancy state and the current time;

[0022] Controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes:

[0023] Control the air conditioner control system to execute the control operation corresponding to the first control instruction according to the first temperature.

[0024] In a possible implementation, determining the first temperature according to the pregnancy state and the current time includes:

[0025] Determine the current season according to the current time;

[0026] Use the temperature corresponding to the pregnancy state in the current season as the first temperature.

[0027] In a possible implementation, controlling the air conditioner control system to execute the control operation corresponding to the first control instruction according to the first temperature includes:

[0028] Send the first control instruction and the first temperature to the air conditioning control system, and the air conditioning control system turns on the air conditioner and adjusts it to the first temperature.

[0029] In a possible implementation, the method further includes:

[0030] When the physical state of the vehicle occupants is non-pregnant, determine the age of the vehicle occupants;

[0031] Determine a second temperature according to the age of the vehicle occupants and the current time;

[0032] Controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes:

[0033] Controlling the air conditioning control system to execute the control operation corresponding to the first control instruction according to the second temperature.

[0034] In a possible implementation, the first control instruction is to adjust the target seat, and the control system corresponding to the first control instruction is the seat control system; the method further includes:

[0035] Obtain a second image of the target seat collected by the image acquisition device at a second time, where the second image includes an image of the vehicle occupant located on the target seat, and the second time is the time when the first control instruction is determined;

[0036] Identify the second image to obtain the physical state of the vehicle occupant located on the target seat;

[0037] When the physical state of the vehicle occupant located on the target seat is pregnant, determine first adjustment information according to the pregnancy state;

[0038] Controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes:

[0039] Controlling the seat control system to adjust the target seat according to the first adjustment information.

[0040] In a possible implementation, after controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction, the method further includes:

[0041] Receive control information returned by the control system corresponding to the first control instruction, where the control information is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction;

[0042] Generate a second gesture action according to the control information, where the second gesture action is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction;

[0043] Display the second gesture action.

[0044] On the other hand, an embodiment of the present application provides a vehicle control device, and the device includes:

[0045] A determination module, configured to determine the running state of the vehicle and the percentage of the remaining power of the vehicle in response to an enabling instruction for a voice assistant function equipped in the vehicle;

[0046] A control module, configured to control an image acquisition device installed in the vehicle to be turned on when the running state of the vehicle meets the running requirements and the percentage of the remaining power of the vehicle is greater than a target threshold;

[0047] An acquisition module, configured to acquire a first video collected by the image acquisition device in a first time period, where the first video includes a first gesture action of a driver or passenger of the vehicle, and the start time of the first time period is the time when the image acquisition device is turned on;

[0048] The determination module is further configured to determine a first control instruction corresponding to the first gesture action;

[0049] The control module is further configured to control the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction.

[0050] In a possible implementation manner, the determination module is configured to determine the similarity between the gesture actions corresponding to each control instruction and the first gesture action;

[0051] Use the control instruction corresponding to the gesture action whose similarity to the first gesture action meets the similarity requirement as the first control instruction.

[0052] In a possible implementation manner, for the gesture action corresponding to any one of the control instructions among the various control instructions, the determination module is configured to determine a first feature vector for characterizing the gesture action corresponding to the any one of the control instructions;

[0053] Determine a second feature vector for characterizing the first gesture action;

[0054] According to the first feature vector and the second feature vector, determine the similarity between the gesture action corresponding to the any one of the control instructions and the first gesture action.

[0055] In a possible implementation, the first control instruction is to turn on the air conditioner, and the control system corresponding to the first control instruction is the air conditioner control system;

[0056] The obtaining module is further configured to obtain a first image collected by the image acquisition device at a second time, where the first image includes an image of the vehicle occupants, and the second time is the time when the first control instruction is determined;

[0057] The determining module is further configured to identify the first image to obtain the physical state of the vehicle occupants;

[0058] When the physical state of the vehicle occupants includes a pregnancy state, a first temperature is determined according to the pregnancy state and the current time;

[0059] The control module is configured to control the air conditioner control system to perform the control operation corresponding to the first control instruction according to the first temperature.

[0060] In a possible implementation, the determining module is configured to determine the current season according to the current time;

[0061] The temperature corresponding to the pregnancy state in the current season is used as the first temperature.

[0062] In a possible implementation, the control module is configured to send the first control instruction and the first temperature to the air conditioner control system, and the air conditioner control system turns on the air conditioner and adjusts it to the first temperature.

[0063] In a possible implementation, the determining module is further configured to determine the age of the vehicle occupants when the physical state of the vehicle occupants is a non-pregnancy state;

[0064] A second temperature is determined according to the age of the vehicle occupants and the current time;

[0065] The control module is configured to control the seat control system to perform the control operation corresponding to the first control instruction according to the second temperature.

[0066] In a possible implementation, the first control instruction is to adjust a target seat, and the control system corresponding to the first control instruction is the seat control system;

[0067] The obtaining module is further configured to obtain a second image corresponding to the target seat collected by the image acquisition device at a second time, where the second image includes an image of the occupants located on the target seat, and the second time is the time when the first control instruction is determined;

[0068] The determining module is further configured to identify the second image to obtain the physical state of the occupant in the target seat;

[0069] When the physical state of the occupant in the target seat is a pregnancy state, first adjustment information is determined according to the pregnancy state;

[0070] The control module is configured to control the seat control system to adjust the target seat according to the first adjustment information.

[0071] In a possible implementation manner, the device further includes:

[0072] A receiving module, configured to receive control information returned by the control system corresponding to the first control instruction, where the control information is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction;

[0073] A generating module, configured to generate a second gesture action according to the control information, where the second gesture action is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction;

[0074] A display module, configured to display the second gesture action.

[0075] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory. At least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to enable the computer device to implement any one of the above vehicle control methods.

[0076] On the other hand, a computer-readable storage medium is further provided. At least one program code is stored in the computer-readable storage medium, and the at least one program code is loaded and executed by a processor to enable a computer to implement any one of the above vehicle control methods.

[0077] On the other hand, a computer program or a computer program product is further provided. At least one computer instruction is stored in the computer program or the computer program product, and the at least one computer instruction is loaded and executed by a processor to enable a computer to implement any one of the above vehicle control methods.

[0078] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0079] When the technical solution provided by the embodiment of the present application enables the voice assistant function, it determines the running state of the vehicle and the percentage of the remaining power of the vehicle. When the running state of the vehicle meets the running requirements and the percentage of the remaining power of the vehicle is greater than the target threshold, it turns on the image acquisition device installed in the vehicle, collects the first gesture actions of the vehicle's driver and passengers in the first time period through the image acquisition device, obtains the corresponding first control instruction by recognizing the first gesture actions, and controls the vehicle according to the first control instruction. This method enables deaf-mute people to also control the vehicle through the voice assistant function, improves the usage frequency of the voice assistant function, and moreover, since deaf-mute people do not need to manually control the vehicle, the efficiency of vehicle control can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0081] Figure 1 is a schematic diagram of the implementation environment of a vehicle control method provided by an embodiment of the present application;

[0082] Figure 2 is a flowchart of a vehicle control method provided by an embodiment of the present application;

[0083] Figure 3 is a schematic structural diagram of a vehicle control device provided by an embodiment of the present application;

[0084] Figure 4 is a schematic structural diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0085] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0086] It should be noted that the terms "first", "second", etc. in the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0087] Figure 1It is a schematic diagram of the implementation environment of a vehicle control method provided by an embodiment of this application. As Figure 1 shown, this implementation environment includes: a terminal device 101. The terminal device 101 is a device installed in the vehicle, and the terminal device 101 is communicatively connected to each control system of the vehicle. The terminal device 101 is used to execute the vehicle control method provided by the embodiment of this application.

[0088] Optionally, the terminal device 101 can be any electronic device product that can perform human-computer interaction with the user through one or more ways such as a keyboard, a touchpad, a remote control, voice interaction, or a handwriting device. For example, a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a tablet computer, a smart in-vehicle device, a smart TV, a smart speaker, a smart watch, etc.

[0089] The terminal device 101 can generally refer to one of multiple terminal devices. In this embodiment, only the terminal device 101 is used as an example for illustration. Those skilled in the art can know that the number of the above terminal devices 101 can be more or less. For example, the above terminal device 101 can be only one, or the above terminal device 101 can be dozens or hundreds, or a larger number. The embodiment of this application does not limit the number and device type of the terminal device 101.

[0090] Those skilled in the art should understand that the above terminal device 101 is only for illustration. Other existing or future terminal devices that are applicable to this application should also be included in the protection scope of this application and are hereby incorporated by reference.

[0091] The embodiment of this application provides a vehicle control method, and this method can be applied to the above Figure 1 shown implementation environment. Taking the flowchart of a vehicle control method provided by the embodiment of this application shown in Figure 2 as an example, this method can be executed by the terminal device 101 in Figure 1 . As Figure 2 shown, this method includes the following steps 201 to step 205.

[0092] In step 201, in response to an enabling instruction for the voice assistant function equipped in the vehicle, determine the running state of the vehicle and the percentage of the remaining battery power of the vehicle.

[0093] In a possible implementation, the vehicle is a battery electric vehicle (BEV), or the vehicle is a hybrid electric vehicle (HEV). The embodiments of the present application do not limit the type of the vehicle. Among them, a battery electric vehicle refers to a vehicle that completely relies on a power battery pack to provide energy and drives the wheels through an electric motor. A battery electric vehicle has zero fuel consumption and zero tailpipe emissions, and is one of the directions for the current vehicle industry to transform towards low-carbon and intelligent development. A hybrid electric vehicle refers to a vehicle that is equipped with two power systems, namely a fuel engine and an electric motor, and coordinates the work of the two through an intelligent control system to achieve power complementarity or independently drive the vehicle. A hybrid electric vehicle combines the advantages of traditional fuel vehicles and battery electric vehicles, can not only reduce fuel consumption and emissions, but also alleviate the range anxiety of battery electric vehicles, and is an important transitional solution in the current transformation of vehicle electrification.

[0094] In a possible implementation, the vehicle is equipped with a voice assistant function, and an activation control corresponding to the voice assistant function is displayed on the terminal device. The activation control corresponding to the voice assistant function is in a first state, and the first state is used to indicate that the voice assistant function is not activated. In response to a triggering operation on the activation control corresponding to the voice assistant function, an activation instruction for the voice assistant function equipped in the vehicle is received, and then the running state of the vehicle and the percentage of the remaining battery power of the vehicle are determined.

[0095] In a possible implementation, the process of determining the running state of the vehicle includes: sending a status acquisition request to the vehicle control system, where the status acquisition request is used to acquire the running state of the vehicle; after receiving the status acquisition request, the vehicle control system returns the running state of the vehicle to the terminal device so that the terminal device can acquire the running state of the vehicle.

[0096] Among them, the terminal device and the vehicle control system are communicatively connected through a CAN (Controller Area Network) network. That is, the terminal device sends a status acquisition request to the vehicle control system through the CAN network. The vehicle control system returns the running state of the vehicle to the terminal device through the CAN network.

[0097] In a possible implementation, the process of determining the remaining power percentage of the vehicle includes: sending a remaining power percentage acquisition request to the vehicle battery system, where the remaining power percentage acquisition request is used to acquire the remaining power percentage of the vehicle; the vehicle battery system receives the remaining power percentage acquisition request, and determines the remaining power of the vehicle and the battery capacity of the vehicle according to the remaining power percentage acquisition request; the vehicle battery system determines the remaining power percentage of the vehicle according to the remaining power of the vehicle and the battery capacity of the vehicle; the vehicle battery system sends the remaining power percentage of the vehicle to the terminal device so that the terminal device can acquire the remaining power percentage of the vehicle.

[0098] Among them, the terminal device is communicatively connected to the vehicle battery system through a CAN network. That is, the terminal device sends a remaining power percentage acquisition request to the vehicle battery system through the CAN network. The vehicle battery system sends the remaining power percentage of the vehicle to the terminal device through the CAN network.

[0099] Optionally, the process in which the vehicle battery system determines the remaining power percentage of the vehicle according to the remaining power of the vehicle and the battery capacity of the vehicle includes: taking the ratio between the remaining power of the vehicle and the battery capacity of the vehicle as the remaining power percentage of the vehicle.

[0100] In another possible implementation, the process of determining the remaining power percentage of the vehicle can also be: sending a power acquisition request to the vehicle battery system, where the power acquisition request is used to acquire the remaining power of the vehicle and the battery capacity of the vehicle; the vehicle battery system sends the remaining power of the vehicle and the battery capacity of the vehicle to the terminal device, and the terminal device determines the remaining power percentage of the vehicle according to the remaining power of the vehicle and the battery capacity of the vehicle.

[0101] Among them, the terminal device is communicatively connected to the vehicle battery system through a CAN network. That is, the terminal device sends a power acquisition request to the vehicle battery system through the CAN network. The vehicle battery system sends the remaining power of the vehicle and the battery capacity of the vehicle to the terminal device through the CAN network.

[0102] Optionally, the process in which the terminal device determines the remaining power percentage of the vehicle according to the remaining power of the vehicle and the battery capacity of the vehicle is similar to the process in which the vehicle battery system determines the remaining power percentage of the vehicle according to the remaining power of the vehicle and the battery capacity of the vehicle, and this is not elaborated in this embodiment of the present application.

[0103] In step 202, when the operating state of the vehicle meets the operating requirements and the remaining power percentage of the vehicle is greater than the target threshold, control the image acquisition device installed in the vehicle to turn on.

[0104] In a possible implementation, an image acquisition device is installed in the vehicle. The number of image acquisition devices installed in the vehicle can be one or multiple, and the embodiments of the present application do not limit this. The embodiments of the present application also do not limit the installation position of the image acquisition device. Exemplarily, if there is one image acquisition device installed in the vehicle, the image acquisition device can be installed in front of the driver's seat. If there are multiple image acquisition devices installed in the vehicle, one image acquisition device can be installed in front of each position. The image acquisition device installed in the vehicle can be any device capable of acquiring images, and the embodiments of the present application do not limit the type of the image acquisition device. Exemplarily, the image acquisition device is a camera.

[0105] After determining the running state of the vehicle and the percentage of the remaining battery power of the vehicle in step 201 above, determine whether the running state of the vehicle meets the running requirements, and determine whether the percentage of the remaining battery power of the vehicle is greater than the target threshold. When the running state of the vehicle meets the running requirements and the percentage of the remaining battery power of the vehicle is greater than the target threshold, control the image acquisition device installed in the vehicle to turn on.

[0106] Among them, the running state of the vehicle meeting the running requirements means that the running state of the vehicle is a normal running state. The target threshold is set based on experience or flexibly adjusted according to the implementation environment, and the embodiments of the present application do not limit this. Exemplarily, the target threshold is 20%.

[0107] Optionally, the process of controlling the image acquisition device installed in the vehicle to turn on includes: sending a turn-on request to the image acquisition device installed in the vehicle. After the image acquisition device receives the turn-on request, it turns on the image acquisition device.

[0108] Among them, the terminal device and the image acquisition device are communicatively connected through a CAN network. That is, the terminal device sends a turn-on request to the image acquisition device through the CAN network.

[0109] Optionally, when the running state of the vehicle meets the running requirements and the percentage of the remaining battery power of the vehicle is greater than the target threshold, control the turn-on control corresponding to the voice assistant function to be in a second state, and the second state is used to indicate that the voice assistant function has been turned on.

[0110] Optionally, after controlling the turn-on control corresponding to the voice assistant function to be in the second state, the voice assistant function turn-on information can also be displayed, and the voice assistant function turn-on information is used to indicate that the voice assistant function of the vehicle has been turned on. The embodiments of the present application do not limit the content of the voice assistant function turn-on information.

[0111] Optionally, after the enabling control corresponding to the voice assistant function is in the second state, voice data generated based on the enabling information of the voice assistant function may also be played, so that non-deaf-mute people in the vehicle can know that the voice assistant function of the vehicle has been enabled through the voice data generated based on the enabling information of the voice assistant function.

[0112] In a possible implementation manner, when the operating state of the vehicle does not meet the operating requirements and / or the remaining battery percentage of the vehicle is not greater than the target threshold, a prompt message is displayed, and the prompt message is used to indicate that the image acquisition device installed in the vehicle cannot be enabled. The embodiments of the present application do not limit the content of the prompt message.

[0113] Wherein, the operating state of the vehicle not meeting the operating requirements means that the operating state of the vehicle is not a normal operating state.

[0114] Optionally, when the operating state of the vehicle does not meet the operating requirements and / or the remaining battery percentage of the vehicle is not greater than the target threshold, voice data generated based on the prompt message is played.

[0115] In step 203, a first video collected by the image acquisition device in the first time period is obtained. The first video includes the first gesture actions of the vehicle occupants. The start time of the first time period is the time when the image acquisition device is enabled.

[0116] In a possible implementation manner, after the image acquisition device is enabled, the image acquisition device starts to collect videos. After the image acquisition device collects the first video in the first time period, the first video in the first time period is sent to the terminal device, so that the terminal device can obtain the first video collected by the image acquisition device in the first time period. The first video includes the first gesture actions of the vehicle occupants. The first gesture actions may include one gesture action or multiple gesture actions. The embodiments of the present application do not limit this.

[0117] Wherein, the start time of the first time period is the time when the image acquisition device is enabled, and the duration of the first time period is the target duration. The target duration is set based on experience or flexibly adjusted according to the implementation environment. The embodiments of the present application do not limit this. Exemplarily, the target duration is 10 seconds.

[0118] In step 204, a first control instruction corresponding to the first gesture action is determined.

[0119] In a possible implementation, gesture actions corresponding to respective control instructions are stored in a terminal device. The process of determining a first control instruction corresponding to a first gesture action includes: determining the similarity between the gesture actions corresponding to the respective control instructions and the first gesture action; using the control instruction corresponding to the gesture action whose similarity to the first gesture action meets the similarity requirement as the first control instruction.

[0120] Optionally, the process of determining the similarity between the gesture actions corresponding to the respective control instructions and the first gesture action includes: for the gesture action corresponding to any one of the respective control instructions, determining a first feature vector for representing the gesture action corresponding to any one of the control instructions; determining a second feature vector for representing the first gesture action; and determining the similarity between the gesture action corresponding to any one of the control instructions and the first gesture action according to the first feature vector and the second feature vector.

[0121] Among them, the process of determining the first feature vector for representing the gesture action corresponding to any one of the control instructions is similar to the process of determining the second feature vector for representing the first gesture action. In the embodiments of the present application, only the process of determining the first feature vector for representing the gesture action corresponding to any one of the control instructions is taken as an example for illustration.

[0122] Optionally, the process of determining the first feature vector for representing the gesture action corresponding to any one of the control instructions includes: invoking a feature vector determination model to determine the first feature vector for representing the gesture action corresponding to any one of the control instructions. That is, inputting the gesture action corresponding to any one of the control instructions into the feature vector determination model to obtain the first feature vector for representing the gesture action corresponding to any one of the control instructions.

[0123] Among them, the feature vector determination model can be any model capable of determining a feature vector, and the embodiments of the present application do not limit this.

[0124] Optionally, the process of determining the similarity between the gesture action corresponding to any one of the control instructions and the first gesture action according to the first feature vector and the second feature vector includes: determining the Euclidean distance between the first feature vector and the second feature vector; using the reciprocal of the Euclidean distance between the first feature vector and the second feature vector as the similarity between the gesture action corresponding to any one of the control instructions and the first gesture action.

[0125] Optionally, according to the first feature vector and the second feature vector, the similarity between the gesture action corresponding to any one of the control instructions and the first gesture action is determined according to the following formula (1).

[0126]

[0127] In the above formula (1), S iis the similarity between the gesture action corresponding to the i-th control instruction and the first gesture action, (x1, x2, x3, …, x n ) is the first feature vector used to characterize the gesture action corresponding to the i-th control instruction, (y1, y2, y3, …, y n ) is the second feature vector used to characterize the first gesture action.

[0128] Optionally, the similarity meeting the similarity requirement means the highest similarity, that is, the control instruction corresponding to the gesture action with the highest similarity to the first gesture action is used as the first control instruction.

[0129] In step 205, control the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction.

[0130] In a possible implementation manner, after determining the first control instruction in the above step 204, the process of controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes: sending the first control instruction to the control system corresponding to the first control instruction, and the control system corresponding to the first control instruction executes the control operation corresponding to the first control instruction.

[0131] Among them, the terminal device and the control system corresponding to the first control instruction are communicatively connected through a CAN network. That is, the terminal device sends the first control instruction to the control system corresponding to the first control instruction through the CAN network.

[0132] In a possible implementation manner, the first control instruction is to turn on the air conditioner, and the control system corresponding to the first control instruction is the air conditioner control system. It is also possible to obtain a first image collected by the image acquisition device at a second time, where the first image includes an image of the vehicle's driver and passengers, and the second time is the time when the first control instruction is determined; identify the first image to obtain the physical state of the vehicle's driver and passengers; in the case where the physical state of the vehicle's driver and passengers includes a pregnancy state, determine the first temperature according to the pregnancy state and the current time. Then control the air conditioner system to execute the control operation corresponding to the first control instruction according to the first temperature.

[0133] Optionally, the process of obtaining the first image collected by the image acquisition device at the second time includes: sending an image acquisition request to the image acquisition device, where the image acquisition request includes the second time; after receiving the image acquisition request, the image acquisition device parses the image acquisition request to obtain the second time, determines the first image collected at the second time, and then sends the first image to the terminal device so that the terminal device can obtain the first image collected by the image acquisition device at the second time.

[0134] Optionally, the first image is recognized by a state recognition model to obtain the physical state of the vehicle occupants. The physical state of the vehicle occupants includes a pregnancy state and a non-pregnancy state.

[0135] In one possible implementation, the process of determining the first temperature according to the pregnancy state and the current time includes: determining the current season according to the current time; using the temperature corresponding to the pregnancy state in the current season as the first temperature.

[0136] Optionally, there are four temperatures corresponding to the pregnancy state, one temperature corresponding to one season. Among the four temperatures corresponding to the pregnancy state, the temperature corresponding to the current season is used as the first temperature.

[0137] Exemplarily, the four temperatures corresponding to the pregnancy state are A, B, C, and D. Among them, the season corresponding to A is spring, the season corresponding to B is summer, the season corresponding to C is autumn, and the season corresponding to D is winter. According to the current time, the determined current season is summer. Therefore, B is determined as the first temperature.

[0138] Optionally, the process of controlling the air conditioning system to perform the control operation corresponding to the first control instruction according to the first temperature includes: sending the first control instruction and the first temperature to the air conditioning system, and the air conditioning control system turns on the air conditioner and adjusts it to the first temperature.

[0139] In one possible implementation, when the physical state of the vehicle occupants is in a non-pregnancy state, the age of the vehicle occupants is determined; according to the age of the vehicle occupants and the current time, the second temperature is determined; the air conditioning control system is controlled to perform the control operation corresponding to the first control instruction according to the second temperature.

[0140] Among them, the process of determining the second temperature according to the age of the vehicle occupants and the current time includes: determining the current season according to the current time; when there are vehicle occupants with an age greater than the first age among the vehicle occupants, using the temperature corresponding to the first age that corresponds to the current season as the second temperature. When there are no vehicle occupants with an age greater than the first age among the vehicle occupants and there are vehicle occupants with an age less than the second age among the vehicle occupants, using the temperature corresponding to the second age that corresponds to the current season as the second temperature. When there are no vehicle occupants with an age greater than the first age among the vehicle occupants and there are no vehicle occupants with an age less than the second age among the vehicle occupants, determining the age range in which the age of the vehicle occupants is located; using the temperature corresponding to the age range that corresponds to the current season as the second temperature.

[0141] Among them, the first age is greater than the second age. Exemplarily, the first age is 60 years old and the second age is 10 years old.

[0142] Exemplarily, if there is a vehicle occupant over 60 years old among the vehicle occupants, the temperature corresponding to the current season in the temperature corresponding to 60 years old is used as the second temperature. If there is no vehicle occupant over 60 years old among the vehicle occupants, but there is a vehicle occupant under 10 years old, the temperature corresponding to the current season in the temperature corresponding to 10 years old is used as the second temperature. When there is no vehicle occupant over 60 years old and no vehicle occupant under 10 years old among the vehicle occupants, determine the age range in which the age of the vehicle occupants is located, and use the temperature corresponding to the age range and corresponding to the current season as the second temperature.

[0143] Optionally, the process of controlling the air conditioning control system to perform the control operation corresponding to the first control instruction according to the second temperature includes: sending the first control instruction and the second temperature to the air conditioning control system, and the air conditioning control system turns on the air conditioner and adjusts it to the second temperature.

[0144] In a possible implementation manner, the first control instruction is to adjust the target seat, and the control system corresponding to the first control instruction is the seat control system. It is also possible to obtain a second image of the target seat collected by the image acquisition device at the second time. The second image includes an image of the occupant located on the target seat. The second time is the time when the first control instruction is determined; identify the second image to obtain the body state of the occupant located on the target seat; in the case where the body state of the occupant located on the target seat is a pregnancy state, determine the first adjustment information according to the pregnancy state; control the seat control system to adjust the target seat according to the first adjustment information.

[0145] Optionally, the process of determining the first adjustment information according to the pregnancy state includes: using the adjustment information corresponding to the pregnancy state as the first adjustment information.

[0146] The process of controlling the seat control system to adjust the target seat according to the first adjustment information includes: sending the first adjustment information and the seat identifier of the target seat to the seat control system, and the seat control system adjusts the target seat according to the first adjustment information.

[0147] Optionally, the terminal device and the seat control system are communicatively connected through a CAN network. That is, the terminal device sends the first adjustment information to the seat control system through the CAN network.

[0148] Wherein, the first adjustment information includes at least one of a seat ventilation gear, a seat heating gear, a seat massage intensity, a seat lumbar support intensity, a seat leg rest intensity, a seat backrest angle, and a seat height.

[0149] In a possible implementation, when the physical state of the occupant in the target seat is a non-pregnant state, determine the age of the occupant in the target seat; determine the second adjustment information according to the age of the occupant in the target seat; control the seat control system to adjust the target seat according to the second adjustment information.

[0150] Among them, the process of determining the second adjustment information according to the age of the occupant in the target seat includes: determining the age range in which the age of the occupant in the target seat is located; taking the adjustment information corresponding to the age range in which the age of the occupant in the target seat is located as the second adjustment information.

[0151] The process of controlling the seat control system to adjust the target seat according to the second adjustment information includes: sending the second adjustment information and the seat identifier of the target seat to the seat control system, and the seat control system adjusts the target seat according to the second adjustment information.

[0152] In a possible implementation, after controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction, it is also possible to receive the control information returned by the control system corresponding to the first control instruction, where the control information is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction; generate a second gesture action according to the control information, where the second gesture action is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction; display the second gesture action. So that the occupant can know that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction by watching the second gesture action.

[0153] Optionally, it is also possible to display the control information so that the occupant can know that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction by watching the control information.

[0154] Optionally, it is also possible to generate first voice data corresponding to the control information and play the first voice data, so that non-deaf occupants can know that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction by playing the first voice data.

[0155] In a possible implementation, the image acquisition device can also acquire images inside the vehicle. When the images acquired by the image acquisition device include vehicle occupants in a smoking state, the image acquisition device sends a first prompt message to the terminal device. The first prompt message is used to indicate that there are vehicle occupants in a smoking state in the vehicle. After receiving the first prompt message, the terminal device generates a third gesture action. The third gesture action is used to indicate that smoking is prohibited in the vehicle. The terminal device displays the third gesture action to remind the vehicle occupants in a smoking state by showing the third gesture action.

[0156] In a possible implementation, when there are vehicle occupants in a smoking state in the vehicle, the smoke concentration in the vehicle is determined. When the smoke concentration in the vehicle is higher than the concentration threshold, the vehicle windows are controlled to open.

[0157] Among them, the concentration threshold is set based on experience or flexibly adjusted according to the implementation environment. The embodiments of the present application do not limit this.

[0158] Optionally, a smoke sensor is installed in the vehicle. The smoke sensor is used to detect the smoke concentration in the vehicle and send the smoke concentration in the vehicle to the terminal device.

[0159] After the vehicle windows are opened, the smoke concentration in the vehicle can also be monitored in real time. When the smoke concentration in the vehicle is lower than the concentration threshold, the vehicle windows are controlled to close.

[0160] Optionally, the terminal device can also generate second voice data. The second voice data is used to indicate that smoking is prohibited in the vehicle. The second voice data is played to remind the vehicle occupants in a smoking state by playing the second voice data.

[0161] In a possible implementation, when the vehicle battery system monitors that the remaining battery percentage of the vehicle is not greater than the target threshold, the vehicle battery system sends a second prompt message to the terminal device. The second prompt message is used to indicate that the remaining battery percentage of the vehicle is not greater than the target threshold. After receiving the second prompt message, the terminal device controls the vehicle engine to start.

[0162] In a possible implementation, after receiving the second prompt message, the terminal device can also determine the charging station closest to the current location of the vehicle, play a seventh gesture action. The seventh gesture action is used to indicate whether to go to the charging station closest to the current location of the vehicle; when a confirmation instruction to go is received, a second navigation route is generated. The end point of the second navigation route is the charging station closest to the current location of the vehicle, so that the driver can navigate to the charging station closest to the current location of the vehicle according to the second navigation route.

[0163] In a possible implementation, when the first control instruction is any one of opening the window at the target position / closing the window at the target position / opening the sunroof / closing the sunroof / half-opening the window at the target position / half-closing the window at the target position / half-opening the sunroof / half-closing the sunroof, the control system corresponding to the first control instruction is the body control system; the terminal device sends the first control instruction to the body control system, and the body control system executes the control operation corresponding to the first control instruction. That is, when the first control instruction is to open the window at the target position, the body control system opens the window at the target position. When the first control instruction is other, the body control system executes the corresponding control operation.

[0164] In a possible implementation, after obtaining the first video collected by the image acquisition device in the first time period, a fourth gesture action is played, and the fourth gesture action is a gesture action for soothing the vehicle occupants.

[0165] In a possible implementation, the image acquisition device collects the image of the vehicle driver in real time. When the image acquisition device determines that the vehicle driver is fatigued, the image acquisition device sends a third prompt message to the terminal device, and the third prompt message is used to indicate that the vehicle driver is fatigued. After receiving the third prompt message, the terminal device determines the service area closest to the vehicle according to the current location of the vehicle, and plays a fifth gesture action, and the fifth gesture action is used to indicate whether to go to the service area closest to the vehicle. When a confirmation signal is received within the reference duration, or when no confirmation signal is received within the reference duration, a first navigation route is generated, and the end point of the first navigation route is the service area closest to the vehicle, so that the driver navigates to the service area closest to the vehicle according to the first navigation route. When a rejection signal is received within the reference duration, music is played to improve the driver's fatigue by playing music.

[0166] Among them, the reference duration is set based on experience, or flexibly adjusted according to the implementation environment, and the embodiments of the present application do not limit this. Exemplarily, the reference duration is 3 seconds.

[0167] Optionally, the image acquisition device determines whether the vehicle driver is fatigued according to the image collected in real time. If the eyes of the vehicle driver are in a closed state in a target number of consecutive images collected by the image acquisition device, it is determined that the vehicle driver is fatigued. Among them, the target number is set based on experience, or flexibly adjusted according to the implementation environment, and the embodiments of the present application do not limit this. Exemplarily, the target number is 5.

[0168] Optionally, the terminal device can also determine whether the driver of the vehicle is fatigued based on the running duration of the vehicle. In the case of determining that the driver of the vehicle is fatigued, based on the current location of the vehicle, determine the service area closest to the vehicle, and play a fifth gesture action, where the fifth gesture action is used to indicate whether to go to the service area closest to the vehicle. When a confirmation signal is received within the reference duration, or when no confirmation signal is received within the reference duration, generate a first navigation route, where the end point of the first navigation route is the service area closest to the vehicle, so that the driver can navigate to the service area closest to the vehicle according to the first navigation route. In the case of receiving a rejection signal within the reference duration, play music to improve the driver's fatigue by playing music.

[0169] Among them, the process of determining whether the driver of the vehicle is fatigued based on the running duration of the vehicle includes: in the case where the running duration of the vehicle is greater than the first duration, determine that the driver of the vehicle is fatigued; in the case where the running duration of the vehicle is not greater than the first duration, determine that the driver of the vehicle is not fatigued. The first duration is set based on experience, or flexibly adjusted according to the implementation environment, and the embodiments of the present application do not limit this. Exemplarily, the first duration is 4 hours.

[0170] In a possible implementation, in the case where the driver of the vehicle is fatigued, control the vehicle's assisted driving to turn on, so as to help the driver of the vehicle control the vehicle through the vehicle's assisted driving.

[0171] In a possible implementation, when the vehicle is in a service area, or when the driver of the vehicle is not fatigued, display a sixth prompt message, where the sixth prompt message is used to indicate whether to exit the assisted driving. In the case of receiving a confirmation operation for the sixth prompt message, exit the assisted driving.

[0172] In a possible implementation, while playing the fifth gesture action, display a confirmation control and a rejection control. In response to a trigger operation on the confirmation control within the reference duration, a confirmation signal is received within the reference duration; in response to a trigger operation on the rejection control within the reference duration, a rejection signal is received within the reference duration.

[0173] In a possible implementation, when the image acquisition device monitors that the driver of the vehicle is not fatigued, the image acquisition device sends a fourth prompt message to the terminal device, where the fourth prompt message is used to indicate that the driver of the vehicle is not fatigued. The terminal device plays a sixth gesture action, where the sixth gesture action is used to indicate whether to turn off the music. In the case of receiving an instruction to turn off the music, turn off the music.

[0174] In a possible implementation, in the event of a vehicle collision, the terminal device sends a fifth prompt message to the emergency contact of the vehicle. The fifth prompt message includes the location information of the target location and the identification of the vehicle, and the fifth prompt message is used to indicate that the vehicle has collided at the target location.

[0175] Optionally, in the event of a vehicle collision, the terminal device also sends the location information of the target location to the cloud, so that the cloud knows that a collision has occurred at the target location. In this way, the cloud can send a prompt message to other vehicles that need to pass through the target location to prompt them that a collision has occurred at the target location.

[0176] In a possible implementation, the terminal device can also establish a communication connection with the device used by the vehicle driver. The terminal device can also obtain the health information of the vehicle driver from the device used by the vehicle driver. The health information includes but is not limited to blood pressure, pulse, sleep information, etc. According to the health information of the vehicle driver, a health information suggestion is generated, and then the health information suggestion is fed back to the device used by the vehicle driver, so that the vehicle driver can view the health information suggestion. Of course, after the terminal device generates the health information suggestion, it can also display the gesture actions representing the health information suggestion, so that the vehicle driver can view the gesture actions representing the health information suggestion to receive the health information suggestion.

[0177] In a possible implementation, when the operating state of the vehicle does not meet the operating requirements, or a shutdown instruction for the voice assistant function equipped in the vehicle is received, the voice assistant function equipped in the vehicle is shut down, and at the same time, the image acquisition device installed in the vehicle is controlled to shut down.

[0178] When the above method enables the voice assistant function, it determines the operating state of the vehicle and the percentage of the remaining power of the vehicle. When the operating state of the vehicle meets the operating requirements and the percentage of the remaining power of the vehicle is greater than the target threshold, the image acquisition device installed in the vehicle is enabled. The first gesture actions of the vehicle occupants in the first time period are acquired through the image acquisition device, and the corresponding first control instruction is obtained by recognizing the first gesture actions, and the vehicle is controlled according to the first control instruction. This method enables deaf-mute people to control the vehicle through the voice assistant function, improves the usage frequency of the voice assistant function, and moreover, since deaf-mute people do not need to manually control the vehicle, the vehicle control efficiency can be improved.

[0179] Figure 3 The following shows a schematic structural diagram of a vehicle control device provided by an embodiment of the present application, as Figure 3 shown, the device includes:

[0180] Determination module 301, configured to determine the running state of the vehicle and the percentage of the remaining power of the vehicle in response to an activation instruction for the voice assistant function equipped in the vehicle;

[0181] Control module 302, configured to control the image acquisition device installed in the vehicle to be turned on when the running state of the vehicle meets the running requirements and the percentage of the remaining power of the vehicle is greater than the target threshold;

[0182] Acquisition module 303, configured to acquire a first video collected by the image acquisition device in a first time period, where the first video includes a first gesture action of the vehicle occupants, and the start time of the first time period is the time when the image acquisition device is turned on;

[0183] Determination module 301, further configured to determine a first control instruction corresponding to the first gesture action;

[0184] Control module 302, further configured to control the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction.

[0185] In a possible implementation manner, determination module 301 is configured to determine the similarity between the gesture actions corresponding to each control instruction and the first gesture action;

[0186] Use the control instruction corresponding to the gesture action whose similarity to the first gesture action meets the similarity requirement as the first control instruction.

[0187] In a possible implementation manner, determination module 301 is configured to determine, for the gesture action corresponding to any one of the control instructions, a first feature vector for characterizing the gesture action corresponding to any one of the control instructions;

[0188] Determine a second feature vector for characterizing the first gesture action;

[0189] Determine the similarity between the gesture action corresponding to any one of the control instructions and the first gesture action according to the first feature vector and the second feature vector.

[0190] In a possible implementation manner, the first control instruction is to turn on the air conditioner, and the control system corresponding to the first control instruction is the air conditioner control system;

[0191] Acquisition module 303, further configured to acquire a first image collected by the image acquisition device at a second time, where the first image includes an image of the vehicle occupants, and the second time is the time when the first control instruction is determined;

[0192] Determination module 301, further configured to identify the first image to obtain the physical state of the vehicle occupants;

[0193] When the physical state of the vehicle occupants includes a pregnancy state, determine a first temperature according to the pregnancy state and the current time;

[0194] The control module 302 is configured to control the air conditioning control system to perform the control operation corresponding to the first control instruction according to the first temperature.

[0195] In a possible implementation, the determination module 301 is configured to determine the current season according to the current time;

[0196] Use the temperature corresponding to the pregnancy state in the current season as the first temperature.

[0197] In a possible implementation, the control module is configured to send a first control instruction and a first temperature to the air conditioning control system, and the air conditioning control system turns on the air conditioner and adjusts it to the first temperature.

[0198] In a possible implementation, the determination module 301 is further configured to determine the age of the vehicle occupants when the physical state of the vehicle occupants is a non-pregnancy state;

[0199] Determine a second temperature according to the age of the vehicle occupants and the current time;

[0200] The control module 302 is configured to control the air conditioning control system to perform the control operation corresponding to the first control instruction according to the second temperature.

[0201] In a possible implementation, the first control instruction is to adjust the target seat, and the control system corresponding to the first control instruction is the seat control system;

[0202] The acquisition module 303 is further configured to acquire a second image of the target seat collected by the image acquisition device at a second time, and the second image includes an image of the occupant located on the target seat, and the second time is the time when the first control instruction is determined;

[0203] The determination module 301 is further configured to identify the second image to obtain the physical state of the occupant located on the target seat;

[0204] When the physical state of the occupant located on the target seat is a pregnancy state, determine a first adjustment information according to the pregnancy state;

[0205] The control module 302 is configured to control the seat control system to adjust the target seat according to the first adjustment information.

[0206] In a possible implementation, the device further includes:

[0207] A receiving module, configured to receive control information returned by a control system corresponding to a first control instruction, where the control information is used to indicate that the control system corresponding to the first control instruction has executed a control operation corresponding to the first control instruction;

[0208] A generating module, configured to generate a second gesture action according to the control information, where the second gesture action is used to indicate that the control system corresponding to the first control instruction has executed a control operation corresponding to the first control instruction;

[0209] A display module, configured to display the second gesture action.

[0210] When the above device enables the voice assistant function, it determines the running state of the vehicle and the percentage of the remaining power of the vehicle. When the running state of the vehicle meets the running requirements and the percentage of the remaining power of the vehicle is greater than the target threshold, it enables an image acquisition device installed in the vehicle, acquires a first gesture action of the vehicle driver and passenger in a first time period through the image acquisition device, obtains a corresponding first control instruction by recognizing the first gesture action, and controls the vehicle according to the first control instruction. This method enables deaf-mute people to control the vehicle through the voice assistant function, improves the usage frequency of the voice assistant function, and moreover, since deaf-mute people do not need to manually control the vehicle, the efficiency of vehicle control can be improved.

[0211] It should be understood that when the above provided device implements its functions, only the above division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be elaborated here.

[0212] Figure 4 FIG. shows a structural block diagram of a terminal device 400 provided in an exemplary embodiment of the present application. The terminal device 400 can be any electronic device product that can perform human-computer interaction with a user in one or more ways such as a keyboard, a touchpad, a remote control, voice interaction, or a handwriting device. For example, a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a tablet computer, a smart vehicle console, a smart TV, a smart speaker, a smart watch, etc.

[0213] Generally, the terminal device 400 includes a processor 401 and a memory 402.

[0214] The processor 401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0215] The memory 402 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 402 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 401 to implement the vehicle control method provided in the method embodiments of the present application.

[0216] In some embodiments, the terminal device 400 may further optionally include: a peripheral device interface 403 and at least one peripheral device. The processor 401, the memory 402, and the peripheral device interface 403 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 403 through a bus, signal lines, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 404, a display screen 405, a camera assembly 406, an audio circuit 407, and a power supply 408.

[0217] The peripheral device interface 403 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 401 and the memory 402. In some embodiments, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 can be implemented on separate chips or circuit boards, and this embodiment does not limit this.

[0218] The radio frequency circuit 404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 404 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 404 converts an electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so on. The radio frequency circuit 404 can communicate with other terminal devices through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, each generation of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 404 can also include a circuit related to NFC (Near Field Communication), and this application does not limit this.

[0219] The display screen 405 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 405 is a touch display screen, the display screen 405 also has the ability to collect touch signals on or above the surface of the display screen 405. The touch signals can be input as control signals to the processor 401 for processing. At this time, the display screen 405 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 405, which is disposed on the front panel of the terminal device 400; in other embodiments, there may be at least two display screens 405, which are respectively disposed on different surfaces of the terminal device 400 or are in a folded design; in other embodiments, the display screen 405 may be a flexible display screen, which is disposed on a curved surface or a folding surface of the terminal device 400. Even further, the display screen 405 can also be set to an irregular non-rectangular shape, that is, a special-shaped screen. The display screen 405 can be prepared using materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0220] The camera module 406 is used to collect images or videos. Optionally, the camera module 406 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the terminal device 400, and the rear camera is disposed on the back of the terminal device 400. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth camera, a wide-angle camera, and a telephoto camera respectively, to implement functions such as background blurring by fusing the main camera and the depth camera, panoramic shooting by fusing the main camera and the wide-angle camera, and VR (Virtual Reality) shooting functions or other fused shooting functions. In some embodiments, the camera module 406 may also include a flash. The flash can be a single-color temperature flash or a two-color temperature flash. A two-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0221] The audio circuit 407 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals for input to the processor 401 for processing, or input to the radio frequency circuit 404 to achieve voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal device 400. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for uses such as ranging. In some embodiments, the audio circuit 407 may also include a headphone jack.

[0222] The power supply 408 is used to supply power to each component in the terminal device 400. The power supply 408 may be alternating current, direct current, a disposable battery or a rechargeable battery. When the power supply 408 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0223] In some embodiments, the terminal device 400 further includes one or more sensors 409. The one or more sensors 409 include but are not limited to: an acceleration sensor 410, a gyroscope sensor 411, a pressure sensor 412, an optical sensor 413, and a proximity sensor 414.

[0224] The acceleration sensor 410 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established with the terminal device 400. For example, the acceleration sensor 410 can be used to detect the components of the gravitational acceleration on the three coordinate axes. The processor 401 can control the display screen 405 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 410. The acceleration sensor 410 can also be used for collecting game or user's motion data.

[0225] The gyroscope sensor 411 can detect the body direction and rotation angle of the terminal device 400. The gyroscope sensor 411 can cooperate with the acceleration sensor 410 to collect the 3D actions of the user on the terminal device 400. According to the data collected by the gyroscope sensor 411, the processor 401 can achieve the following functions: motion sensing (such as changing the UI according to the user's tilting operation), image stabilization during shooting, game control, and inertial navigation.

[0226] The pressure sensor 412 can be disposed on the side frame of the terminal device 400 and / or the lower layer of the display screen 405. When the pressure sensor 412 is disposed on the side frame of the terminal device 400, it can detect the holding signal of the user on the terminal device 400, and the processor 401 can perform left / right hand recognition or quick operation according to the holding signal collected by the pressure sensor 412. When the pressure sensor 412 is disposed on the lower layer of the display screen 405, the processor 401 can control the operable controls on the UI interface according to the pressure operation of the user on the display screen 405. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0227] The optical sensor 413 is used to collect the ambient light intensity. In one embodiment, the processor 401 can control the display brightness of the display screen 405 according to the ambient light intensity collected by the optical sensor 413. Specifically, when the ambient light intensity is high, the display brightness of the display screen 405 is increased; when the ambient light intensity is low, the display brightness of the display screen 405 is decreased. In another embodiment, the processor 401 can also dynamically adjust the shooting parameters of the camera assembly 406 according to the ambient light intensity collected by the optical sensor 413.

[0228] The proximity sensor 414, also known as the distance sensor, is usually disposed on the front panel of the terminal device 400. The proximity sensor 414 is used to collect the distance between the user and the front of the terminal device 400. In one embodiment, when the proximity sensor 414 detects that the distance between the user and the front of the terminal device 400 is gradually decreasing, the processor 401 controls the display screen 405 to switch from the lit state to the off state; when the proximity sensor 414 detects that the distance between the user and the front of the terminal device 400 is gradually increasing, the processor 401 controls the display screen 405 to switch from the off state to the lit state.

[0229] Those skilled in the art can understand that Figure 4 the structure shown in does not constitute a limitation on the terminal device 400, and it may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.

[0230] In an exemplary embodiment, a computer-readable storage medium is further provided, and at least one program code is stored in the storage medium. The at least one program code is loaded and executed by a processor to enable a computer to implement any one of the above vehicle control methods.

[0231] Optionally, the above computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, an optical data storage device, or the like.

[0232] In an exemplary embodiment, a computer program or a computer program product is further provided. At least one computer instruction is stored in the computer program or the computer program product, and the at least one computer instruction is loaded and executed by a processor to enable a computer to implement any one of the above vehicle control methods.

[0233] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0234] It should be understood that "a plurality of" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0235] The above are only exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that, The method includes: In response to an activation instruction for a voice assistant function equipped in a vehicle, determining an operating state of the vehicle and a percentage of remaining power of the vehicle; When the operating state of the vehicle meets the operating requirements and the percentage of remaining power of the vehicle is greater than a target threshold, controlling an image acquisition device installed in the vehicle to be activated; Obtaining a first video collected by the image acquisition device in a first time period, where the first video includes first gesture actions of a driver and passengers of the vehicle, and a start time of the first time period is the time when the image acquisition device is activated; Determining a first control instruction corresponding to the first gesture actions; Controlling a control system corresponding to the first control instruction to execute a control operation corresponding to the first control instruction.

2. The method according to claim 1, characterized in that, The determining the first control instruction corresponding to the first gesture actions includes: Determining a similarity between gesture actions corresponding to each control instruction and the first gesture actions; Taking the control instruction corresponding to the gesture action whose similarity to the first gesture actions meets the similarity requirement as the first control instruction.

3. The method according to claim 2, wherein The determining the similarity between gesture actions corresponding to each control instruction and the first gesture actions includes: For a gesture action corresponding to any one of the control instructions, determining a first feature vector for characterizing the gesture action corresponding to the any one of the control instructions; Determining a second feature vector for characterizing the first gesture actions; Determining a similarity between the gesture action corresponding to the any one of the control instructions and the first gesture actions according to the first feature vector and the second feature vector.

4. The method according to any one of claims 1 to 3, characterized in that The first control instruction is to turn on the air conditioner, and the control system corresponding to the first control instruction is an air conditioner control system; the method further includes: Obtaining a first image collected by the image acquisition device at a second time, where the first image includes an image of a driver and passengers of the vehicle, and the second time is the time when the first control instruction is determined; Identifying the first image to obtain a physical state of a driver and passengers of the vehicle; When the physical state of a driver and passengers of the vehicle includes a pregnancy state, determining a first temperature according to the pregnancy state and the current time; The controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes: Controlling the air conditioner control system to execute the control operation corresponding to the first control instruction according to the first temperature.

5. The method according to claim 4, characterized in that The determining the first temperature according to the pregnancy state and the current time includes: Determining a current season according to the current time; Taking the temperature corresponding to the pregnancy state in the current season as the first temperature.

6. The method according to claim 4, characterized in that, The controlling the air conditioner control system to execute the control operation corresponding to the first control instruction according to the first temperature includes: Sending the first control instruction and the first temperature to the air conditioner control system, and the air conditioner control system turns on the air conditioner and adjusts it to the first temperature.

7. The method according to claim 4, characterized in that The method further includes: When the physical state of a driver and passengers of the vehicle is a non-pregnancy state, determining an age of a driver and passengers of the vehicle; Determine a second temperature according to the age of the vehicle occupants and the current time; The controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes: Controlling the air-conditioning control system to execute the control operation corresponding to the first control instruction according to the second temperature.

8. The method according to any one of claims 1 to 3, characterized in that, The first control instruction is to adjust the target seat, and the control system corresponding to the first control instruction is the seat control system; the method further includes: Obtain a second image of the target seat collected by the image acquisition device at a second time, where the second image includes an image of the vehicle occupant located on the target seat, and the second time is the time when the first control instruction is determined; Identify the second image to obtain the physical state of the vehicle occupant located on the target seat; In the case where the physical state of the vehicle occupant located on the target seat is a pregnancy state, determine first adjustment information according to the pregnancy state; The controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction includes: Controlling the seat control system to adjust the target seat according to the first adjustment information.

9. The method according to any one of claims 1 to 3, characterized in that, After the controlling the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction, the method further includes: Receiving control information returned by the control system corresponding to the first control instruction, where the control information is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction; Generating a second gesture action according to the control information, where the second gesture action is used to indicate that the control system corresponding to the first control instruction has executed the control operation corresponding to the first control instruction; Displaying the second gesture action.

10. A vehicle control device, characterized in that, The device includes: A determination module, configured to determine the operating state of the vehicle and the remaining battery percentage of the vehicle in response to an activation instruction for the voice assistant function equipped in the vehicle; A control module, configured to control the image acquisition device installed in the vehicle to be turned on when the operating state of the vehicle meets the operating requirements and the remaining battery percentage of the vehicle is greater than a target threshold; An acquisition module, configured to acquire a first video collected by the image acquisition device in a first time period, where the first video includes a first gesture action of the vehicle occupant, and the start time of the first time period is the time when the image acquisition device is turned on; The determination module is further configured to determine a first control instruction corresponding to the first gesture action; The control module is further configured to control the control system corresponding to the first control instruction to execute the control operation corresponding to the first control instruction.

Images